You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2018/01/19 06:00:30 UTC
Build failed in Jenkins: beam_PerformanceTests_AvroIOIT #33
See <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/33/display/redirect?page=changes>
Changes:
[ankurgoenka] Fixing Work Progress reporting
[chamikara] [BEAM-3496] Add missing dependencies in hadoopInputFormats build.gradle
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam8 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 0d918b7cab8c4ccb2b5e050501327912161d40a7 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 0d918b7cab8c4ccb2b5e050501327912161d40a7
Commit message: "[BEAM-3496] Add missing dependencies in hadoopInputFormats build.gradle (#4442)"
> git rev-list ca63b05b61b05fd887caf7a2225f6b0c4d55b9b6 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins8042885640956254520.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins911499066855095814.sh
+ rm -rf .env
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins2225883846708953504.sh
+ virtualenv .env --system-site-packages
New python executable in .env/bin/python
Installing setuptools, pip...done.
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins7506340316976642701.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins7395237486384268172.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied (use --upgrade to upgrade): absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied (use --upgrade to upgrade): jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied (use --upgrade to upgrade): setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Installing extra requirements: 'windows'
Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied (use --upgrade to upgrade): futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied (use --upgrade to upgrade): pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied (use --upgrade to upgrade): numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied (use --upgrade to upgrade): functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied (use --upgrade to upgrade): contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied (use --upgrade to upgrade): pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Cleaning up...
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins8863390195157727463.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python>
Running setup.py (path:<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python/setup.py)> egg_info for package from file://<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python>
<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
no previously-included directories found matching 'doc/.build'
Installed <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python/nose-1.3.7-py2.7.egg>
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Installing extra requirements: 'gcp,test'
Requirement already satisfied (use --upgrade to upgrade): avro>=1.8.1,<2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): crcmod>=1.7,<2.0 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): grpcio>=1.0,<2 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): httplib2>=0.8,<0.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): mock>=1.0.1,<3.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): oauth2client>=2.0.1,<5 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): protobuf>=3.5.0.post1,<4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyyaml>=3.12,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyvcf>=0.6.8,<0.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): six>=1.9,<1.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): typing>=3.6.0,<3.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): futures>=3.1.1,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): hdfs3>=0.3.0,<0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyhamcrest>=1.9,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-apitools>=0.5.18,<=0.5.20 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio>=1.0,<2->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<5->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<5->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<5->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googleapis-common-protos>=1.5.2,<2.0dev in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Running setup.py develop for apache-beam
<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Creating <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/.env/lib/python2.7/site-packages/apache-beam.egg-link> (link to .)
Adding apache-beam 2.3.0.dev to easy-install.pth file
Installed <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python>
Successfully installed apache-beam
Cleaning up...
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins4690917745129654850.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.avroioit_pkb_results --official=true --benchmarks=beam_integration_benchmark --beam_it_timeout=1200 --beam_it_profile=io-it --beam_prebuilt=true --beam_sdk=java --beam_it_module=sdks/java/io/file-based-io-tests --beam_it_class=org.apache.beam.sdk.io.avro.AvroIOIT '--beam_it_options=[--project=apache-beam-testing,--tempRoot=gs://temp-storage-for-perf-tests,--numberOfRecords=1000000,--filenamePrefix=gs://temp-storage-for-perf-tests/beam_PerformanceTests_AvroIOIT/33/]' '--beam_extra_mvn_properties=[filesystem=gcs]'
Traceback (most recent call last):
File "PerfKitBenchmarker/pkb.py", line 21, in <module>
sys.exit(Main())
File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 869, in Main
SetUpPKB()
File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 740, in SetUpPKB
vm_util.GenTempDir()
File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/PerfKitBenchmarker/perfkitbenchmarker/vm_util.py",> line 121, in GenTempDir
temp_dir.CreateTemporaryDirectories()
File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/PerfKitBenchmarker/perfkitbenchmarker/temp_dir.py",> line 62, in CreateTemporaryDirectories
os.makedirs(path)
File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/.env/lib/python2.7/os.py",> line 157, in makedirs
mkdir(name, mode)
OSError: [Errno 13] Permission denied: '/tmp/perfkitbenchmarker/runs/d1b271fa'
Build step 'Execute shell' marked build as failure
Jenkins build is back to normal : beam_PerformanceTests_AvroIOIT
#47
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/47/display/redirect?page=changes>
Build failed in Jenkins: beam_PerformanceTests_AvroIOIT #46
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/46/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam8 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 3c4a2001e5364657f6596bfc338f792e4797712d (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 3c4a2001e5364657f6596bfc338f792e4797712d
Commit message: "[BEAM-3502] Remove usage of proto.Builder.clone() in DatastoreIO (#4449)"
> git rev-list 3c4a2001e5364657f6596bfc338f792e4797712d # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins9135049724167558196.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins7596450057394805521.sh
+ rm -rf .env
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins2028857789707532207.sh
+ virtualenv .env --system-site-packages
New python executable in .env/bin/python
Installing setuptools, pip...done.
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins2840442585143084409.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins855866899092555918.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied (use --upgrade to upgrade): absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied (use --upgrade to upgrade): jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied (use --upgrade to upgrade): setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Installing extra requirements: 'windows'
Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied (use --upgrade to upgrade): futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied (use --upgrade to upgrade): pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied (use --upgrade to upgrade): numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied (use --upgrade to upgrade): functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied (use --upgrade to upgrade): contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied (use --upgrade to upgrade): pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Cleaning up...
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins7653346617264956562.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python>
Running setup.py (path:<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python/setup.py)> egg_info for package from file://<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python>
<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
no previously-included directories found matching 'doc/.build'
Installed <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python/nose-1.3.7-py2.7.egg>
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Installing extra requirements: 'gcp,test'
Requirement already satisfied (use --upgrade to upgrade): avro>=1.8.1,<2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): crcmod>=1.7,<2.0 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): grpcio>=1.0,<2 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): httplib2>=0.8,<0.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): mock>=1.0.1,<3.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): oauth2client>=2.0.1,<5 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): protobuf>=3.5.0.post1,<4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyyaml>=3.12,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyvcf>=0.6.8,<0.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): six>=1.9,<1.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): typing>=3.6.0,<3.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): futures>=3.1.1,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): hdfs3>=0.3.0,<0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyhamcrest>=1.9,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-apitools>=0.5.18,<=0.5.20 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio>=1.0,<2->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<5->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<5->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<5->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googleapis-common-protos>=1.5.2,<2.0dev in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Running setup.py develop for apache-beam
<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Creating <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/.env/lib/python2.7/site-packages/apache-beam.egg-link> (link to .)
Adding apache-beam 2.3.0.dev to easy-install.pth file
Installed <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python>
Successfully installed apache-beam
Cleaning up...
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins676988370668813201.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.avroioit_pkb_results --official=true --benchmarks=beam_integration_benchmark --beam_it_timeout=1200 --beam_it_profile=io-it --beam_prebuilt=true --beam_sdk=java --beam_it_module=sdks/java/io/file-based-io-tests --beam_it_class=org.apache.beam.sdk.io.avro.AvroIOIT '--beam_it_options=[--project=apache-beam-testing,--tempRoot=gs://temp-storage-for-perf-tests,--numberOfRecords=1000000,--filenamePrefix=gs://temp-storage-for-perf-tests/beam_PerformanceTests_AvroIOIT/46/]' '--beam_extra_mvn_properties=[filesystem=gcs]'
Traceback (most recent call last):
File "PerfKitBenchmarker/pkb.py", line 21, in <module>
sys.exit(Main())
File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 869, in Main
SetUpPKB()
File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 740, in SetUpPKB
vm_util.GenTempDir()
File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/PerfKitBenchmarker/perfkitbenchmarker/vm_util.py",> line 121, in GenTempDir
temp_dir.CreateTemporaryDirectories()
File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/PerfKitBenchmarker/perfkitbenchmarker/temp_dir.py",> line 62, in CreateTemporaryDirectories
os.makedirs(path)
File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/.env/lib/python2.7/os.py",> line 157, in makedirs
mkdir(name, mode)
OSError: [Errno 13] Permission denied: '/tmp/perfkitbenchmarker/runs/4d35fbe6'
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PerformanceTests_AvroIOIT #45
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/45/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam8 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 3c4a2001e5364657f6596bfc338f792e4797712d (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 3c4a2001e5364657f6596bfc338f792e4797712d
Commit message: "[BEAM-3502] Remove usage of proto.Builder.clone() in DatastoreIO (#4449)"
> git rev-list 3c4a2001e5364657f6596bfc338f792e4797712d # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins357490442207095606.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins4349362379996686765.sh
+ rm -rf .env
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins2385934952559505203.sh
+ virtualenv .env --system-site-packages
New python executable in .env/bin/python
Installing setuptools, pip...done.
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins6120931350354142859.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins1387844950272539976.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied (use --upgrade to upgrade): absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied (use --upgrade to upgrade): jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied (use --upgrade to upgrade): setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Installing extra requirements: 'windows'
Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied (use --upgrade to upgrade): futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied (use --upgrade to upgrade): pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied (use --upgrade to upgrade): numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied (use --upgrade to upgrade): functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied (use --upgrade to upgrade): contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied (use --upgrade to upgrade): pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Cleaning up...
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins4265787081737333755.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python>
Running setup.py (path:<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python/setup.py)> egg_info for package from file://<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python>
<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
no previously-included directories found matching 'doc/.build'
Installed <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python/nose-1.3.7-py2.7.egg>
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Installing extra requirements: 'gcp,test'
Requirement already satisfied (use --upgrade to upgrade): avro>=1.8.1,<2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): crcmod>=1.7,<2.0 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): grpcio>=1.0,<2 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): httplib2>=0.8,<0.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): mock>=1.0.1,<3.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): oauth2client>=2.0.1,<5 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): protobuf>=3.5.0.post1,<4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyyaml>=3.12,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyvcf>=0.6.8,<0.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): six>=1.9,<1.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): typing>=3.6.0,<3.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): futures>=3.1.1,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): hdfs3>=0.3.0,<0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyhamcrest>=1.9,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-apitools>=0.5.18,<=0.5.20 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio>=1.0,<2->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<5->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<5->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<5->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googleapis-common-protos>=1.5.2,<2.0dev in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Running setup.py develop for apache-beam
<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Creating <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/.env/lib/python2.7/site-packages/apache-beam.egg-link> (link to .)
Adding apache-beam 2.3.0.dev to easy-install.pth file
Installed <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python>
Successfully installed apache-beam
Cleaning up...
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins562831556947008598.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.avroioit_pkb_results --official=true --benchmarks=beam_integration_benchmark --beam_it_timeout=1200 --beam_it_profile=io-it --beam_prebuilt=true --beam_sdk=java --beam_it_module=sdks/java/io/file-based-io-tests --beam_it_class=org.apache.beam.sdk.io.avro.AvroIOIT '--beam_it_options=[--project=apache-beam-testing,--tempRoot=gs://temp-storage-for-perf-tests,--numberOfRecords=1000000,--filenamePrefix=gs://temp-storage-for-perf-tests/beam_PerformanceTests_AvroIOIT/45/]' '--beam_extra_mvn_properties=[filesystem=gcs]'
Traceback (most recent call last):
File "PerfKitBenchmarker/pkb.py", line 21, in <module>
sys.exit(Main())
File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 869, in Main
SetUpPKB()
File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 740, in SetUpPKB
vm_util.GenTempDir()
File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/PerfKitBenchmarker/perfkitbenchmarker/vm_util.py",> line 121, in GenTempDir
temp_dir.CreateTemporaryDirectories()
File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/PerfKitBenchmarker/perfkitbenchmarker/temp_dir.py",> line 62, in CreateTemporaryDirectories
os.makedirs(path)
File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/.env/lib/python2.7/os.py",> line 157, in makedirs
mkdir(name, mode)
OSError: [Errno 13] Permission denied: '/tmp/perfkitbenchmarker/runs/00080a64'
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PerformanceTests_AvroIOIT #44
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/44/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam8 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 3c4a2001e5364657f6596bfc338f792e4797712d (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 3c4a2001e5364657f6596bfc338f792e4797712d
Commit message: "[BEAM-3502] Remove usage of proto.Builder.clone() in DatastoreIO (#4449)"
> git rev-list 3c4a2001e5364657f6596bfc338f792e4797712d # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins6965714572579721099.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins7418538549736931532.sh
+ rm -rf .env
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins1444548835043384473.sh
+ virtualenv .env --system-site-packages
New python executable in .env/bin/python
Installing setuptools, pip...done.
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins774536451821231872.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins4776679972954467050.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied (use --upgrade to upgrade): absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied (use --upgrade to upgrade): jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied (use --upgrade to upgrade): setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Installing extra requirements: 'windows'
Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied (use --upgrade to upgrade): futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied (use --upgrade to upgrade): pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied (use --upgrade to upgrade): numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied (use --upgrade to upgrade): functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied (use --upgrade to upgrade): contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied (use --upgrade to upgrade): pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Cleaning up...
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins8002296357415828782.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python>
Running setup.py (path:<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python/setup.py)> egg_info for package from file://<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python>
<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
no previously-included directories found matching 'doc/.build'
Installed <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python/nose-1.3.7-py2.7.egg>
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Installing extra requirements: 'gcp,test'
Requirement already satisfied (use --upgrade to upgrade): avro>=1.8.1,<2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): crcmod>=1.7,<2.0 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): grpcio>=1.0,<2 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): httplib2>=0.8,<0.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): mock>=1.0.1,<3.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): oauth2client>=2.0.1,<5 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): protobuf>=3.5.0.post1,<4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyyaml>=3.12,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyvcf>=0.6.8,<0.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): six>=1.9,<1.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): typing>=3.6.0,<3.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): futures>=3.1.1,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): hdfs3>=0.3.0,<0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyhamcrest>=1.9,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-apitools>=0.5.18,<=0.5.20 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio>=1.0,<2->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<5->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<5->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<5->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googleapis-common-protos>=1.5.2,<2.0dev in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Running setup.py develop for apache-beam
<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Creating <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/.env/lib/python2.7/site-packages/apache-beam.egg-link> (link to .)
Adding apache-beam 2.3.0.dev to easy-install.pth file
Installed <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python>
Successfully installed apache-beam
Cleaning up...
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins7180062137569210471.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.avroioit_pkb_results --official=true --benchmarks=beam_integration_benchmark --beam_it_timeout=1200 --beam_it_profile=io-it --beam_prebuilt=true --beam_sdk=java --beam_it_module=sdks/java/io/file-based-io-tests --beam_it_class=org.apache.beam.sdk.io.avro.AvroIOIT '--beam_it_options=[--project=apache-beam-testing,--tempRoot=gs://temp-storage-for-perf-tests,--numberOfRecords=1000000,--filenamePrefix=gs://temp-storage-for-perf-tests/beam_PerformanceTests_AvroIOIT/44/]' '--beam_extra_mvn_properties=[filesystem=gcs]'
Traceback (most recent call last):
File "PerfKitBenchmarker/pkb.py", line 21, in <module>
sys.exit(Main())
File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 869, in Main
SetUpPKB()
File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 740, in SetUpPKB
vm_util.GenTempDir()
File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/PerfKitBenchmarker/perfkitbenchmarker/vm_util.py",> line 121, in GenTempDir
temp_dir.CreateTemporaryDirectories()
File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/PerfKitBenchmarker/perfkitbenchmarker/temp_dir.py",> line 62, in CreateTemporaryDirectories
os.makedirs(path)
File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/.env/lib/python2.7/os.py",> line 157, in makedirs
mkdir(name, mode)
OSError: [Errno 13] Permission denied: '/tmp/perfkitbenchmarker/runs/0f083163'
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PerformanceTests_AvroIOIT #43
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/43/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam8 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 3c4a2001e5364657f6596bfc338f792e4797712d (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 3c4a2001e5364657f6596bfc338f792e4797712d
Commit message: "[BEAM-3502] Remove usage of proto.Builder.clone() in DatastoreIO (#4449)"
> git rev-list 3c4a2001e5364657f6596bfc338f792e4797712d # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins4701663013120487802.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins7934942133802653385.sh
+ rm -rf .env
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins4667939188970287044.sh
+ virtualenv .env --system-site-packages
New python executable in .env/bin/python
Installing setuptools, pip...done.
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins2268728564484886692.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins4164942157004819233.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied (use --upgrade to upgrade): absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied (use --upgrade to upgrade): jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied (use --upgrade to upgrade): setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Installing extra requirements: 'windows'
Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied (use --upgrade to upgrade): futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied (use --upgrade to upgrade): pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied (use --upgrade to upgrade): numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied (use --upgrade to upgrade): functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied (use --upgrade to upgrade): contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied (use --upgrade to upgrade): pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Cleaning up...
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins5265888389102984807.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python>
Running setup.py (path:<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python/setup.py)> egg_info for package from file://<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python>
<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
no previously-included directories found matching 'doc/.build'
Installed <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python/nose-1.3.7-py2.7.egg>
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Installing extra requirements: 'gcp,test'
Requirement already satisfied (use --upgrade to upgrade): avro>=1.8.1,<2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): crcmod>=1.7,<2.0 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): grpcio>=1.0,<2 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): httplib2>=0.8,<0.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): mock>=1.0.1,<3.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): oauth2client>=2.0.1,<5 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): protobuf>=3.5.0.post1,<4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyyaml>=3.12,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyvcf>=0.6.8,<0.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): six>=1.9,<1.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): typing>=3.6.0,<3.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): futures>=3.1.1,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): hdfs3>=0.3.0,<0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyhamcrest>=1.9,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-apitools>=0.5.18,<=0.5.20 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio>=1.0,<2->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<5->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<5->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<5->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googleapis-common-protos>=1.5.2,<2.0dev in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Running setup.py develop for apache-beam
<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Creating <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/.env/lib/python2.7/site-packages/apache-beam.egg-link> (link to .)
Adding apache-beam 2.3.0.dev to easy-install.pth file
Installed <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python>
Successfully installed apache-beam
Cleaning up...
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins447195626570033420.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.avroioit_pkb_results --official=true --benchmarks=beam_integration_benchmark --beam_it_timeout=1200 --beam_it_profile=io-it --beam_prebuilt=true --beam_sdk=java --beam_it_module=sdks/java/io/file-based-io-tests --beam_it_class=org.apache.beam.sdk.io.avro.AvroIOIT '--beam_it_options=[--project=apache-beam-testing,--tempRoot=gs://temp-storage-for-perf-tests,--numberOfRecords=1000000,--filenamePrefix=gs://temp-storage-for-perf-tests/beam_PerformanceTests_AvroIOIT/43/]' '--beam_extra_mvn_properties=[filesystem=gcs]'
Traceback (most recent call last):
File "PerfKitBenchmarker/pkb.py", line 21, in <module>
sys.exit(Main())
File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 869, in Main
SetUpPKB()
File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 740, in SetUpPKB
vm_util.GenTempDir()
File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/PerfKitBenchmarker/perfkitbenchmarker/vm_util.py",> line 121, in GenTempDir
temp_dir.CreateTemporaryDirectories()
File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/PerfKitBenchmarker/perfkitbenchmarker/temp_dir.py",> line 62, in CreateTemporaryDirectories
os.makedirs(path)
File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/.env/lib/python2.7/os.py",> line 157, in makedirs
mkdir(name, mode)
OSError: [Errno 13] Permission denied: '/tmp/perfkitbenchmarker/runs/ace19a54'
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PerformanceTests_AvroIOIT #42
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/42/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam8 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 3c4a2001e5364657f6596bfc338f792e4797712d (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 3c4a2001e5364657f6596bfc338f792e4797712d
Commit message: "[BEAM-3502] Remove usage of proto.Builder.clone() in DatastoreIO (#4449)"
> git rev-list 3c4a2001e5364657f6596bfc338f792e4797712d # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins3769133777076971665.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins8523168941582436981.sh
+ rm -rf .env
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins2143913141932287635.sh
+ virtualenv .env --system-site-packages
New python executable in .env/bin/python
Installing setuptools, pip...done.
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins4458436098546739707.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins2200451100258434087.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied (use --upgrade to upgrade): absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied (use --upgrade to upgrade): jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied (use --upgrade to upgrade): setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Installing extra requirements: 'windows'
Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied (use --upgrade to upgrade): futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied (use --upgrade to upgrade): pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied (use --upgrade to upgrade): numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied (use --upgrade to upgrade): functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied (use --upgrade to upgrade): contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied (use --upgrade to upgrade): pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Cleaning up...
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins8098738135991144376.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python>
Running setup.py (path:<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python/setup.py)> egg_info for package from file://<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python>
<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
no previously-included directories found matching 'doc/.build'
Installed <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python/nose-1.3.7-py2.7.egg>
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Installing extra requirements: 'gcp,test'
Requirement already satisfied (use --upgrade to upgrade): avro>=1.8.1,<2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): crcmod>=1.7,<2.0 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): grpcio>=1.0,<2 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): httplib2>=0.8,<0.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): mock>=1.0.1,<3.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): oauth2client>=2.0.1,<5 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): protobuf>=3.5.0.post1,<4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyyaml>=3.12,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyvcf>=0.6.8,<0.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): six>=1.9,<1.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): typing>=3.6.0,<3.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): futures>=3.1.1,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): hdfs3>=0.3.0,<0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyhamcrest>=1.9,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-apitools>=0.5.18,<=0.5.20 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio>=1.0,<2->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<5->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<5->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<5->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googleapis-common-protos>=1.5.2,<2.0dev in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Running setup.py develop for apache-beam
<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Creating <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/.env/lib/python2.7/site-packages/apache-beam.egg-link> (link to .)
Adding apache-beam 2.3.0.dev to easy-install.pth file
Installed <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python>
Successfully installed apache-beam
Cleaning up...
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins6907561620157415027.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.avroioit_pkb_results --official=true --benchmarks=beam_integration_benchmark --beam_it_timeout=1200 --beam_it_profile=io-it --beam_prebuilt=true --beam_sdk=java --beam_it_module=sdks/java/io/file-based-io-tests --beam_it_class=org.apache.beam.sdk.io.avro.AvroIOIT '--beam_it_options=[--project=apache-beam-testing,--tempRoot=gs://temp-storage-for-perf-tests,--numberOfRecords=1000000,--filenamePrefix=gs://temp-storage-for-perf-tests/beam_PerformanceTests_AvroIOIT/42/]' '--beam_extra_mvn_properties=[filesystem=gcs]'
Traceback (most recent call last):
File "PerfKitBenchmarker/pkb.py", line 21, in <module>
sys.exit(Main())
File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 869, in Main
SetUpPKB()
File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 740, in SetUpPKB
vm_util.GenTempDir()
File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/PerfKitBenchmarker/perfkitbenchmarker/vm_util.py",> line 121, in GenTempDir
temp_dir.CreateTemporaryDirectories()
File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/PerfKitBenchmarker/perfkitbenchmarker/temp_dir.py",> line 62, in CreateTemporaryDirectories
os.makedirs(path)
File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/.env/lib/python2.7/os.py",> line 157, in makedirs
mkdir(name, mode)
OSError: [Errno 13] Permission denied: '/tmp/perfkitbenchmarker/runs/8823a828'
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PerformanceTests_AvroIOIT #41
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/41/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam8 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 3c4a2001e5364657f6596bfc338f792e4797712d (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 3c4a2001e5364657f6596bfc338f792e4797712d
Commit message: "[BEAM-3502] Remove usage of proto.Builder.clone() in DatastoreIO (#4449)"
> git rev-list 3c4a2001e5364657f6596bfc338f792e4797712d # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins5367854165822058453.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins1290272457676971663.sh
+ rm -rf .env
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins991404934964353636.sh
+ virtualenv .env --system-site-packages
New python executable in .env/bin/python
Installing setuptools, pip...done.
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins8538682930712569653.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins4494514684553852467.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied (use --upgrade to upgrade): absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied (use --upgrade to upgrade): jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied (use --upgrade to upgrade): setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Installing extra requirements: 'windows'
Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied (use --upgrade to upgrade): futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied (use --upgrade to upgrade): pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied (use --upgrade to upgrade): numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied (use --upgrade to upgrade): functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied (use --upgrade to upgrade): contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied (use --upgrade to upgrade): pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Cleaning up...
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins4464936328386647318.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python>
Running setup.py (path:<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python/setup.py)> egg_info for package from file://<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python>
<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
no previously-included directories found matching 'doc/.build'
Installed <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python/nose-1.3.7-py2.7.egg>
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Installing extra requirements: 'gcp,test'
Requirement already satisfied (use --upgrade to upgrade): avro>=1.8.1,<2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): crcmod>=1.7,<2.0 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): grpcio>=1.0,<2 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): httplib2>=0.8,<0.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): mock>=1.0.1,<3.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): oauth2client>=2.0.1,<5 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): protobuf>=3.5.0.post1,<4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyyaml>=3.12,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyvcf>=0.6.8,<0.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): six>=1.9,<1.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): typing>=3.6.0,<3.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): futures>=3.1.1,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): hdfs3>=0.3.0,<0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyhamcrest>=1.9,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-apitools>=0.5.18,<=0.5.20 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio>=1.0,<2->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<5->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<5->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<5->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googleapis-common-protos>=1.5.2,<2.0dev in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Running setup.py develop for apache-beam
<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Creating <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/.env/lib/python2.7/site-packages/apache-beam.egg-link> (link to .)
Adding apache-beam 2.3.0.dev to easy-install.pth file
Installed <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python>
Successfully installed apache-beam
Cleaning up...
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins430846820868087999.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.avroioit_pkb_results --official=true --benchmarks=beam_integration_benchmark --beam_it_timeout=1200 --beam_it_profile=io-it --beam_prebuilt=true --beam_sdk=java --beam_it_module=sdks/java/io/file-based-io-tests --beam_it_class=org.apache.beam.sdk.io.avro.AvroIOIT '--beam_it_options=[--project=apache-beam-testing,--tempRoot=gs://temp-storage-for-perf-tests,--numberOfRecords=1000000,--filenamePrefix=gs://temp-storage-for-perf-tests/beam_PerformanceTests_AvroIOIT/41/]' '--beam_extra_mvn_properties=[filesystem=gcs]'
Traceback (most recent call last):
File "PerfKitBenchmarker/pkb.py", line 21, in <module>
sys.exit(Main())
File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 869, in Main
SetUpPKB()
File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 740, in SetUpPKB
vm_util.GenTempDir()
File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/PerfKitBenchmarker/perfkitbenchmarker/vm_util.py",> line 121, in GenTempDir
temp_dir.CreateTemporaryDirectories()
File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/PerfKitBenchmarker/perfkitbenchmarker/temp_dir.py",> line 62, in CreateTemporaryDirectories
os.makedirs(path)
File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/.env/lib/python2.7/os.py",> line 157, in makedirs
mkdir(name, mode)
OSError: [Errno 13] Permission denied: '/tmp/perfkitbenchmarker/runs/6efb11dd'
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PerformanceTests_AvroIOIT #40
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/40/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam8 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 3c4a2001e5364657f6596bfc338f792e4797712d (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 3c4a2001e5364657f6596bfc338f792e4797712d
Commit message: "[BEAM-3502] Remove usage of proto.Builder.clone() in DatastoreIO (#4449)"
> git rev-list 3c4a2001e5364657f6596bfc338f792e4797712d # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins2727430600843507985.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins3784490624215226880.sh
+ rm -rf .env
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins6888339216528000219.sh
+ virtualenv .env --system-site-packages
New python executable in .env/bin/python
Installing setuptools, pip...done.
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins3239243871347137309.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins7221266651131231918.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied (use --upgrade to upgrade): absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied (use --upgrade to upgrade): jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied (use --upgrade to upgrade): setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Installing extra requirements: 'windows'
Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied (use --upgrade to upgrade): futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied (use --upgrade to upgrade): pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied (use --upgrade to upgrade): numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied (use --upgrade to upgrade): functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied (use --upgrade to upgrade): contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied (use --upgrade to upgrade): pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Cleaning up...
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins2292621547624698849.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python>
Running setup.py (path:<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python/setup.py)> egg_info for package from file://<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python>
<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
no previously-included directories found matching 'doc/.build'
Installed <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python/nose-1.3.7-py2.7.egg>
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Installing extra requirements: 'gcp,test'
Requirement already satisfied (use --upgrade to upgrade): avro>=1.8.1,<2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): crcmod>=1.7,<2.0 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): grpcio>=1.0,<2 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): httplib2>=0.8,<0.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): mock>=1.0.1,<3.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): oauth2client>=2.0.1,<5 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): protobuf>=3.5.0.post1,<4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyyaml>=3.12,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyvcf>=0.6.8,<0.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): six>=1.9,<1.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): typing>=3.6.0,<3.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): futures>=3.1.1,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): hdfs3>=0.3.0,<0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyhamcrest>=1.9,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-apitools>=0.5.18,<=0.5.20 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio>=1.0,<2->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<5->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<5->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<5->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googleapis-common-protos>=1.5.2,<2.0dev in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Running setup.py develop for apache-beam
<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Creating <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/.env/lib/python2.7/site-packages/apache-beam.egg-link> (link to .)
Adding apache-beam 2.3.0.dev to easy-install.pth file
Installed <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python>
Successfully installed apache-beam
Cleaning up...
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins6569844042033121963.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.avroioit_pkb_results --official=true --benchmarks=beam_integration_benchmark --beam_it_timeout=1200 --beam_it_profile=io-it --beam_prebuilt=true --beam_sdk=java --beam_it_module=sdks/java/io/file-based-io-tests --beam_it_class=org.apache.beam.sdk.io.avro.AvroIOIT '--beam_it_options=[--project=apache-beam-testing,--tempRoot=gs://temp-storage-for-perf-tests,--numberOfRecords=1000000,--filenamePrefix=gs://temp-storage-for-perf-tests/beam_PerformanceTests_AvroIOIT/40/]' '--beam_extra_mvn_properties=[filesystem=gcs]'
Traceback (most recent call last):
File "PerfKitBenchmarker/pkb.py", line 21, in <module>
sys.exit(Main())
File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 869, in Main
SetUpPKB()
File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 740, in SetUpPKB
vm_util.GenTempDir()
File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/PerfKitBenchmarker/perfkitbenchmarker/vm_util.py",> line 121, in GenTempDir
temp_dir.CreateTemporaryDirectories()
File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/PerfKitBenchmarker/perfkitbenchmarker/temp_dir.py",> line 62, in CreateTemporaryDirectories
os.makedirs(path)
File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/.env/lib/python2.7/os.py",> line 157, in makedirs
mkdir(name, mode)
OSError: [Errno 13] Permission denied: '/tmp/perfkitbenchmarker/runs/b8bf15a3'
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PerformanceTests_AvroIOIT #39
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/39/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam8 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 3c4a2001e5364657f6596bfc338f792e4797712d (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 3c4a2001e5364657f6596bfc338f792e4797712d
Commit message: "[BEAM-3502] Remove usage of proto.Builder.clone() in DatastoreIO (#4449)"
> git rev-list 3c4a2001e5364657f6596bfc338f792e4797712d # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins6134997001046648630.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins8016783155566289134.sh
+ rm -rf .env
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins393882001144288044.sh
+ virtualenv .env --system-site-packages
New python executable in .env/bin/python
Installing setuptools, pip...done.
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins1977128308172491108.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins8604868871477739839.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied (use --upgrade to upgrade): absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied (use --upgrade to upgrade): jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied (use --upgrade to upgrade): setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Installing extra requirements: 'windows'
Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied (use --upgrade to upgrade): futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied (use --upgrade to upgrade): pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied (use --upgrade to upgrade): numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied (use --upgrade to upgrade): functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied (use --upgrade to upgrade): contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied (use --upgrade to upgrade): pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Cleaning up...
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins5814583399866376772.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python>
Running setup.py (path:<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python/setup.py)> egg_info for package from file://<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python>
<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
no previously-included directories found matching 'doc/.build'
Installed <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python/nose-1.3.7-py2.7.egg>
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Installing extra requirements: 'gcp,test'
Requirement already satisfied (use --upgrade to upgrade): avro>=1.8.1,<2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): crcmod>=1.7,<2.0 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): grpcio>=1.0,<2 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): httplib2>=0.8,<0.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): mock>=1.0.1,<3.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): oauth2client>=2.0.1,<5 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): protobuf>=3.5.0.post1,<4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyyaml>=3.12,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyvcf>=0.6.8,<0.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): six>=1.9,<1.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): typing>=3.6.0,<3.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): futures>=3.1.1,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): hdfs3>=0.3.0,<0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyhamcrest>=1.9,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-apitools>=0.5.18,<=0.5.20 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio>=1.0,<2->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<5->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<5->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<5->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googleapis-common-protos>=1.5.2,<2.0dev in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Running setup.py develop for apache-beam
<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Creating <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/.env/lib/python2.7/site-packages/apache-beam.egg-link> (link to .)
Adding apache-beam 2.3.0.dev to easy-install.pth file
Installed <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python>
Successfully installed apache-beam
Cleaning up...
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins8086742997057721359.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.avroioit_pkb_results --official=true --benchmarks=beam_integration_benchmark --beam_it_timeout=1200 --beam_it_profile=io-it --beam_prebuilt=true --beam_sdk=java --beam_it_module=sdks/java/io/file-based-io-tests --beam_it_class=org.apache.beam.sdk.io.avro.AvroIOIT '--beam_it_options=[--project=apache-beam-testing,--tempRoot=gs://temp-storage-for-perf-tests,--numberOfRecords=1000000,--filenamePrefix=gs://temp-storage-for-perf-tests/beam_PerformanceTests_AvroIOIT/39/]' '--beam_extra_mvn_properties=[filesystem=gcs]'
Traceback (most recent call last):
File "PerfKitBenchmarker/pkb.py", line 21, in <module>
sys.exit(Main())
File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 869, in Main
SetUpPKB()
File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 740, in SetUpPKB
vm_util.GenTempDir()
File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/PerfKitBenchmarker/perfkitbenchmarker/vm_util.py",> line 121, in GenTempDir
temp_dir.CreateTemporaryDirectories()
File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/PerfKitBenchmarker/perfkitbenchmarker/temp_dir.py",> line 62, in CreateTemporaryDirectories
os.makedirs(path)
File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/.env/lib/python2.7/os.py",> line 157, in makedirs
mkdir(name, mode)
OSError: [Errno 13] Permission denied: '/tmp/perfkitbenchmarker/runs/c167b418'
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PerformanceTests_AvroIOIT #38
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/38/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam8 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 3c4a2001e5364657f6596bfc338f792e4797712d (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 3c4a2001e5364657f6596bfc338f792e4797712d
Commit message: "[BEAM-3502] Remove usage of proto.Builder.clone() in DatastoreIO (#4449)"
> git rev-list 3c4a2001e5364657f6596bfc338f792e4797712d # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins4390522949829181717.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins8427867803332677883.sh
+ rm -rf .env
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins2288420318496816324.sh
+ virtualenv .env --system-site-packages
New python executable in .env/bin/python
Installing setuptools, pip...done.
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins8260782036789720362.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins6393107211565357935.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied (use --upgrade to upgrade): absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied (use --upgrade to upgrade): jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied (use --upgrade to upgrade): setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Installing extra requirements: 'windows'
Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied (use --upgrade to upgrade): futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied (use --upgrade to upgrade): pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied (use --upgrade to upgrade): numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied (use --upgrade to upgrade): functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied (use --upgrade to upgrade): contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied (use --upgrade to upgrade): pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Cleaning up...
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins3190321126619424162.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python>
Running setup.py (path:<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python/setup.py)> egg_info for package from file://<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python>
<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
no previously-included directories found matching 'doc/.build'
Installed <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python/nose-1.3.7-py2.7.egg>
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Installing extra requirements: 'gcp,test'
Requirement already satisfied (use --upgrade to upgrade): avro>=1.8.1,<2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): crcmod>=1.7,<2.0 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): grpcio>=1.0,<2 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): httplib2>=0.8,<0.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): mock>=1.0.1,<3.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): oauth2client>=2.0.1,<5 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): protobuf>=3.5.0.post1,<4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyyaml>=3.12,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyvcf>=0.6.8,<0.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): six>=1.9,<1.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): typing>=3.6.0,<3.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): futures>=3.1.1,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): hdfs3>=0.3.0,<0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyhamcrest>=1.9,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-apitools>=0.5.18,<=0.5.20 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio>=1.0,<2->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<5->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<5->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<5->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googleapis-common-protos>=1.5.2,<2.0dev in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Running setup.py develop for apache-beam
<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Creating <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/.env/lib/python2.7/site-packages/apache-beam.egg-link> (link to .)
Adding apache-beam 2.3.0.dev to easy-install.pth file
Installed <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python>
Successfully installed apache-beam
Cleaning up...
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins1953818423469776274.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.avroioit_pkb_results --official=true --benchmarks=beam_integration_benchmark --beam_it_timeout=1200 --beam_it_profile=io-it --beam_prebuilt=true --beam_sdk=java --beam_it_module=sdks/java/io/file-based-io-tests --beam_it_class=org.apache.beam.sdk.io.avro.AvroIOIT '--beam_it_options=[--project=apache-beam-testing,--tempRoot=gs://temp-storage-for-perf-tests,--numberOfRecords=1000000,--filenamePrefix=gs://temp-storage-for-perf-tests/beam_PerformanceTests_AvroIOIT/38/]' '--beam_extra_mvn_properties=[filesystem=gcs]'
Traceback (most recent call last):
File "PerfKitBenchmarker/pkb.py", line 21, in <module>
sys.exit(Main())
File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 869, in Main
SetUpPKB()
File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 740, in SetUpPKB
vm_util.GenTempDir()
File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/PerfKitBenchmarker/perfkitbenchmarker/vm_util.py",> line 121, in GenTempDir
temp_dir.CreateTemporaryDirectories()
File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/PerfKitBenchmarker/perfkitbenchmarker/temp_dir.py",> line 62, in CreateTemporaryDirectories
os.makedirs(path)
File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/.env/lib/python2.7/os.py",> line 157, in makedirs
mkdir(name, mode)
OSError: [Errno 13] Permission denied: '/tmp/perfkitbenchmarker/runs/db37f429'
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PerformanceTests_AvroIOIT #37
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/37/display/redirect?page=changes>
Changes:
[lcwik] [BEAM-3502] Remove usage of proto.Builder.clone() in DatastoreIO (#4449)
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam8 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 3c4a2001e5364657f6596bfc338f792e4797712d (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 3c4a2001e5364657f6596bfc338f792e4797712d
Commit message: "[BEAM-3502] Remove usage of proto.Builder.clone() in DatastoreIO (#4449)"
> git rev-list 898277297be7160900fdb48ed486a257a6bc92c0 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
java.util.concurrent.TimeoutException: Timeout waiting for task.
at com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:259)
at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:91)
at com.google.common.util.concurrent.ForwardingFuture.get(ForwardingFuture.java:69)
at com.atlassian.jira.rest.client.internal.async.DelegatingPromise.get(DelegatingPromise.java:107)
at hudson.plugins.jira.JiraRestService.getIssuesFromJqlSearch(JiraRestService.java:177)
at hudson.plugins.jira.JiraSession.getIssuesFromJqlSearch(JiraSession.java:135)
at io.jenkins.blueocean.service.embedded.jira.JiraSCMListener.onChangeLogParsed(JiraSCMListener.java:43)
at hudson.model.listeners.SCMListener.onChangeLogParsed(SCMListener.java:120)
at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:590)
Caused: java.io.IOException: Failed to parse changelog
at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:592)
at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
at hudson.model.Run.execute(Run.java:1724)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:421)
Retrying after 10 seconds
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 3c4a2001e5364657f6596bfc338f792e4797712d (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 3c4a2001e5364657f6596bfc338f792e4797712d
Commit message: "[BEAM-3502] Remove usage of proto.Builder.clone() in DatastoreIO (#4449)"
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
java.util.concurrent.TimeoutException: Timeout waiting for task.
at com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:259)
at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:91)
at com.google.common.util.concurrent.ForwardingFuture.get(ForwardingFuture.java:69)
at com.atlassian.jira.rest.client.internal.async.DelegatingPromise.get(DelegatingPromise.java:107)
at hudson.plugins.jira.JiraRestService.getIssuesFromJqlSearch(JiraRestService.java:177)
at hudson.plugins.jira.JiraSession.getIssuesFromJqlSearch(JiraSession.java:135)
at io.jenkins.blueocean.service.embedded.jira.JiraSCMListener.onChangeLogParsed(JiraSCMListener.java:43)
at hudson.model.listeners.SCMListener.onChangeLogParsed(SCMListener.java:120)
at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:590)
Caused: java.io.IOException: Failed to parse changelog
at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:592)
at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
at hudson.model.Run.execute(Run.java:1724)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:421)
Retrying after 10 seconds
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 3c4a2001e5364657f6596bfc338f792e4797712d (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 3c4a2001e5364657f6596bfc338f792e4797712d
Commit message: "[BEAM-3502] Remove usage of proto.Builder.clone() in DatastoreIO (#4449)"
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
java.util.concurrent.TimeoutException: Timeout waiting for task.
at com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:259)
at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:91)
at com.google.common.util.concurrent.ForwardingFuture.get(ForwardingFuture.java:69)
at com.atlassian.jira.rest.client.internal.async.DelegatingPromise.get(DelegatingPromise.java:107)
at hudson.plugins.jira.JiraRestService.getIssuesFromJqlSearch(JiraRestService.java:177)
at hudson.plugins.jira.JiraSession.getIssuesFromJqlSearch(JiraSession.java:135)
at io.jenkins.blueocean.service.embedded.jira.JiraSCMListener.onChangeLogParsed(JiraSCMListener.java:43)
at hudson.model.listeners.SCMListener.onChangeLogParsed(SCMListener.java:120)
at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:590)
Caused: java.io.IOException: Failed to parse changelog
at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:592)
at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
at hudson.model.Run.execute(Run.java:1724)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:421)
Build failed in Jenkins: beam_PerformanceTests_AvroIOIT #36
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/36/display/redirect?page=changes>
Changes:
[batbat] Moved floating point equality findbugs annotation from generic xml file
[melissapa] [BEAM-3351] Fix Javadoc formatting issues
[melissapa] Update Context references to links
[tgroh] Implement a GRPC Provision Service
[tgroh] Add InboundDataClient
[lcwik] Use platformThreadFactory for default thread pool.
[robertwb] [BEAM-3490] Wrap DistributionData in a DistributionResult for
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam1 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 898277297be7160900fdb48ed486a257a6bc92c0 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 898277297be7160900fdb48ed486a257a6bc92c0
Commit message: "Merge pull request #4450 [BEAM-3490] Wrap DistributionData in a DistributionResult"
> git rev-list a91f7ada2075b8d6adbbb7002a15f1078d8eae45 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins8434807541768946444.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins7847789217083601198.sh
+ rm -rf .env
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins5555218703169749260.sh
+ virtualenv .env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/.env/bin/python>
Installing setuptools, pip, wheel...done.
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins1976952698185606208.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins7141304261411922329.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins2008323951485349225.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: hdfs3<0.4.0,>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.3.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Not uninstalling apache-beam at /home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_TextIOIT/src/sdks/python, outside environment <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/.env>
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins7466615613756961476.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.avroioit_pkb_results --official=true --benchmarks=beam_integration_benchmark --beam_it_timeout=1200 --beam_it_profile=io-it --beam_prebuilt=true --beam_sdk=java --beam_it_module=sdks/java/io/file-based-io-tests --beam_it_class=org.apache.beam.sdk.io.avro.AvroIOIT '--beam_it_options=[--project=apache-beam-testing,--tempRoot=gs://temp-storage-for-perf-tests,--numberOfRecords=1000000,--filenamePrefix=gs://temp-storage-for-perf-tests/beam_PerformanceTests_AvroIOIT/36/]' '--beam_extra_mvn_properties=[filesystem=gcs]'
Traceback (most recent call last):
File "PerfKitBenchmarker/pkb.py", line 21, in <module>
sys.exit(Main())
File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 869, in Main
SetUpPKB()
File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 740, in SetUpPKB
vm_util.GenTempDir()
File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/PerfKitBenchmarker/perfkitbenchmarker/vm_util.py",> line 121, in GenTempDir
temp_dir.CreateTemporaryDirectories()
File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/PerfKitBenchmarker/perfkitbenchmarker/temp_dir.py",> line 62, in CreateTemporaryDirectories
os.makedirs(path)
File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/.env/lib/python2.7/os.py",> line 157, in makedirs
mkdir(name, mode)
OSError: [Errno 13] Permission denied: '/tmp/perfkitbenchmarker/runs/d94404ac'
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PerformanceTests_AvroIOIT #35
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/35/display/redirect?page=changes>
Changes:
[holdenkarau] Apply futurize on gen_protos
[holdenkarau] Run futurize regardless
[holdenkarau] Only futurize for sys version geq 3
[holdenkarau] Long form command line options
[iemejia] [BEAM-3275] Update KafkaIO to support Kafka version 1.0.0
[lcwik] Update generated version of Fn API code.
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam8 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision a91f7ada2075b8d6adbbb7002a15f1078d8eae45 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f a91f7ada2075b8d6adbbb7002a15f1078d8eae45
Commit message: "Merge pull request #4077 from holdenk/BEAM-3142-fix-proto-gen-for-py3"
> git rev-list a7f4e56cdd47fd7432cd70e9897fdc18d40eb74c # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins8814628915390540445.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins8422503740571621159.sh
+ rm -rf .env
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins8828470613230248638.sh
+ virtualenv .env --system-site-packages
New python executable in .env/bin/python
Installing setuptools, pip...done.
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins4172887151794349563.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins2871875509335430002.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied (use --upgrade to upgrade): absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied (use --upgrade to upgrade): jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied (use --upgrade to upgrade): setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Installing extra requirements: 'windows'
Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied (use --upgrade to upgrade): futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied (use --upgrade to upgrade): pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied (use --upgrade to upgrade): numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied (use --upgrade to upgrade): functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied (use --upgrade to upgrade): contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied (use --upgrade to upgrade): pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Cleaning up...
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins4151712454803104183.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python>
Running setup.py (path:<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python/setup.py)> egg_info for package from file://<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python>
<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
no previously-included directories found matching 'doc/.build'
Installed <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python/nose-1.3.7-py2.7.egg>
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Installing extra requirements: 'gcp,test'
Requirement already satisfied (use --upgrade to upgrade): avro>=1.8.1,<2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): crcmod>=1.7,<2.0 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): grpcio>=1.0,<2 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): httplib2>=0.8,<0.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): mock>=1.0.1,<3.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): oauth2client>=2.0.1,<5 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): protobuf>=3.5.0.post1,<4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyyaml>=3.12,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyvcf>=0.6.8,<0.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): six>=1.9,<1.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): typing>=3.6.0,<3.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): futures>=3.1.1,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): hdfs3>=0.3.0,<0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyhamcrest>=1.9,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-apitools>=0.5.18,<=0.5.20 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio>=1.0,<2->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<5->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<5->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<5->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googleapis-common-protos>=1.5.2,<2.0dev in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Running setup.py develop for apache-beam
<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Creating <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/.env/lib/python2.7/site-packages/apache-beam.egg-link> (link to .)
Adding apache-beam 2.3.0.dev to easy-install.pth file
Installed <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python>
Successfully installed apache-beam
Cleaning up...
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins6631201299072810202.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.avroioit_pkb_results --official=true --benchmarks=beam_integration_benchmark --beam_it_timeout=1200 --beam_it_profile=io-it --beam_prebuilt=true --beam_sdk=java --beam_it_module=sdks/java/io/file-based-io-tests --beam_it_class=org.apache.beam.sdk.io.avro.AvroIOIT '--beam_it_options=[--project=apache-beam-testing,--tempRoot=gs://temp-storage-for-perf-tests,--numberOfRecords=1000000,--filenamePrefix=gs://temp-storage-for-perf-tests/beam_PerformanceTests_AvroIOIT/35/]' '--beam_extra_mvn_properties=[filesystem=gcs]'
Traceback (most recent call last):
File "PerfKitBenchmarker/pkb.py", line 21, in <module>
sys.exit(Main())
File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 869, in Main
SetUpPKB()
File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 740, in SetUpPKB
vm_util.GenTempDir()
File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/PerfKitBenchmarker/perfkitbenchmarker/vm_util.py",> line 121, in GenTempDir
temp_dir.CreateTemporaryDirectories()
File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/PerfKitBenchmarker/perfkitbenchmarker/temp_dir.py",> line 62, in CreateTemporaryDirectories
os.makedirs(path)
File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/.env/lib/python2.7/os.py",> line 157, in makedirs
mkdir(name, mode)
OSError: [Errno 13] Permission denied: '/tmp/perfkitbenchmarker/runs/ae7e00f5'
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PerformanceTests_AvroIOIT #34
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/34/display/redirect?page=changes>
Changes:
[kedin] [Nexmark][SQL] Implement sql query 2
[kenn] Add unshaded configurations in the default profile
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam8 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision a7f4e56cdd47fd7432cd70e9897fdc18d40eb74c (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f a7f4e56cdd47fd7432cd70e9897fdc18d40eb74c
Commit message: "Merge pull request #4428: Add unshaded configurations in the default profile"
> git rev-list 0d918b7cab8c4ccb2b5e050501327912161d40a7 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins7116143803375161344.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins3770348884008471672.sh
+ rm -rf .env
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins5889018473144929141.sh
+ virtualenv .env --system-site-packages
New python executable in .env/bin/python
Installing setuptools, pip...done.
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins6861097173824180037.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins5104686547883668594.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied (use --upgrade to upgrade): absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied (use --upgrade to upgrade): jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied (use --upgrade to upgrade): setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Installing extra requirements: 'windows'
Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied (use --upgrade to upgrade): futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied (use --upgrade to upgrade): pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied (use --upgrade to upgrade): numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied (use --upgrade to upgrade): functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied (use --upgrade to upgrade): contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied (use --upgrade to upgrade): pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Cleaning up...
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins4625088125051866245.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python>
Running setup.py (path:<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python/setup.py)> egg_info for package from file://<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python>
<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
no previously-included directories found matching 'doc/.build'
Installed <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python/nose-1.3.7-py2.7.egg>
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Installing extra requirements: 'gcp,test'
Requirement already satisfied (use --upgrade to upgrade): avro>=1.8.1,<2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): crcmod>=1.7,<2.0 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): grpcio>=1.0,<2 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): httplib2>=0.8,<0.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): mock>=1.0.1,<3.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): oauth2client>=2.0.1,<5 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): protobuf>=3.5.0.post1,<4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyyaml>=3.12,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyvcf>=0.6.8,<0.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): six>=1.9,<1.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): typing>=3.6.0,<3.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): futures>=3.1.1,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): hdfs3>=0.3.0,<0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyhamcrest>=1.9,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-apitools>=0.5.18,<=0.5.20 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio>=1.0,<2->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<5->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<5->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<5->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googleapis-common-protos>=1.5.2,<2.0dev in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Running setup.py develop for apache-beam
<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Creating <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/.env/lib/python2.7/site-packages/apache-beam.egg-link> (link to .)
Adding apache-beam 2.3.0.dev to easy-install.pth file
Installed <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/src/sdks/python>
Successfully installed apache-beam
Cleaning up...
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins4218680657662160122.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.avroioit_pkb_results --official=true --benchmarks=beam_integration_benchmark --beam_it_timeout=1200 --beam_it_profile=io-it --beam_prebuilt=true --beam_sdk=java --beam_it_module=sdks/java/io/file-based-io-tests --beam_it_class=org.apache.beam.sdk.io.avro.AvroIOIT '--beam_it_options=[--project=apache-beam-testing,--tempRoot=gs://temp-storage-for-perf-tests,--numberOfRecords=1000000,--filenamePrefix=gs://temp-storage-for-perf-tests/beam_PerformanceTests_AvroIOIT/34/]' '--beam_extra_mvn_properties=[filesystem=gcs]'
Traceback (most recent call last):
File "PerfKitBenchmarker/pkb.py", line 21, in <module>
sys.exit(Main())
File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 869, in Main
SetUpPKB()
File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 740, in SetUpPKB
vm_util.GenTempDir()
File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/PerfKitBenchmarker/perfkitbenchmarker/vm_util.py",> line 121, in GenTempDir
temp_dir.CreateTemporaryDirectories()
File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/PerfKitBenchmarker/perfkitbenchmarker/temp_dir.py",> line 62, in CreateTemporaryDirectories
os.makedirs(path)
File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/.env/lib/python2.7/os.py",> line 157, in makedirs
mkdir(name, mode)
OSError: [Errno 13] Permission denied: '/tmp/perfkitbenchmarker/runs/a15f553a'
Build step 'Execute shell' marked build as failure