You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2017/07/21 01:00:58 UTC

Build failed in Jenkins: beam_PerformanceTests_Python #124

See <https://builds.apache.org/job/beam_PerformanceTests_Python/124/display/redirect?page=changes>

Changes:

[tgroh] Add maven support for invoking perfkit benchmarker to run IO ITs

[tgroh] io-it-suite-local independent of io-it-suite, k8s properties -> root pom

[kirpichov] Minor changes to AvroSource in preparation for refactoring

[kirpichov] Gets rid of opening Avro files in createForSubrangeOfFile codepath

------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
 > git --version # timeout=10
 > git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
 > git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 1d9160fa3337704b9fbdc423796925be78b0087e (origin/master)
Commit message: "This closes #3590: [BEAM-2628] Makes AvroSource not open files while splitting"
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 1d9160fa3337704b9fbdc423796925be78b0087e
 > git rev-list afeba3715c806b53115f8f7994eb7bc207c68932 # timeout=10
Cleaning workspace
 > git rev-parse --verify HEAD # timeout=10
Resetting working tree
 > git reset --hard # timeout=10
 > git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content 
SPARK_LOCAL_IP=127.0.0.1

[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6185997070931709586.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6964434050081114127.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6689424679235806941.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied (use --upgrade to upgrade): python-gflags==3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied (use --upgrade to upgrade): jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
  Installing extra requirements: 'windows'
Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied (use --upgrade to upgrade): futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied (use --upgrade to upgrade): pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied (use --upgrade to upgrade): numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied (use --upgrade to upgrade): functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied (use --upgrade to upgrade): contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins894325712106013578.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
  Running setup.py (path:<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/setup.py)> egg_info for package from file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
    <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
      _PIP_VERSION, REQUIRED_PIP_VERSION
    no previously-included directories found matching 'doc/.build'
    
    Installed <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/nose-1.3.7-py2.7.egg>
    
    warning: no files found matching 'README.md'
    warning: no files found matching 'NOTICE'
    warning: no files found matching 'LICENSE'
  Installing extra requirements: 'gcp,test'
Requirement already satisfied (use --upgrade to upgrade): avro>=1.8.1,<2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.2.0.dev)
Requirement already satisfied (use --upgrade to upgrade): crcmod>=1.7,<2.0 in /usr/lib/python2.7/dist-packages (from apache-beam==2.2.0.dev)
Requirement already satisfied (use --upgrade to upgrade): dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.2.0.dev)
Requirement already satisfied (use --upgrade to upgrade): grpcio>=1.0,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.2.0.dev)
Requirement already satisfied (use --upgrade to upgrade): httplib2>=0.8,<0.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.2.0.dev)
Requirement already satisfied (use --upgrade to upgrade): mock>=1.0.1,<3.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.2.0.dev)
Requirement already satisfied (use --upgrade to upgrade): oauth2client>=2.0.1,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.2.0.dev)
Requirement already satisfied (use --upgrade to upgrade): protobuf>=3.2.0,<=3.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.2.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyyaml>=3.12,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.2.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyhamcrest>=1.9,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.2.0.dev)
Requirement already satisfied (use --upgrade to upgrade): six>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.2.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-apitools>=0.5.10,<=0.5.11 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.2.0.dev)
Requirement already satisfied (use --upgrade to upgrade): proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.2.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.2.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.2.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.2.0.dev)
Requirement already satisfied (use --upgrade to upgrade): futures>=2.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio>=1.0,<2.0->apache-beam==2.2.0.dev)
Requirement already satisfied (use --upgrade to upgrade): enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio>=1.0,<2.0->apache-beam==2.2.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<4.0.0->apache-beam==2.2.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<4.0.0->apache-beam==2.2.0.dev)
Requirement already satisfied (use --upgrade to upgrade): rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<4.0.0->apache-beam==2.2.0.dev)
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/lib/python2.7/dist-packages (from protobuf>=3.2.0,<=3.3.0->apache-beam==2.2.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googleapis-common-protos>=1.5.2,<2.0dev in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4->apache-beam==2.2.0.dev)
Installing collected packages: apache-beam
  Running setup.py develop for apache-beam
    <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
      _PIP_VERSION, REQUIRED_PIP_VERSION
    
    warning: no files found matching 'README.md'
    warning: no files found matching 'NOTICE'
    warning: no files found matching 'LICENSE'
    Creating /home/jenkins/.local/lib/python2.7/site-packages/apache-beam.egg-link (link to .)
    apache-beam 2.2.0.dev is already the active version in easy-install.pth
    
    Installed <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Successfully installed apache-beam
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8780225048677237494.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
WARNING:root:File resource loader root perfkitbenchmarker/data/ycsb is not a directory.
2017-07-21 01:00:33,984 8336e237 MainThread INFO     Verbose logging to: /tmp/perfkitbenchmarker/runs/8336e237/pkb.log
2017-07-21 01:00:33,985 8336e237 MainThread INFO     PerfKitBenchmarker version: v1.12.0-74-gd26cc62
2017-07-21 01:00:33,985 8336e237 MainThread INFO     Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-07-21 01:00:34,573 8336e237 MainThread WARNING  The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-07-21 01:00:34,633 8336e237 MainThread ERROR    Prerequisite check failed for beam_integration_benchmark
Traceback (most recent call last):
  File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 374, in _CreateBenchmarkSpecs
    check_prereqs(config)
  File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 99, in CheckPrerequisites
    if (len(FLAGS.beam_it_options) > 0 and
TypeError: object of type 'NoneType' has no len()
Traceback (most recent call last):
  File "PerfKitBenchmarker/pkb.py", line 21, in <module>
    sys.exit(Main())
  File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 824, in Main
    return RunBenchmarks()
  File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 736, in RunBenchmarks
    benchmark_specs = _CreateBenchmarkSpecs()
  File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 374, in _CreateBenchmarkSpecs
    check_prereqs(config)
  File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 99, in CheckPrerequisites
    if (len(FLAGS.beam_it_options) > 0 and
TypeError: object of type 'NoneType' has no len()
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user sourabhbajaj@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user klk@google.com