You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2017/05/01 18:00:12 UTC
Build failed in Jenkins: beam_PerformanceTests_Dataflow #360
See <https://builds.apache.org/job/beam_PerformanceTests_Dataflow/360/display/redirect?page=changes>
Changes:
[staslevin] [BEAM-2074,BEAM-2073] Fixed SourceDStream's rate control usage.
[lcwik] Use a new ReflectData for each AvroCoder instance
[lcwik] [BEAM-2130] Ensure the options id is never null.
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam5 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/*:refs/remotes/origin/pr/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision fe2a32000be87bbed1e8ae5a81a8e0368f0b0234 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f fe2a32000be87bbed1e8ae5a81a8e0368f0b0234
> git rev-list 254470e626edb6a013ba5cb2d3312dc3bfbdcb51 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Dataflow] $ /bin/bash -xe /tmp/hudson7363877203267415167.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Dataflow] $ /bin/bash -xe /tmp/hudson5694443500676162576.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Dataflow] $ /bin/bash -xe /tmp/hudson2268732869890752569.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied (use --upgrade to upgrade): python-gflags==3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied (use --upgrade to upgrade): jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Installing extra requirements: 'windows'
Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied (use --upgrade to upgrade): futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.11 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied (use --upgrade to upgrade): pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied (use --upgrade to upgrade): numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied (use --upgrade to upgrade): functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied (use --upgrade to upgrade): contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Cleaning up...
[beam_PerformanceTests_Dataflow] $ /bin/bash -xe /tmp/hudson7788084381097802984.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --benchmarks=dpb_wordcount_benchmark --dpb_dataflow_staging_location=gs://temp-storage-for-perf-tests/staging --dpb_wordcount_input=dataflow-samples/shakespeare/kinglear.txt --config_override=dpb_wordcount_benchmark.dpb_service.service_type=dataflow
WARNING:root:File resource loader root perfkitbenchmarker/data/ycsb is not a directory.
2017-05-01 18:00:12,066 e7cfe82e MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/e7cfe82e/pkb.log
2017-05-01 18:00:12,066 e7cfe82e MainThread INFO PerfKitBenchmarker version: v1.11.0-47-gb38dddb
2017-05-01 18:00:12,066 e7cfe82e MainThread INFO Flag values:
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--dpb_wordcount_input=dataflow-samples/shakespeare/kinglear.txt
--dpb_log_level=INFO
--official
--benchmarks=dpb_wordcount_benchmark
--config_override=dpb_wordcount_benchmark.dpb_service.service_type=dataflow
--dpb_dataflow_staging_location=gs://temp-storage-for-perf-tests/staging
2017-05-01 18:00:12,193 e7cfe82e MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-05-01 18:00:12,280 e7cfe82e MainThread dpb_wordcount_benchmark(1/1) INFO Provisioning resources for benchmark dpb_wordcount_benchmark
2017-05-01 18:00:12,282 e7cfe82e MainThread dpb_wordcount_benchmark(1/1) INFO Preparing benchmark dpb_wordcount_benchmark
2017-05-01 18:00:12,282 e7cfe82e MainThread dpb_wordcount_benchmark(1/1) INFO Running benchmark dpb_wordcount_benchmark
2017-05-01 18:00:12,288 e7cfe82e MainThread dpb_wordcount_benchmark(1/1) ERROR Error during benchmark dpb_wordcount_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 543, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 459, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/dpb_wordcount_benchmark.py",> line 162, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 115, in SubmitJob
stdout, _, _ = vm_util.IssueCommand(cmd)
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/vm_util.py",> line 287, in IssueCommand
full_cmd = ' '.join(cmd)
TypeError: sequence item 2: expected string, NoneType found
2017-05-01 18:00:12,290 e7cfe82e MainThread dpb_wordcount_benchmark(1/1) ERROR Benchmark 1/1 dpb_wordcount_benchmark (UID: dpb_wordcount_benchmark0) failed. Execution will continue.
2017-05-01 18:00:12,316 e7cfe82e MainThread INFO Benchmark run statuses:
---------------------------------------------------------
Name UID Status
---------------------------------------------------------
dpb_wordcount_benchmark dpb_wordcount_benchmark0 FAILED
---------------------------------------------------------
Success rate: 0.00% (0/1)
2017-05-01 18:00:12,317 e7cfe82e MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/e7cfe82e/pkb.log
2017-05-01 18:00:12,317 e7cfe82e MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/e7cfe82e/completion_statuses.json
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PerformanceTests_Dataflow #376
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Dataflow/376/display/redirect?page=changes>
Changes:
[klk] Activate WindowedWordCountIT on Apex runner
[klk] No parallelism for Apex WindowedWordCountIT
[dhalperi] [BEAM-2016] Delete HdfsFileSource & Sink
[tgroh] Run Optimize Imports on the Repository
[tgroh] Move AppliedPTransform into the Runners package
[tgroh] Convert Coder into an Abstract Static Class
[dhalperi] [BEAM-59] Fully delete IOChannelUtils
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam8 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/*:refs/remotes/origin/pr/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 0d2a0aeeef92bdbc23e4061ec066da47e3c7de38 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 0d2a0aeeef92bdbc23e4061ec066da47e3c7de38
> git rev-list 3bffe0e0014bdd6ae73dc2e8ecfc2b61d066120c # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Dataflow] $ /bin/bash -xe /tmp/hudson6165280624966686801.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Dataflow] $ /bin/bash -xe /tmp/hudson4580259411695657974.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Dataflow] $ /bin/bash -xe /tmp/hudson991341416332915944.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied (use --upgrade to upgrade): python-gflags==3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied (use --upgrade to upgrade): jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Installing extra requirements: 'windows'
Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied (use --upgrade to upgrade): futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.11 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied (use --upgrade to upgrade): pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied (use --upgrade to upgrade): numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied (use --upgrade to upgrade): functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied (use --upgrade to upgrade): contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Cleaning up...
[beam_PerformanceTests_Dataflow] $ /bin/bash -xe /tmp/hudson4900402012477053724.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --benchmarks=dpb_wordcount_benchmark --dpb_dataflow_staging_location=gs://temp-storage-for-perf-tests/staging --dpb_wordcount_input=dataflow-samples/shakespeare/kinglear.txt --config_override=dpb_wordcount_benchmark.dpb_service.service_type=dataflow
WARNING:root:File resource loader root perfkitbenchmarker/data/ycsb is not a directory.
2017-05-05 18:54:23,115 424c35d3 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/424c35d3/pkb.log
2017-05-05 18:54:23,115 424c35d3 MainThread INFO PerfKitBenchmarker version: v1.11.0-49-g10dd95b
2017-05-05 18:54:23,116 424c35d3 MainThread INFO Flag values:
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--dpb_wordcount_input=dataflow-samples/shakespeare/kinglear.txt
--dpb_log_level=INFO
--official
--benchmarks=dpb_wordcount_benchmark
--config_override=dpb_wordcount_benchmark.dpb_service.service_type=dataflow
--dpb_dataflow_staging_location=gs://temp-storage-for-perf-tests/staging
2017-05-05 18:54:23,645 424c35d3 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-05-05 18:54:23,740 424c35d3 MainThread dpb_wordcount_benchmark(1/1) INFO Provisioning resources for benchmark dpb_wordcount_benchmark
2017-05-05 18:54:23,743 424c35d3 MainThread dpb_wordcount_benchmark(1/1) INFO Preparing benchmark dpb_wordcount_benchmark
2017-05-05 18:54:23,743 424c35d3 MainThread dpb_wordcount_benchmark(1/1) INFO Running benchmark dpb_wordcount_benchmark
2017-05-05 18:54:23,748 424c35d3 MainThread dpb_wordcount_benchmark(1/1) ERROR Error during benchmark dpb_wordcount_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 543, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 459, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/dpb_wordcount_benchmark.py",> line 162, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 115, in SubmitJob
stdout, _, _ = vm_util.IssueCommand(cmd)
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/vm_util.py",> line 287, in IssueCommand
full_cmd = ' '.join(cmd)
TypeError: sequence item 2: expected string, NoneType found
2017-05-05 18:54:23,750 424c35d3 MainThread dpb_wordcount_benchmark(1/1) ERROR Benchmark 1/1 dpb_wordcount_benchmark (UID: dpb_wordcount_benchmark0) failed. Execution will continue.
2017-05-05 18:54:23,776 424c35d3 MainThread INFO Benchmark run statuses:
---------------------------------------------------------
Name UID Status
---------------------------------------------------------
dpb_wordcount_benchmark dpb_wordcount_benchmark0 FAILED
---------------------------------------------------------
Success rate: 0.00% (0/1)
2017-05-05 18:54:23,776 424c35d3 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/424c35d3/pkb.log
2017-05-05 18:54:23,777 424c35d3 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/424c35d3/completion_statuses.json
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PerformanceTests_Dataflow #375
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Dataflow/375/display/redirect?page=changes>
Changes:
[aljoscha.krettek] [BEAM-1862] SplittableDoFnOperator should close the
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam5 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/*:refs/remotes/origin/pr/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 3bffe0e0014bdd6ae73dc2e8ecfc2b61d066120c (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 3bffe0e0014bdd6ae73dc2e8ecfc2b61d066120c
> git rev-list 7903e59c4e07822f8def85ee6a7f8ef3ccb1ca7a # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Dataflow] $ /bin/bash -xe /tmp/hudson2417919858563827519.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Dataflow] $ /bin/bash -xe /tmp/hudson7895987214291620795.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Dataflow] $ /bin/bash -xe /tmp/hudson2766035856415306403.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied (use --upgrade to upgrade): python-gflags==3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied (use --upgrade to upgrade): jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Installing extra requirements: 'windows'
Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied (use --upgrade to upgrade): futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.11 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied (use --upgrade to upgrade): pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied (use --upgrade to upgrade): numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied (use --upgrade to upgrade): functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied (use --upgrade to upgrade): contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Cleaning up...
[beam_PerformanceTests_Dataflow] $ /bin/bash -xe /tmp/hudson5946974480346757738.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --benchmarks=dpb_wordcount_benchmark --dpb_dataflow_staging_location=gs://temp-storage-for-perf-tests/staging --dpb_wordcount_input=dataflow-samples/shakespeare/kinglear.txt --config_override=dpb_wordcount_benchmark.dpb_service.service_type=dataflow
WARNING:root:File resource loader root perfkitbenchmarker/data/ycsb is not a directory.
2017-05-05 12:00:18,732 d6d41712 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/d6d41712/pkb.log
2017-05-05 12:00:18,732 d6d41712 MainThread INFO PerfKitBenchmarker version: v1.11.0-49-g10dd95b
2017-05-05 12:00:18,733 d6d41712 MainThread INFO Flag values:
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--dpb_wordcount_input=dataflow-samples/shakespeare/kinglear.txt
--dpb_log_level=INFO
--official
--benchmarks=dpb_wordcount_benchmark
--config_override=dpb_wordcount_benchmark.dpb_service.service_type=dataflow
--dpb_dataflow_staging_location=gs://temp-storage-for-perf-tests/staging
2017-05-05 12:00:19,014 d6d41712 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-05-05 12:00:19,172 d6d41712 MainThread dpb_wordcount_benchmark(1/1) INFO Provisioning resources for benchmark dpb_wordcount_benchmark
2017-05-05 12:00:19,176 d6d41712 MainThread dpb_wordcount_benchmark(1/1) INFO Preparing benchmark dpb_wordcount_benchmark
2017-05-05 12:00:19,177 d6d41712 MainThread dpb_wordcount_benchmark(1/1) INFO Running benchmark dpb_wordcount_benchmark
2017-05-05 12:00:19,185 d6d41712 MainThread dpb_wordcount_benchmark(1/1) ERROR Error during benchmark dpb_wordcount_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 543, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 459, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/dpb_wordcount_benchmark.py",> line 162, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 115, in SubmitJob
stdout, _, _ = vm_util.IssueCommand(cmd)
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/vm_util.py",> line 287, in IssueCommand
full_cmd = ' '.join(cmd)
TypeError: sequence item 2: expected string, NoneType found
2017-05-05 12:00:19,187 d6d41712 MainThread dpb_wordcount_benchmark(1/1) ERROR Benchmark 1/1 dpb_wordcount_benchmark (UID: dpb_wordcount_benchmark0) failed. Execution will continue.
2017-05-05 12:00:19,240 d6d41712 MainThread INFO Benchmark run statuses:
---------------------------------------------------------
Name UID Status
---------------------------------------------------------
dpb_wordcount_benchmark dpb_wordcount_benchmark0 FAILED
---------------------------------------------------------
Success rate: 0.00% (0/1)
2017-05-05 12:00:19,240 d6d41712 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/d6d41712/pkb.log
2017-05-05 12:00:19,240 d6d41712 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/d6d41712/completion_statuses.json
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PerformanceTests_Dataflow #374
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Dataflow/374/display/redirect?page=changes>
Changes:
[klk] Move Java sdk.util.state to sdk.state
[klk] Add @Internal and @Experimental to state package
[klk] Move user-facing timer-related classes out of util
[klk] Mark TimeDomain experimental alongside Timers; improve javadoc
[klk] Move WindowingStrategy from util to values
[klk] Move PathValidator adjacent to other filesystem bits
[klk] Move SideInputReader to runners/core-java
[klk] Move Reshuffle and ReshuffleTrigger out of util
[klk] Move ValueWithRecordId to sdk.values, annotated
[klk] Move some PCollectionView bits out of util
[klk] Update Dataflow worker to 20170504-2
[tgroh] Update expectation in ProcessBundleHandlerTest
[dhalperi] [BEAM-1871] Move out test only utilities used only in examples/java out
[dhalperi] Revert "This closes #2905"
[klk] Adjust error message expectation in DoFnSignaturesTest
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam5 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/*:refs/remotes/origin/pr/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 7903e59c4e07822f8def85ee6a7f8ef3ccb1ca7a (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 7903e59c4e07822f8def85ee6a7f8ef3ccb1ca7a
> git rev-list 2f4dd8dfaf9e94ff2a1f97edc695fbcd75be34cd # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Dataflow] $ /bin/bash -xe /tmp/hudson7538084356404224735.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Dataflow] $ /bin/bash -xe /tmp/hudson4042714355130252232.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Dataflow] $ /bin/bash -xe /tmp/hudson426470528539862612.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied (use --upgrade to upgrade): python-gflags==3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied (use --upgrade to upgrade): jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Installing extra requirements: 'windows'
Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied (use --upgrade to upgrade): futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.11 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied (use --upgrade to upgrade): pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied (use --upgrade to upgrade): numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied (use --upgrade to upgrade): functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied (use --upgrade to upgrade): contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Cleaning up...
[beam_PerformanceTests_Dataflow] $ /bin/bash -xe /tmp/hudson4859371696513139763.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --benchmarks=dpb_wordcount_benchmark --dpb_dataflow_staging_location=gs://temp-storage-for-perf-tests/staging --dpb_wordcount_input=dataflow-samples/shakespeare/kinglear.txt --config_override=dpb_wordcount_benchmark.dpb_service.service_type=dataflow
WARNING:root:File resource loader root perfkitbenchmarker/data/ycsb is not a directory.
2017-05-05 06:00:18,988 fd9fd549 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/fd9fd549/pkb.log
2017-05-05 06:00:18,988 fd9fd549 MainThread INFO PerfKitBenchmarker version: v1.11.0-49-g10dd95b
2017-05-05 06:00:18,989 fd9fd549 MainThread INFO Flag values:
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--dpb_wordcount_input=dataflow-samples/shakespeare/kinglear.txt
--dpb_log_level=INFO
--official
--benchmarks=dpb_wordcount_benchmark
--config_override=dpb_wordcount_benchmark.dpb_service.service_type=dataflow
--dpb_dataflow_staging_location=gs://temp-storage-for-perf-tests/staging
2017-05-05 06:00:19,229 fd9fd549 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-05-05 06:00:19,314 fd9fd549 MainThread dpb_wordcount_benchmark(1/1) INFO Provisioning resources for benchmark dpb_wordcount_benchmark
2017-05-05 06:00:20,165 fd9fd549 MainThread dpb_wordcount_benchmark(1/1) INFO Preparing benchmark dpb_wordcount_benchmark
2017-05-05 06:00:20,165 fd9fd549 MainThread dpb_wordcount_benchmark(1/1) INFO Running benchmark dpb_wordcount_benchmark
2017-05-05 06:00:20,170 fd9fd549 MainThread dpb_wordcount_benchmark(1/1) ERROR Error during benchmark dpb_wordcount_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 543, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 459, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/dpb_wordcount_benchmark.py",> line 162, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 115, in SubmitJob
stdout, _, _ = vm_util.IssueCommand(cmd)
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/vm_util.py",> line 287, in IssueCommand
full_cmd = ' '.join(cmd)
TypeError: sequence item 2: expected string, NoneType found
2017-05-05 06:00:20,172 fd9fd549 MainThread dpb_wordcount_benchmark(1/1) ERROR Benchmark 1/1 dpb_wordcount_benchmark (UID: dpb_wordcount_benchmark0) failed. Execution will continue.
2017-05-05 06:00:20,203 fd9fd549 MainThread INFO Benchmark run statuses:
---------------------------------------------------------
Name UID Status
---------------------------------------------------------
dpb_wordcount_benchmark dpb_wordcount_benchmark0 FAILED
---------------------------------------------------------
Success rate: 0.00% (0/1)
2017-05-05 06:00:20,204 fd9fd549 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/fd9fd549/pkb.log
2017-05-05 06:00:20,204 fd9fd549 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/fd9fd549/completion_statuses.json
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PerformanceTests_Dataflow #373
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Dataflow/373/display/redirect?page=changes>
Changes:
[dhalperi] GCP-IO: manage versions in top level pom
[dhalperi] Remove IOChannelUtil/Factory from BigQueryIO
[klk] Eliminate Pipeline.getOptions
[robertwb] Cleanup old cython files before testing again
[klk] [BEAM-1727] Add align and offset to Timer
[klk] Fix initial watermark of DoFnOperator in Flink runner
[dhalperi] [BEAM-59] Remove IOChannelUtils, IOChannelFactory
[dhalperi] Move thrown.expect to correct place
[tgroh] Note when checkpoints may be committed.
[tgroh] Split StartBundleContext and FinishBundleContext
[kirpichov] [BEAM-2170] PubsubMessageWithAttributesCoder should not NPE on messages
[kirpichov] Makes PubsubMessagePayloadOnlyCoder not require whole-stream context
[kirpichov] Moves coder choice into PubsubSource
[davor] [BEAM-2144] Remove transform.reflect and utils from Javadocs
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam5 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/*:refs/remotes/origin/pr/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 2f4dd8dfaf9e94ff2a1f97edc695fbcd75be34cd (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 2f4dd8dfaf9e94ff2a1f97edc695fbcd75be34cd
> git rev-list b73918b55ab06e5a47ef9dc33ae3dbaebaed330a # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Dataflow] $ /bin/bash -xe /tmp/hudson227170201862895542.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Dataflow] $ /bin/bash -xe /tmp/hudson5916819713713284246.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Dataflow] $ /bin/bash -xe /tmp/hudson5309455617960359779.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied (use --upgrade to upgrade): python-gflags==3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied (use --upgrade to upgrade): jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Installing extra requirements: 'windows'
Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied (use --upgrade to upgrade): futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.11 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied (use --upgrade to upgrade): pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied (use --upgrade to upgrade): numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied (use --upgrade to upgrade): functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied (use --upgrade to upgrade): contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Cleaning up...
[beam_PerformanceTests_Dataflow] $ /bin/bash -xe /tmp/hudson7566161025314604258.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --benchmarks=dpb_wordcount_benchmark --dpb_dataflow_staging_location=gs://temp-storage-for-perf-tests/staging --dpb_wordcount_input=dataflow-samples/shakespeare/kinglear.txt --config_override=dpb_wordcount_benchmark.dpb_service.service_type=dataflow
WARNING:root:File resource loader root perfkitbenchmarker/data/ycsb is not a directory.
2017-05-05 00:00:58,427 4a3fc96e MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/4a3fc96e/pkb.log
2017-05-05 00:00:58,465 4a3fc96e MainThread INFO PerfKitBenchmarker version: v1.11.0-49-g10dd95b
2017-05-05 00:00:58,465 4a3fc96e MainThread INFO Flag values:
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--dpb_wordcount_input=dataflow-samples/shakespeare/kinglear.txt
--dpb_log_level=INFO
--official
--benchmarks=dpb_wordcount_benchmark
--config_override=dpb_wordcount_benchmark.dpb_service.service_type=dataflow
--dpb_dataflow_staging_location=gs://temp-storage-for-perf-tests/staging
2017-05-05 00:00:59,060 4a3fc96e MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-05-05 00:00:59,163 4a3fc96e MainThread dpb_wordcount_benchmark(1/1) INFO Provisioning resources for benchmark dpb_wordcount_benchmark
2017-05-05 00:00:59,165 4a3fc96e MainThread dpb_wordcount_benchmark(1/1) INFO Preparing benchmark dpb_wordcount_benchmark
2017-05-05 00:00:59,166 4a3fc96e MainThread dpb_wordcount_benchmark(1/1) INFO Running benchmark dpb_wordcount_benchmark
2017-05-05 00:00:59,170 4a3fc96e MainThread dpb_wordcount_benchmark(1/1) ERROR Error during benchmark dpb_wordcount_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 543, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 459, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/dpb_wordcount_benchmark.py",> line 162, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 115, in SubmitJob
stdout, _, _ = vm_util.IssueCommand(cmd)
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/vm_util.py",> line 287, in IssueCommand
full_cmd = ' '.join(cmd)
TypeError: sequence item 2: expected string, NoneType found
2017-05-05 00:00:59,172 4a3fc96e MainThread dpb_wordcount_benchmark(1/1) ERROR Benchmark 1/1 dpb_wordcount_benchmark (UID: dpb_wordcount_benchmark0) failed. Execution will continue.
2017-05-05 00:00:59,200 4a3fc96e MainThread INFO Benchmark run statuses:
---------------------------------------------------------
Name UID Status
---------------------------------------------------------
dpb_wordcount_benchmark dpb_wordcount_benchmark0 FAILED
---------------------------------------------------------
Success rate: 0.00% (0/1)
2017-05-05 00:00:59,201 4a3fc96e MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/4a3fc96e/pkb.log
2017-05-05 00:00:59,201 4a3fc96e MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/4a3fc96e/completion_statuses.json
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PerformanceTests_Dataflow #372
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Dataflow/372/display/redirect?page=changes>
Changes:
[klk] Annotate internal methods of PCollection
[klk] Annotate internal pieces of sdks.transforms
[klk] Tighten access control and internal annotations for triggers
[klk] The transforms.reflect package is not for users
[klk] Annotate internal-only bits of Java sdk.runners
[klk] Tighten access in sdk.options
[klk] Annotate internal methods on Pipeline
[lcwik] [BEAM-2165] Update Apex to support serializing/deserializing custom user
[lcwik] [BEAM-2165] Update Dataflow to support serializing/deserializing custom
[lcwik] [BEAM-2165] Update Flink to support serializing/deserializing custom
[lcwik] [BEAM-1871] Migrate ReleaseInfo away from Google API client GenericJson
[dhalperi] DataflowRunner: integration test GCP-IO
[lcwik] [BEAM-1491]Identify HADOOP_CONF_DIR(or YARN_CONF_DIR) environment
[lcwik] Split Coder's encode/decode methods into two methods depending on
[dhalperi] [BEAM-2162] Add logging to long BigQuery jobs
[tgroh] Use BinaryCombineLongFn in GroupIntoBatches
[altay] [BEAM-2152] Remove gcloud auth as application default credentials does
[dhalperi] AvroIOTest: stop using IOChannelUtils, remove invalid test
[dhalperi] DataflowRunner: switch from IOChannels to FileSystems for creating files
[dhalperi] DataflowPipelineOptions: use FileSystems, not IOChannelUtils, to resolve
[dhalperi] Stop registering IOChannelFactories in SDK harness, runners, tests
[dhalperi] FileSystems: add match(String) for single spec
[dhalperi] WriteWithShardingFactoryTest: switch to FileSystems
[dhalperi] ReadTranslator: remove outdated comment
[dhalperi] TFRecordIO: switch to FileSystems from IOChannelUtils
[lcwik] Remove POutput#recordAsOutput
[robertwb] One more TaggedOutput name change.
[dhalperi] Update JavaDoc of checkpoint, and expose `enable.auto.commit` for more
[lcwik] Use CloudObject encodings in the Beam Fn Harness
[aviemzur] [BEAM-1763] Verify PAssert execution in runners which support metrics.
[aviemzur] [BEAM-1726] Fix empty side inputs in Flink Streaming Runner
[aviemzur] [BEAM-1726] Fix RuntimeException throwing in FlinkStateInternals
[aviemzur] [BEAM-1726] Fix Flatten with input copies in Flink Streaming Runner
[aviemzur] [BEAM-1763] Replace usage of PipelineRule with TestPipeline in Spark
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam2 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/*:refs/remotes/origin/pr/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision b73918b55ab06e5a47ef9dc33ae3dbaebaed330a (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f b73918b55ab06e5a47ef9dc33ae3dbaebaed330a
> git rev-list ff6bb353002610169a8079d1164118606bebb21e # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Dataflow] $ /bin/bash -xe /tmp/hudson8302199996735440149.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Dataflow] $ /bin/bash -xe /tmp/hudson6393172615410736925.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Dataflow] $ /bin/bash -xe /tmp/hudson4182315741520362001.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied (use --upgrade to upgrade): python-gflags==3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied (use --upgrade to upgrade): jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Installing extra requirements: 'windows'
Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied (use --upgrade to upgrade): futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.11 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied (use --upgrade to upgrade): pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied (use --upgrade to upgrade): numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied (use --upgrade to upgrade): functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied (use --upgrade to upgrade): contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Cleaning up...
[beam_PerformanceTests_Dataflow] $ /bin/bash -xe /tmp/hudson2998196018294298494.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --benchmarks=dpb_wordcount_benchmark --dpb_dataflow_staging_location=gs://temp-storage-for-perf-tests/staging --dpb_wordcount_input=dataflow-samples/shakespeare/kinglear.txt --config_override=dpb_wordcount_benchmark.dpb_service.service_type=dataflow
WARNING:root:File resource loader root perfkitbenchmarker/data/ycsb is not a directory.
2017-05-04 18:18:28,736 8fcb4f87 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/8fcb4f87/pkb.log
2017-05-04 18:18:28,736 8fcb4f87 MainThread INFO PerfKitBenchmarker version: v1.11.0-48-ge557505
2017-05-04 18:18:28,737 8fcb4f87 MainThread INFO Flag values:
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--dpb_wordcount_input=dataflow-samples/shakespeare/kinglear.txt
--dpb_log_level=INFO
--official
--benchmarks=dpb_wordcount_benchmark
--config_override=dpb_wordcount_benchmark.dpb_service.service_type=dataflow
--dpb_dataflow_staging_location=gs://temp-storage-for-perf-tests/staging
2017-05-04 18:18:29,449 8fcb4f87 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-05-04 18:18:29,614 8fcb4f87 MainThread dpb_wordcount_benchmark(1/1) INFO Provisioning resources for benchmark dpb_wordcount_benchmark
2017-05-04 18:18:29,617 8fcb4f87 MainThread dpb_wordcount_benchmark(1/1) INFO Preparing benchmark dpb_wordcount_benchmark
2017-05-04 18:18:29,617 8fcb4f87 MainThread dpb_wordcount_benchmark(1/1) INFO Running benchmark dpb_wordcount_benchmark
2017-05-04 18:18:29,624 8fcb4f87 MainThread dpb_wordcount_benchmark(1/1) ERROR Error during benchmark dpb_wordcount_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 543, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 459, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/dpb_wordcount_benchmark.py",> line 162, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 115, in SubmitJob
stdout, _, _ = vm_util.IssueCommand(cmd)
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/vm_util.py",> line 287, in IssueCommand
full_cmd = ' '.join(cmd)
TypeError: sequence item 2: expected string, NoneType found
2017-05-04 18:18:29,627 8fcb4f87 MainThread dpb_wordcount_benchmark(1/1) ERROR Benchmark 1/1 dpb_wordcount_benchmark (UID: dpb_wordcount_benchmark0) failed. Execution will continue.
2017-05-04 18:18:29,681 8fcb4f87 MainThread INFO Benchmark run statuses:
---------------------------------------------------------
Name UID Status
---------------------------------------------------------
dpb_wordcount_benchmark dpb_wordcount_benchmark0 FAILED
---------------------------------------------------------
Success rate: 0.00% (0/1)
2017-05-04 18:18:29,682 8fcb4f87 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/8fcb4f87/pkb.log
2017-05-04 18:18:29,682 8fcb4f87 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/8fcb4f87/completion_statuses.json
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PerformanceTests_Dataflow #371
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Dataflow/371/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam5 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/*:refs/remotes/origin/pr/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision ff6bb353002610169a8079d1164118606bebb21e (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f ff6bb353002610169a8079d1164118606bebb21e
> git rev-list ff6bb353002610169a8079d1164118606bebb21e # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Dataflow] $ /bin/bash -xe /tmp/hudson1079305215059373945.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Dataflow] $ /bin/bash -xe /tmp/hudson400584237070228302.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Dataflow] $ /bin/bash -xe /tmp/hudson6782621028537445695.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied (use --upgrade to upgrade): python-gflags==3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied (use --upgrade to upgrade): jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Installing extra requirements: 'windows'
Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied (use --upgrade to upgrade): futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.11 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied (use --upgrade to upgrade): pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied (use --upgrade to upgrade): numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied (use --upgrade to upgrade): functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied (use --upgrade to upgrade): contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Cleaning up...
[beam_PerformanceTests_Dataflow] $ /bin/bash -xe /tmp/hudson2869572068032251778.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --benchmarks=dpb_wordcount_benchmark --dpb_dataflow_staging_location=gs://temp-storage-for-perf-tests/staging --dpb_wordcount_input=dataflow-samples/shakespeare/kinglear.txt --config_override=dpb_wordcount_benchmark.dpb_service.service_type=dataflow
WARNING:root:File resource loader root perfkitbenchmarker/data/ycsb is not a directory.
2017-05-04 12:01:19,146 397f971b MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/397f971b/pkb.log
2017-05-04 12:01:19,147 397f971b MainThread INFO PerfKitBenchmarker version: v1.11.0-48-ge557505
2017-05-04 12:01:19,147 397f971b MainThread INFO Flag values:
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--dpb_wordcount_input=dataflow-samples/shakespeare/kinglear.txt
--dpb_log_level=INFO
--official
--benchmarks=dpb_wordcount_benchmark
--config_override=dpb_wordcount_benchmark.dpb_service.service_type=dataflow
--dpb_dataflow_staging_location=gs://temp-storage-for-perf-tests/staging
2017-05-04 12:01:19,263 397f971b MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-05-04 12:01:19,350 397f971b MainThread dpb_wordcount_benchmark(1/1) INFO Provisioning resources for benchmark dpb_wordcount_benchmark
2017-05-04 12:01:19,352 397f971b MainThread dpb_wordcount_benchmark(1/1) INFO Preparing benchmark dpb_wordcount_benchmark
2017-05-04 12:01:19,352 397f971b MainThread dpb_wordcount_benchmark(1/1) INFO Running benchmark dpb_wordcount_benchmark
2017-05-04 12:01:19,358 397f971b MainThread dpb_wordcount_benchmark(1/1) ERROR Error during benchmark dpb_wordcount_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 543, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 459, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/dpb_wordcount_benchmark.py",> line 162, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 115, in SubmitJob
stdout, _, _ = vm_util.IssueCommand(cmd)
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/vm_util.py",> line 287, in IssueCommand
full_cmd = ' '.join(cmd)
TypeError: sequence item 2: expected string, NoneType found
2017-05-04 12:01:19,360 397f971b MainThread dpb_wordcount_benchmark(1/1) ERROR Benchmark 1/1 dpb_wordcount_benchmark (UID: dpb_wordcount_benchmark0) failed. Execution will continue.
2017-05-04 12:01:19,386 397f971b MainThread INFO Benchmark run statuses:
---------------------------------------------------------
Name UID Status
---------------------------------------------------------
dpb_wordcount_benchmark dpb_wordcount_benchmark0 FAILED
---------------------------------------------------------
Success rate: 0.00% (0/1)
2017-05-04 12:01:19,386 397f971b MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/397f971b/pkb.log
2017-05-04 12:01:19,386 397f971b MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/397f971b/completion_statuses.json
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PerformanceTests_Dataflow #370
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Dataflow/370/display/redirect?page=changes>
Changes:
[dhalperi] Convert WriteFiles/FileBasedSink from IOChannelFactory to FileSystems
[tgroh] Inline TypedPValue
[lcwik] [BEAM-2157] Fix HDFS Test That Failed in Other JDK Versions
[kirpichov] Provides a default coder for PubsubMessage
[kirpichov] Moves PubsubMessage to upper level and renames payload
[klk] BigQueryIO: Remove tempLocation usage at pipeline construction time
[dhalperi] Removal of Aggregator class. Also removal from comments.
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam5 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/*:refs/remotes/origin/pr/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision ff6bb353002610169a8079d1164118606bebb21e (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f ff6bb353002610169a8079d1164118606bebb21e
> git rev-list b4bafd09282cf66adf121b750d3e2d9e5a0fefbe # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Dataflow] $ /bin/bash -xe /tmp/hudson1444438434785476686.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Dataflow] $ /bin/bash -xe /tmp/hudson7081035498281775005.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Dataflow] $ /bin/bash -xe /tmp/hudson2859973814943599527.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied (use --upgrade to upgrade): python-gflags==3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied (use --upgrade to upgrade): jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Installing extra requirements: 'windows'
Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied (use --upgrade to upgrade): futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.11 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied (use --upgrade to upgrade): pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied (use --upgrade to upgrade): numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied (use --upgrade to upgrade): functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied (use --upgrade to upgrade): contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Cleaning up...
[beam_PerformanceTests_Dataflow] $ /bin/bash -xe /tmp/hudson7637998100796022197.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --benchmarks=dpb_wordcount_benchmark --dpb_dataflow_staging_location=gs://temp-storage-for-perf-tests/staging --dpb_wordcount_input=dataflow-samples/shakespeare/kinglear.txt --config_override=dpb_wordcount_benchmark.dpb_service.service_type=dataflow
WARNING:root:File resource loader root perfkitbenchmarker/data/ycsb is not a directory.
2017-05-04 06:00:42,550 b7a79798 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/b7a79798/pkb.log
2017-05-04 06:00:42,550 b7a79798 MainThread INFO PerfKitBenchmarker version: v1.11.0-48-ge557505
2017-05-04 06:00:42,550 b7a79798 MainThread INFO Flag values:
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--dpb_wordcount_input=dataflow-samples/shakespeare/kinglear.txt
--dpb_log_level=INFO
--official
--benchmarks=dpb_wordcount_benchmark
--config_override=dpb_wordcount_benchmark.dpb_service.service_type=dataflow
--dpb_dataflow_staging_location=gs://temp-storage-for-perf-tests/staging
2017-05-04 06:00:42,855 b7a79798 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-05-04 06:00:42,964 b7a79798 MainThread dpb_wordcount_benchmark(1/1) INFO Provisioning resources for benchmark dpb_wordcount_benchmark
2017-05-04 06:00:42,966 b7a79798 MainThread dpb_wordcount_benchmark(1/1) INFO Preparing benchmark dpb_wordcount_benchmark
2017-05-04 06:00:42,967 b7a79798 MainThread dpb_wordcount_benchmark(1/1) INFO Running benchmark dpb_wordcount_benchmark
2017-05-04 06:00:42,972 b7a79798 MainThread dpb_wordcount_benchmark(1/1) ERROR Error during benchmark dpb_wordcount_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 543, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 459, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/dpb_wordcount_benchmark.py",> line 162, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 115, in SubmitJob
stdout, _, _ = vm_util.IssueCommand(cmd)
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/vm_util.py",> line 287, in IssueCommand
full_cmd = ' '.join(cmd)
TypeError: sequence item 2: expected string, NoneType found
2017-05-04 06:00:42,974 b7a79798 MainThread dpb_wordcount_benchmark(1/1) ERROR Benchmark 1/1 dpb_wordcount_benchmark (UID: dpb_wordcount_benchmark0) failed. Execution will continue.
2017-05-04 06:00:43,001 b7a79798 MainThread INFO Benchmark run statuses:
---------------------------------------------------------
Name UID Status
---------------------------------------------------------
dpb_wordcount_benchmark dpb_wordcount_benchmark0 FAILED
---------------------------------------------------------
Success rate: 0.00% (0/1)
2017-05-04 06:00:43,002 b7a79798 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/b7a79798/pkb.log
2017-05-04 06:00:43,002 b7a79798 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/b7a79798/completion_statuses.json
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PerformanceTests_Dataflow #369
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Dataflow/369/display/redirect?page=changes>
Changes:
[klk] Use Thread context ClassLoader as parent in AvroCoderTest
[tgroh] Mark all PCollectionView methods internal
[lcwik] [BEAM-1871] Move GCP specific serialization CloudObject and supporting
[tgroh] Reduce visibility of DirectRunner classes
[altay] Retry on correct error codes for datastoreio
[robertwb] Try PTransform-based coder inference before using fallback coder.
[robertwb] Fix coder inference ambiguity in CombineFnsTest.
[klk] Remove job name usages from BigQueryIO at pipeline construction time
[kirpichov] Fully general dynamic tables (including schemas) in BigQueryIO.
[kirpichov] Fixes PubsubIO.Write translation in Dataflow runner
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam2 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/>
Cloning the remote Git repository
Cloning repository https://github.com/apache/beam.git
> git init <https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/> # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/*
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
> git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/*:refs/remotes/origin/pr/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision b4bafd09282cf66adf121b750d3e2d9e5a0fefbe (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f b4bafd09282cf66adf121b750d3e2d9e5a0fefbe
> git rev-list 0d19d3cb210ef179e1831b971f7cf6d3d6bb1dde # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Dataflow] $ /bin/bash -xe /tmp/hudson197114110450717999.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Dataflow] $ /bin/bash -xe /tmp/hudson2867269520771524429.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Dataflow] $ /bin/bash -xe /tmp/hudson4022959279102052127.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied (use --upgrade to upgrade): python-gflags==3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied (use --upgrade to upgrade): jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Installing extra requirements: 'windows'
Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied (use --upgrade to upgrade): futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.11 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied (use --upgrade to upgrade): pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied (use --upgrade to upgrade): numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied (use --upgrade to upgrade): functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied (use --upgrade to upgrade): contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Cleaning up...
[beam_PerformanceTests_Dataflow] $ /bin/bash -xe /tmp/hudson1410710530977960850.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --benchmarks=dpb_wordcount_benchmark --dpb_dataflow_staging_location=gs://temp-storage-for-perf-tests/staging --dpb_wordcount_input=dataflow-samples/shakespeare/kinglear.txt --config_override=dpb_wordcount_benchmark.dpb_service.service_type=dataflow
WARNING:root:File resource loader root perfkitbenchmarker/data/ycsb is not a directory.
2017-05-04 00:03:05,091 822da785 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/822da785/pkb.log
2017-05-04 00:03:05,093 822da785 MainThread INFO PerfKitBenchmarker version: v1.11.0-48-ge557505
2017-05-04 00:03:05,094 822da785 MainThread INFO Flag values:
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--dpb_wordcount_input=dataflow-samples/shakespeare/kinglear.txt
--dpb_log_level=INFO
--official
--benchmarks=dpb_wordcount_benchmark
--config_override=dpb_wordcount_benchmark.dpb_service.service_type=dataflow
--dpb_dataflow_staging_location=gs://temp-storage-for-perf-tests/staging
2017-05-04 00:03:05,598 822da785 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-05-04 00:03:05,733 822da785 MainThread dpb_wordcount_benchmark(1/1) INFO Provisioning resources for benchmark dpb_wordcount_benchmark
2017-05-04 00:03:05,736 822da785 MainThread dpb_wordcount_benchmark(1/1) INFO Preparing benchmark dpb_wordcount_benchmark
2017-05-04 00:03:05,736 822da785 MainThread dpb_wordcount_benchmark(1/1) INFO Running benchmark dpb_wordcount_benchmark
2017-05-04 00:03:05,748 822da785 MainThread dpb_wordcount_benchmark(1/1) ERROR Error during benchmark dpb_wordcount_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 543, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 459, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/dpb_wordcount_benchmark.py",> line 162, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 115, in SubmitJob
stdout, _, _ = vm_util.IssueCommand(cmd)
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/vm_util.py",> line 287, in IssueCommand
full_cmd = ' '.join(cmd)
TypeError: sequence item 2: expected string, NoneType found
2017-05-04 00:03:05,750 822da785 MainThread dpb_wordcount_benchmark(1/1) ERROR Benchmark 1/1 dpb_wordcount_benchmark (UID: dpb_wordcount_benchmark0) failed. Execution will continue.
2017-05-04 00:03:05,803 822da785 MainThread INFO Benchmark run statuses:
---------------------------------------------------------
Name UID Status
---------------------------------------------------------
dpb_wordcount_benchmark dpb_wordcount_benchmark0 FAILED
---------------------------------------------------------
Success rate: 0.00% (0/1)
2017-05-04 00:03:05,804 822da785 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/822da785/pkb.log
2017-05-04 00:03:05,804 822da785 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/822da785/completion_statuses.json
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PerformanceTests_Dataflow #368
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Dataflow/368/display/redirect?page=changes>
Changes:
[klk] Use LinkedHashMap for step contexts in BaseExecutionContext
[dhalperi] Remove IoChannelUtils from PackageUtil
[dhalperi] [BEAM-59] FileChecksumMatcherTest: switch from IOChannelUtils to java
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/*:refs/remotes/origin/pr/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 0d19d3cb210ef179e1831b971f7cf6d3d6bb1dde (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 0d19d3cb210ef179e1831b971f7cf6d3d6bb1dde
> git rev-list 3b61f6a9034b3dbb8b08f695b7b526656660c8a6 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Dataflow] $ /bin/bash -xe /tmp/hudson8822402256232841916.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Dataflow] $ /bin/bash -xe /tmp/hudson6143726776491349836.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Dataflow] $ /bin/bash -xe /tmp/hudson5896650652856289214.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied (use --upgrade to upgrade): python-gflags==3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied (use --upgrade to upgrade): jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Installing extra requirements: 'windows'
Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied (use --upgrade to upgrade): futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.11 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied (use --upgrade to upgrade): pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied (use --upgrade to upgrade): numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied (use --upgrade to upgrade): functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied (use --upgrade to upgrade): contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Cleaning up...
[beam_PerformanceTests_Dataflow] $ /bin/bash -xe /tmp/hudson3659993746409279566.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --benchmarks=dpb_wordcount_benchmark --dpb_dataflow_staging_location=gs://temp-storage-for-perf-tests/staging --dpb_wordcount_input=dataflow-samples/shakespeare/kinglear.txt --config_override=dpb_wordcount_benchmark.dpb_service.service_type=dataflow
WARNING:root:File resource loader root perfkitbenchmarker/data/ycsb is not a directory.
2017-05-03 18:04:37,849 33130c4f MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/33130c4f/pkb.log
2017-05-03 18:04:37,849 33130c4f MainThread INFO PerfKitBenchmarker version: v1.11.0-48-ge557505
2017-05-03 18:04:37,850 33130c4f MainThread INFO Flag values:
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--dpb_wordcount_input=dataflow-samples/shakespeare/kinglear.txt
--dpb_log_level=INFO
--official
--benchmarks=dpb_wordcount_benchmark
--config_override=dpb_wordcount_benchmark.dpb_service.service_type=dataflow
--dpb_dataflow_staging_location=gs://temp-storage-for-perf-tests/staging
2017-05-03 18:04:38,526 33130c4f MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-05-03 18:04:38,662 33130c4f MainThread dpb_wordcount_benchmark(1/1) INFO Provisioning resources for benchmark dpb_wordcount_benchmark
2017-05-03 18:04:38,664 33130c4f MainThread dpb_wordcount_benchmark(1/1) INFO Preparing benchmark dpb_wordcount_benchmark
2017-05-03 18:04:38,665 33130c4f MainThread dpb_wordcount_benchmark(1/1) INFO Running benchmark dpb_wordcount_benchmark
2017-05-03 18:04:38,673 33130c4f MainThread dpb_wordcount_benchmark(1/1) ERROR Error during benchmark dpb_wordcount_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 543, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 459, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/dpb_wordcount_benchmark.py",> line 162, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 115, in SubmitJob
stdout, _, _ = vm_util.IssueCommand(cmd)
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/vm_util.py",> line 287, in IssueCommand
full_cmd = ' '.join(cmd)
TypeError: sequence item 2: expected string, NoneType found
2017-05-03 18:04:38,676 33130c4f MainThread dpb_wordcount_benchmark(1/1) ERROR Benchmark 1/1 dpb_wordcount_benchmark (UID: dpb_wordcount_benchmark0) failed. Execution will continue.
2017-05-03 18:04:38,709 33130c4f MainThread INFO Benchmark run statuses:
---------------------------------------------------------
Name UID Status
---------------------------------------------------------
dpb_wordcount_benchmark dpb_wordcount_benchmark0 FAILED
---------------------------------------------------------
Success rate: 0.00% (0/1)
2017-05-03 18:04:38,709 33130c4f MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/33130c4f/pkb.log
2017-05-03 18:04:38,710 33130c4f MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/33130c4f/completion_statuses.json
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PerformanceTests_Dataflow #367
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Dataflow/367/display/redirect?page=changes>
Changes:
[vikasrk] Triggers for Java SDK post submit tests
[kirpichov] Removed coder and parseFn from PubsubIO.Read
[kirpichov] Removed coder and formatFn from PubsubIO.Write
[kirpichov] Introduces read/writePubsubMessages
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam5 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/*:refs/remotes/origin/pr/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 3b61f6a9034b3dbb8b08f695b7b526656660c8a6 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 3b61f6a9034b3dbb8b08f695b7b526656660c8a6
> git rev-list dc0fdcb7e62067fdc629b8558659fce7e37e4856 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Dataflow] $ /bin/bash -xe /tmp/hudson5223032132045342938.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Dataflow] $ /bin/bash -xe /tmp/hudson3069112977640495240.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Dataflow] $ /bin/bash -xe /tmp/hudson2562948682559758564.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied (use --upgrade to upgrade): python-gflags==3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied (use --upgrade to upgrade): jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Installing extra requirements: 'windows'
Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied (use --upgrade to upgrade): futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.11 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied (use --upgrade to upgrade): pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied (use --upgrade to upgrade): numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied (use --upgrade to upgrade): functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied (use --upgrade to upgrade): contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Cleaning up...
[beam_PerformanceTests_Dataflow] $ /bin/bash -xe /tmp/hudson4871808138669294515.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --benchmarks=dpb_wordcount_benchmark --dpb_dataflow_staging_location=gs://temp-storage-for-perf-tests/staging --dpb_wordcount_input=dataflow-samples/shakespeare/kinglear.txt --config_override=dpb_wordcount_benchmark.dpb_service.service_type=dataflow
WARNING:root:File resource loader root perfkitbenchmarker/data/ycsb is not a directory.
2017-05-03 12:00:18,040 00eb0167 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/00eb0167/pkb.log
2017-05-03 12:00:18,041 00eb0167 MainThread INFO PerfKitBenchmarker version: v1.11.0-48-ge557505
2017-05-03 12:00:18,041 00eb0167 MainThread INFO Flag values:
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--dpb_wordcount_input=dataflow-samples/shakespeare/kinglear.txt
--dpb_log_level=INFO
--official
--benchmarks=dpb_wordcount_benchmark
--config_override=dpb_wordcount_benchmark.dpb_service.service_type=dataflow
--dpb_dataflow_staging_location=gs://temp-storage-for-perf-tests/staging
2017-05-03 12:00:18,338 00eb0167 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-05-03 12:00:18,436 00eb0167 MainThread dpb_wordcount_benchmark(1/1) INFO Provisioning resources for benchmark dpb_wordcount_benchmark
2017-05-03 12:00:18,439 00eb0167 MainThread dpb_wordcount_benchmark(1/1) INFO Preparing benchmark dpb_wordcount_benchmark
2017-05-03 12:00:18,439 00eb0167 MainThread dpb_wordcount_benchmark(1/1) INFO Running benchmark dpb_wordcount_benchmark
2017-05-03 12:00:18,445 00eb0167 MainThread dpb_wordcount_benchmark(1/1) ERROR Error during benchmark dpb_wordcount_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 543, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 459, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/dpb_wordcount_benchmark.py",> line 162, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 115, in SubmitJob
stdout, _, _ = vm_util.IssueCommand(cmd)
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/vm_util.py",> line 287, in IssueCommand
full_cmd = ' '.join(cmd)
TypeError: sequence item 2: expected string, NoneType found
2017-05-03 12:00:18,448 00eb0167 MainThread dpb_wordcount_benchmark(1/1) ERROR Benchmark 1/1 dpb_wordcount_benchmark (UID: dpb_wordcount_benchmark0) failed. Execution will continue.
2017-05-03 12:00:18,467 00eb0167 MainThread INFO Benchmark run statuses:
---------------------------------------------------------
Name UID Status
---------------------------------------------------------
dpb_wordcount_benchmark dpb_wordcount_benchmark0 FAILED
---------------------------------------------------------
Success rate: 0.00% (0/1)
2017-05-03 12:00:18,468 00eb0167 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/00eb0167/pkb.log
2017-05-03 12:00:18,468 00eb0167 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/00eb0167/completion_statuses.json
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PerformanceTests_Dataflow #366
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Dataflow/366/display/redirect?page=changes>
Changes:
[robertwb] Deprecate <pipeline>.options usage
[lcwik] Remove Coder.asCloudObject and related methods
[altay] Updating Dataflow API protos and client
[tgroh] Add Additional CloudObjectTranslators
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam5 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/*:refs/remotes/origin/pr/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision dc0fdcb7e62067fdc629b8558659fce7e37e4856 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f dc0fdcb7e62067fdc629b8558659fce7e37e4856
> git rev-list 3bb0f8e6ad8a2b378a656731732a79c9c16e4a8d # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Dataflow] $ /bin/bash -xe /tmp/hudson4609501167634798019.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Dataflow] $ /bin/bash -xe /tmp/hudson9117219116175138012.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Dataflow] $ /bin/bash -xe /tmp/hudson380355071913316710.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied (use --upgrade to upgrade): python-gflags==3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied (use --upgrade to upgrade): jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Installing extra requirements: 'windows'
Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied (use --upgrade to upgrade): futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.11 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied (use --upgrade to upgrade): pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied (use --upgrade to upgrade): numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied (use --upgrade to upgrade): functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied (use --upgrade to upgrade): contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Cleaning up...
[beam_PerformanceTests_Dataflow] $ /bin/bash -xe /tmp/hudson6791561348024361129.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --benchmarks=dpb_wordcount_benchmark --dpb_dataflow_staging_location=gs://temp-storage-for-perf-tests/staging --dpb_wordcount_input=dataflow-samples/shakespeare/kinglear.txt --config_override=dpb_wordcount_benchmark.dpb_service.service_type=dataflow
WARNING:root:File resource loader root perfkitbenchmarker/data/ycsb is not a directory.
2017-05-03 06:00:33,733 714e3efe MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/714e3efe/pkb.log
2017-05-03 06:00:33,733 714e3efe MainThread INFO PerfKitBenchmarker version: v1.11.0-48-ge557505
2017-05-03 06:00:33,734 714e3efe MainThread INFO Flag values:
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--dpb_wordcount_input=dataflow-samples/shakespeare/kinglear.txt
--dpb_log_level=INFO
--official
--benchmarks=dpb_wordcount_benchmark
--config_override=dpb_wordcount_benchmark.dpb_service.service_type=dataflow
--dpb_dataflow_staging_location=gs://temp-storage-for-perf-tests/staging
2017-05-03 06:00:33,998 714e3efe MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-05-03 06:00:34,082 714e3efe MainThread dpb_wordcount_benchmark(1/1) INFO Provisioning resources for benchmark dpb_wordcount_benchmark
2017-05-03 06:00:34,084 714e3efe MainThread dpb_wordcount_benchmark(1/1) INFO Preparing benchmark dpb_wordcount_benchmark
2017-05-03 06:00:34,085 714e3efe MainThread dpb_wordcount_benchmark(1/1) INFO Running benchmark dpb_wordcount_benchmark
2017-05-03 06:00:34,090 714e3efe MainThread dpb_wordcount_benchmark(1/1) ERROR Error during benchmark dpb_wordcount_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 543, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 459, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/dpb_wordcount_benchmark.py",> line 162, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 115, in SubmitJob
stdout, _, _ = vm_util.IssueCommand(cmd)
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/vm_util.py",> line 287, in IssueCommand
full_cmd = ' '.join(cmd)
TypeError: sequence item 2: expected string, NoneType found
2017-05-03 06:00:34,092 714e3efe MainThread dpb_wordcount_benchmark(1/1) ERROR Benchmark 1/1 dpb_wordcount_benchmark (UID: dpb_wordcount_benchmark0) failed. Execution will continue.
2017-05-03 06:00:34,119 714e3efe MainThread INFO Benchmark run statuses:
---------------------------------------------------------
Name UID Status
---------------------------------------------------------
dpb_wordcount_benchmark dpb_wordcount_benchmark0 FAILED
---------------------------------------------------------
Success rate: 0.00% (0/1)
2017-05-03 06:00:34,119 714e3efe MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/714e3efe/pkb.log
2017-05-03 06:00:34,120 714e3efe MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/714e3efe/completion_statuses.json
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PerformanceTests_Dataflow #365
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Dataflow/365/display/redirect?page=changes>
Changes:
[klk] Use processing time for synchronized processing time in Flink runner
[klk] Simplify type parameters of StateSpec and related
[klk] Make DoFnSignatures robust to StateSpec subclasses
[kirpichov] Removes leftover no-op withoutValidation methods
[kirpichov] Removes TextIO.Read.Bound
[kirpichov] Converts TextIO.Read to AutoValue
[kirpichov] Removes TextIO.Write.Bound
[kirpichov] Converts TextIO.Write to AutoValue
[kirpichov] Moves TextSource and TextSink to top level
[lcwik] [BEAM-2135] Move hdfs to hadoop-file-system
[dhalperi] [BEAM-2141] Disable JDBC tests
[dhalperi] BigQuery: swap from asSingleton to asIterable for Cleanup
[lcwik] [BEAM-2020] Convert all unknown Coders into CustomCoder CloudObjects
[chamikara] [BEAM-539] Fixes several issues of FileSink.
[altay] [BEAM-1316] Remove the usage of mock from ptransform tests
[lcwik] [BEAM-2135] Fix pointers to GCP-core
[klk] Skip null options when converting back to argv
[klk] Instantiate runner briefly in Pipeline
[lcwik] [BEAM-2135] Fix pointers to sdks/java/io/hadoop-file-system
[davor] [BEAM-2093] Use the jackson version from the maven property in maven
[dhalperi] [BEAM-59] Delete old restrictions on output file paths
[lcwik] [BEAM-2139] Disable SplittableDoFn ValidatesRunner tests for Streaming
[dhalperi] [BEAM-59] Move GcsFileSystem to gcp-core
[dhalperi] Full removal of Aggregators in Java SDK and Runners
[davor] Include LICENSE and NOTICE in python dist files
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam1 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/*:refs/remotes/origin/pr/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 3bb0f8e6ad8a2b378a656731732a79c9c16e4a8d (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 3bb0f8e6ad8a2b378a656731732a79c9c16e4a8d
> git rev-list eec903fb538eb9c3d41d737dba94a41f7c1c469e # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Dataflow] $ /bin/bash -xe /tmp/hudson4423006436169070601.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Dataflow] $ /bin/bash -xe /tmp/hudson5338004820536756484.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Dataflow] $ /bin/bash -xe /tmp/hudson1500182718279178360.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied (use --upgrade to upgrade): python-gflags==3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied (use --upgrade to upgrade): jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Installing extra requirements: 'windows'
Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied (use --upgrade to upgrade): futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.11 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied (use --upgrade to upgrade): pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied (use --upgrade to upgrade): numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied (use --upgrade to upgrade): functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied (use --upgrade to upgrade): contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Cleaning up...
[beam_PerformanceTests_Dataflow] $ /bin/bash -xe /tmp/hudson5189078248202599501.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --benchmarks=dpb_wordcount_benchmark --dpb_dataflow_staging_location=gs://temp-storage-for-perf-tests/staging --dpb_wordcount_input=dataflow-samples/shakespeare/kinglear.txt --config_override=dpb_wordcount_benchmark.dpb_service.service_type=dataflow
WARNING:root:File resource loader root perfkitbenchmarker/data/ycsb is not a directory.
2017-05-03 00:02:38,285 bc160be8 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/bc160be8/pkb.log
2017-05-03 00:02:38,285 bc160be8 MainThread INFO PerfKitBenchmarker version: v1.11.0-48-ge557505
2017-05-03 00:02:38,286 bc160be8 MainThread INFO Flag values:
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--dpb_wordcount_input=dataflow-samples/shakespeare/kinglear.txt
--dpb_log_level=INFO
--official
--benchmarks=dpb_wordcount_benchmark
--config_override=dpb_wordcount_benchmark.dpb_service.service_type=dataflow
--dpb_dataflow_staging_location=gs://temp-storage-for-perf-tests/staging
2017-05-03 00:02:38,833 bc160be8 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-05-03 00:02:38,932 bc160be8 MainThread dpb_wordcount_benchmark(1/1) INFO Provisioning resources for benchmark dpb_wordcount_benchmark
2017-05-03 00:02:38,934 bc160be8 MainThread dpb_wordcount_benchmark(1/1) INFO Preparing benchmark dpb_wordcount_benchmark
2017-05-03 00:02:38,934 bc160be8 MainThread dpb_wordcount_benchmark(1/1) INFO Running benchmark dpb_wordcount_benchmark
2017-05-03 00:02:38,939 bc160be8 MainThread dpb_wordcount_benchmark(1/1) ERROR Error during benchmark dpb_wordcount_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 543, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 459, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/dpb_wordcount_benchmark.py",> line 162, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 115, in SubmitJob
stdout, _, _ = vm_util.IssueCommand(cmd)
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/vm_util.py",> line 287, in IssueCommand
full_cmd = ' '.join(cmd)
TypeError: sequence item 2: expected string, NoneType found
2017-05-03 00:02:38,941 bc160be8 MainThread dpb_wordcount_benchmark(1/1) ERROR Benchmark 1/1 dpb_wordcount_benchmark (UID: dpb_wordcount_benchmark0) failed. Execution will continue.
2017-05-03 00:02:38,962 bc160be8 MainThread INFO Benchmark run statuses:
---------------------------------------------------------
Name UID Status
---------------------------------------------------------
dpb_wordcount_benchmark dpb_wordcount_benchmark0 FAILED
---------------------------------------------------------
Success rate: 0.00% (0/1)
2017-05-03 00:02:38,963 bc160be8 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/bc160be8/pkb.log
2017-05-03 00:02:38,963 bc160be8 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/bc160be8/completion_statuses.json
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PerformanceTests_Dataflow #364
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Dataflow/364/display/redirect?page=changes>
Changes:
[wtanaka] Add comments to Trigger.java
[iemejia] [BEAM-1764] Remove aggregators from Flink Runner
[davor] Update github PR template after travis removal.
[altay] Add retry logic for job creation
[tgroh] Add an @Internal Annotation
[robertwb] Revert clean else-return lint changes.
[lcwik] [BEAM-2135] Move gcp-core to google-cloud-platform-core
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam4 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/*:refs/remotes/origin/pr/*
ERROR: Error fetching remote repo 'origin'
hudson.plugins.git.GitException: Failed to fetch from https://github.com/apache/beam.git
at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:809)
at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1076)
at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1107)
at hudson.scm.SCM.checkout(SCM.java:496)
at hudson.model.AbstractProject.checkout(AbstractProject.java:1281)
at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:604)
at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:529)
at hudson.model.Run.execute(Run.java:1728)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:98)
at hudson.model.Executor.run(Executor.java:405)
Caused by: hudson.plugins.git.GitException: Command "git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/*:refs/remotes/origin/pr/*" returned status code 128:
stdout:
stderr: error: RPC failed; result=18, HTTP code = 200
fatal: The remote end hung up unexpectedly
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1866)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1585)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$300(CliGitAPIImpl.java:71)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:348)
at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:153)
at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:146)
at hudson.remoting.UserRequest.perform(UserRequest.java:153)
at hudson.remoting.UserRequest.perform(UserRequest.java:50)
at hudson.remoting.Request$2.run(Request.java:336)
at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
at ......remote call to beam4(Native Method)
at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1545)
at hudson.remoting.UserResponse.retrieve(UserRequest.java:253)
at hudson.remoting.Channel.call(Channel.java:830)
at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:146)
at sun.reflect.GeneratedMethodAccessor788.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:132)
at com.sun.proxy.$Proxy103.execute(Unknown Source)
at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:807)
... 11 more
ERROR: null
Retrying after 10 seconds
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/*:refs/remotes/origin/pr/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision eec903fb538eb9c3d41d737dba94a41f7c1c469e (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f eec903fb538eb9c3d41d737dba94a41f7c1c469e
> git rev-list ae72456b7125a28f2db4b8dad2990b2bb063ca9b # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Dataflow] $ /bin/bash -xe /tmp/hudson8706026561929483622.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Dataflow] $ /bin/bash -xe /tmp/hudson425751425269746362.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Dataflow] $ /bin/bash -xe /tmp/hudson4441358561143021270.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied (use --upgrade to upgrade): python-gflags==3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied (use --upgrade to upgrade): jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Installing extra requirements: 'windows'
Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied (use --upgrade to upgrade): futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.11 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied (use --upgrade to upgrade): pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied (use --upgrade to upgrade): numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied (use --upgrade to upgrade): functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied (use --upgrade to upgrade): contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Cleaning up...
[beam_PerformanceTests_Dataflow] $ /bin/bash -xe /tmp/hudson1024427901516202230.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --benchmarks=dpb_wordcount_benchmark --dpb_dataflow_staging_location=gs://temp-storage-for-perf-tests/staging --dpb_wordcount_input=dataflow-samples/shakespeare/kinglear.txt --config_override=dpb_wordcount_benchmark.dpb_service.service_type=dataflow
WARNING:root:File resource loader root perfkitbenchmarker/data/ycsb is not a directory.
2017-05-02 18:25:21,716 554ad8e1 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/554ad8e1/pkb.log
2017-05-02 18:25:21,718 554ad8e1 MainThread INFO PerfKitBenchmarker version: v1.11.0-47-gb38dddb
2017-05-02 18:25:21,719 554ad8e1 MainThread INFO Flag values:
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--dpb_wordcount_input=dataflow-samples/shakespeare/kinglear.txt
--dpb_log_level=INFO
--official
--benchmarks=dpb_wordcount_benchmark
--config_override=dpb_wordcount_benchmark.dpb_service.service_type=dataflow
--dpb_dataflow_staging_location=gs://temp-storage-for-perf-tests/staging
2017-05-02 18:25:22,240 554ad8e1 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-05-02 18:25:22,450 554ad8e1 MainThread dpb_wordcount_benchmark(1/1) INFO Provisioning resources for benchmark dpb_wordcount_benchmark
2017-05-02 18:25:22,454 554ad8e1 MainThread dpb_wordcount_benchmark(1/1) INFO Preparing benchmark dpb_wordcount_benchmark
2017-05-02 18:25:22,454 554ad8e1 MainThread dpb_wordcount_benchmark(1/1) INFO Running benchmark dpb_wordcount_benchmark
2017-05-02 18:25:22,462 554ad8e1 MainThread dpb_wordcount_benchmark(1/1) ERROR Error during benchmark dpb_wordcount_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 543, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 459, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/dpb_wordcount_benchmark.py",> line 162, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 115, in SubmitJob
stdout, _, _ = vm_util.IssueCommand(cmd)
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/vm_util.py",> line 287, in IssueCommand
full_cmd = ' '.join(cmd)
TypeError: sequence item 2: expected string, NoneType found
2017-05-02 18:25:22,465 554ad8e1 MainThread dpb_wordcount_benchmark(1/1) ERROR Benchmark 1/1 dpb_wordcount_benchmark (UID: dpb_wordcount_benchmark0) failed. Execution will continue.
2017-05-02 18:25:22,509 554ad8e1 MainThread INFO Benchmark run statuses:
---------------------------------------------------------
Name UID Status
---------------------------------------------------------
dpb_wordcount_benchmark dpb_wordcount_benchmark0 FAILED
---------------------------------------------------------
Success rate: 0.00% (0/1)
2017-05-02 18:25:22,510 554ad8e1 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/554ad8e1/pkb.log
2017-05-02 18:25:22,510 554ad8e1 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/554ad8e1/completion_statuses.json
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PerformanceTests_Dataflow #363
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Dataflow/363/display/redirect?page=changes>
Changes:
[dhalperi] Remove Aggregators from StatefulDoFn runner
[dhalperi] Removing Aggregator from core runner code
[dhalperi] Remove accumulators from DoFn tester.
[jbonofre] Remove unneeded elements from the POM files
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam5 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/*:refs/remotes/origin/pr/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision ae72456b7125a28f2db4b8dad2990b2bb063ca9b (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f ae72456b7125a28f2db4b8dad2990b2bb063ca9b
> git rev-list e92ead58cd661ff7ee952278f2983a578bdd2412 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Dataflow] $ /bin/bash -xe /tmp/hudson8808595505571951135.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Dataflow] $ /bin/bash -xe /tmp/hudson5575995026223943279.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Dataflow] $ /bin/bash -xe /tmp/hudson4231598413718564875.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied (use --upgrade to upgrade): python-gflags==3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied (use --upgrade to upgrade): jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Installing extra requirements: 'windows'
Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied (use --upgrade to upgrade): futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.11 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied (use --upgrade to upgrade): pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied (use --upgrade to upgrade): numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied (use --upgrade to upgrade): functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied (use --upgrade to upgrade): contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Cleaning up...
[beam_PerformanceTests_Dataflow] $ /bin/bash -xe /tmp/hudson4168573447482597019.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --benchmarks=dpb_wordcount_benchmark --dpb_dataflow_staging_location=gs://temp-storage-for-perf-tests/staging --dpb_wordcount_input=dataflow-samples/shakespeare/kinglear.txt --config_override=dpb_wordcount_benchmark.dpb_service.service_type=dataflow
WARNING:root:File resource loader root perfkitbenchmarker/data/ycsb is not a directory.
2017-05-02 12:00:25,291 c5c7fb81 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/c5c7fb81/pkb.log
2017-05-02 12:00:25,291 c5c7fb81 MainThread INFO PerfKitBenchmarker version: v1.11.0-47-gb38dddb
2017-05-02 12:00:25,292 c5c7fb81 MainThread INFO Flag values:
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--dpb_wordcount_input=dataflow-samples/shakespeare/kinglear.txt
--dpb_log_level=INFO
--official
--benchmarks=dpb_wordcount_benchmark
--config_override=dpb_wordcount_benchmark.dpb_service.service_type=dataflow
--dpb_dataflow_staging_location=gs://temp-storage-for-perf-tests/staging
2017-05-02 12:00:25,623 c5c7fb81 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-05-02 12:00:25,715 c5c7fb81 MainThread dpb_wordcount_benchmark(1/1) INFO Provisioning resources for benchmark dpb_wordcount_benchmark
2017-05-02 12:00:25,717 c5c7fb81 MainThread dpb_wordcount_benchmark(1/1) INFO Preparing benchmark dpb_wordcount_benchmark
2017-05-02 12:00:25,718 c5c7fb81 MainThread dpb_wordcount_benchmark(1/1) INFO Running benchmark dpb_wordcount_benchmark
2017-05-02 12:00:25,723 c5c7fb81 MainThread dpb_wordcount_benchmark(1/1) ERROR Error during benchmark dpb_wordcount_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 543, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 459, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/dpb_wordcount_benchmark.py",> line 162, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 115, in SubmitJob
stdout, _, _ = vm_util.IssueCommand(cmd)
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/vm_util.py",> line 287, in IssueCommand
full_cmd = ' '.join(cmd)
TypeError: sequence item 2: expected string, NoneType found
2017-05-02 12:00:25,726 c5c7fb81 MainThread dpb_wordcount_benchmark(1/1) ERROR Benchmark 1/1 dpb_wordcount_benchmark (UID: dpb_wordcount_benchmark0) failed. Execution will continue.
2017-05-02 12:00:25,746 c5c7fb81 MainThread INFO Benchmark run statuses:
---------------------------------------------------------
Name UID Status
---------------------------------------------------------
dpb_wordcount_benchmark dpb_wordcount_benchmark0 FAILED
---------------------------------------------------------
Success rate: 0.00% (0/1)
2017-05-02 12:00:25,746 c5c7fb81 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/c5c7fb81/pkb.log
2017-05-02 12:00:25,747 c5c7fb81 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/c5c7fb81/completion_statuses.json
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PerformanceTests_Dataflow #362
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Dataflow/362/display/redirect?page=changes>
Changes:
[robertwb] Fn API support for Python.
[robertwb] Rename OutputValue to TaggedOutput.
[lcwik] Copy CloudObject to the Dataflow Module
[lcwik] [BEAM-1871] Remove deprecated org.apache.beam.sdk.options.GcsOptions
[kirpichov] Fixes javadoc of TextIO to not point to AvroIO
[kirpichov] Moves AvroSink to upper level
[kirpichov] Removes AvroIO.Read.Bound
[kirpichov] Adds AvroIO.readGenericRecords()
[kirpichov] Converts AvroIO.Read to AutoValue
[kirpichov] Removes AvroIO.Write.Bound
[kirpichov] Moves AvroIO.Read.withSchema into read()
[kirpichov] Converts AvroIO.Write to AutoValue; adds writeGenericRecords()
[kirpichov] Moves AvroIO.write().withSchema into write()
[kirpichov] Scattered minor improvements per review comments
[dhalperi] maptask_executor_runner_test: build fix
[lcwik] [BEAM-2005, BEAM-2030, BEAM-2031, BEAM-2032, BEAM-2033, BEAM-2070] Base
[lcwik] [BEAM-2005] Fix build break, ignore test due to change in
[lcwik] Remove aggregators from DoFn contexts and internal SDK usage
[dhalperi] [BEAM-59] Core tests: stop using gs:// paths
[dhalperi] [BEAM-59] TFRecordIOTest: cleanup
[dhalperi] [BEAM-59] DataflowRunnerTests: configure FileSystems in test
[dhalperi] [BEAM-59] AvroIOTest: use absolute paths for display data
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam5 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/*:refs/remotes/origin/pr/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision e92ead58cd661ff7ee952278f2983a578bdd2412 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f e92ead58cd661ff7ee952278f2983a578bdd2412
> git rev-list b8131fe935d00db7f760b6410a4eb2239db1d25a # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Dataflow] $ /bin/bash -xe /tmp/hudson7353765744138912579.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Dataflow] $ /bin/bash -xe /tmp/hudson5378899830644152793.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Dataflow] $ /bin/bash -xe /tmp/hudson3104503862056589697.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied (use --upgrade to upgrade): python-gflags==3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied (use --upgrade to upgrade): jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Installing extra requirements: 'windows'
Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied (use --upgrade to upgrade): futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.11 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied (use --upgrade to upgrade): pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied (use --upgrade to upgrade): numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied (use --upgrade to upgrade): functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied (use --upgrade to upgrade): contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Cleaning up...
[beam_PerformanceTests_Dataflow] $ /bin/bash -xe /tmp/hudson6892438915894804756.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --benchmarks=dpb_wordcount_benchmark --dpb_dataflow_staging_location=gs://temp-storage-for-perf-tests/staging --dpb_wordcount_input=dataflow-samples/shakespeare/kinglear.txt --config_override=dpb_wordcount_benchmark.dpb_service.service_type=dataflow
WARNING:root:File resource loader root perfkitbenchmarker/data/ycsb is not a directory.
2017-05-02 06:00:32,557 3603e1c1 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/3603e1c1/pkb.log
2017-05-02 06:00:32,558 3603e1c1 MainThread INFO PerfKitBenchmarker version: v1.11.0-47-gb38dddb
2017-05-02 06:00:32,558 3603e1c1 MainThread INFO Flag values:
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--dpb_wordcount_input=dataflow-samples/shakespeare/kinglear.txt
--dpb_log_level=INFO
--official
--benchmarks=dpb_wordcount_benchmark
--config_override=dpb_wordcount_benchmark.dpb_service.service_type=dataflow
--dpb_dataflow_staging_location=gs://temp-storage-for-perf-tests/staging
2017-05-02 06:00:32,904 3603e1c1 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-05-02 06:00:32,994 3603e1c1 MainThread dpb_wordcount_benchmark(1/1) INFO Provisioning resources for benchmark dpb_wordcount_benchmark
2017-05-02 06:00:32,997 3603e1c1 MainThread dpb_wordcount_benchmark(1/1) INFO Preparing benchmark dpb_wordcount_benchmark
2017-05-02 06:00:32,997 3603e1c1 MainThread dpb_wordcount_benchmark(1/1) INFO Running benchmark dpb_wordcount_benchmark
2017-05-02 06:00:33,002 3603e1c1 MainThread dpb_wordcount_benchmark(1/1) ERROR Error during benchmark dpb_wordcount_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 543, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 459, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/dpb_wordcount_benchmark.py",> line 162, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 115, in SubmitJob
stdout, _, _ = vm_util.IssueCommand(cmd)
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/vm_util.py",> line 287, in IssueCommand
full_cmd = ' '.join(cmd)
TypeError: sequence item 2: expected string, NoneType found
2017-05-02 06:00:33,004 3603e1c1 MainThread dpb_wordcount_benchmark(1/1) ERROR Benchmark 1/1 dpb_wordcount_benchmark (UID: dpb_wordcount_benchmark0) failed. Execution will continue.
2017-05-02 06:00:33,031 3603e1c1 MainThread INFO Benchmark run statuses:
---------------------------------------------------------
Name UID Status
---------------------------------------------------------
dpb_wordcount_benchmark dpb_wordcount_benchmark0 FAILED
---------------------------------------------------------
Success rate: 0.00% (0/1)
2017-05-02 06:00:33,032 3603e1c1 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/3603e1c1/pkb.log
2017-05-02 06:00:33,032 3603e1c1 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/3603e1c1/completion_statuses.json
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PerformanceTests_Dataflow #361
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Dataflow/361/display/redirect?page=changes>
Changes:
[klk] Do not repeat log messages in DataflowPipelineJob
[tgroh] Add A CoderTranslator Interface
[lcwik] Re-enable WindowedWordCountIT on Flink runner in precommit
[tgroh] Add Cloud Object Translators for Coders
[dhalperi] [BEAM-1676] Internalize ClassPath from Guava 21.0 to fix the failures
[lcwik] [BEAM-1871] Hide CloudResourceManager client in
[chamikara] [BEAM-1925] Remove deprecated context param from DoFn
[dhalperi] Remove IOChannelUtils from BigQuery TableRowWriter
[dhalperi] [BEAM-59] Switch mimeType from mutable protected field to constructor
[lcwik] Add CloudObjectTranslators for Avro, Serializable
[altay] [BEAM-1441] remove manual join from apiclient
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam1 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/*:refs/remotes/origin/pr/*
ERROR: Error fetching remote repo 'origin'
hudson.plugins.git.GitException: Failed to fetch from https://github.com/apache/beam.git
at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:809)
at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1076)
at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1107)
at hudson.scm.SCM.checkout(SCM.java:496)
at hudson.model.AbstractProject.checkout(AbstractProject.java:1281)
at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:604)
at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:529)
at hudson.model.Run.execute(Run.java:1728)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:98)
at hudson.model.Executor.run(Executor.java:405)
Caused by: hudson.plugins.git.GitException: Command "git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/*:refs/remotes/origin/pr/*" returned status code 128:
stdout:
stderr: error: RPC failed; result=18, HTTP code = 200
fatal: The remote end hung up unexpectedly
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1866)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1585)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$300(CliGitAPIImpl.java:71)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:348)
at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:153)
at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:146)
at hudson.remoting.UserRequest.perform(UserRequest.java:153)
at hudson.remoting.UserRequest.perform(UserRequest.java:50)
at hudson.remoting.Request$2.run(Request.java:336)
at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
at ......remote call to beam1(Native Method)
at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1545)
at hudson.remoting.UserResponse.retrieve(UserRequest.java:253)
at hudson.remoting.Channel.call(Channel.java:830)
at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:146)
at sun.reflect.GeneratedMethodAccessor788.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:132)
at com.sun.proxy.$Proxy103.execute(Unknown Source)
at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:807)
... 11 more
ERROR: null
Retrying after 10 seconds
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/*:refs/remotes/origin/pr/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision b8131fe935d00db7f760b6410a4eb2239db1d25a (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f b8131fe935d00db7f760b6410a4eb2239db1d25a
> git rev-list fe2a32000be87bbed1e8ae5a81a8e0368f0b0234 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Dataflow] $ /bin/bash -xe /tmp/hudson6960404443284401470.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Dataflow] $ /bin/bash -xe /tmp/hudson6555700586416662445.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Dataflow] $ /bin/bash -xe /tmp/hudson6545264446819990109.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied (use --upgrade to upgrade): python-gflags==3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied (use --upgrade to upgrade): jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Installing extra requirements: 'windows'
Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied (use --upgrade to upgrade): futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.11 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied (use --upgrade to upgrade): pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied (use --upgrade to upgrade): numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied (use --upgrade to upgrade): functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied (use --upgrade to upgrade): contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Cleaning up...
[beam_PerformanceTests_Dataflow] $ /bin/bash -xe /tmp/hudson3892156480876229696.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --benchmarks=dpb_wordcount_benchmark --dpb_dataflow_staging_location=gs://temp-storage-for-perf-tests/staging --dpb_wordcount_input=dataflow-samples/shakespeare/kinglear.txt --config_override=dpb_wordcount_benchmark.dpb_service.service_type=dataflow
WARNING:root:File resource loader root perfkitbenchmarker/data/ycsb is not a directory.
2017-05-02 00:05:08,036 b3752e07 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/b3752e07/pkb.log
2017-05-02 00:05:08,036 b3752e07 MainThread INFO PerfKitBenchmarker version: v1.11.0-47-gb38dddb
2017-05-02 00:05:08,036 b3752e07 MainThread INFO Flag values:
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--dpb_wordcount_input=dataflow-samples/shakespeare/kinglear.txt
--dpb_log_level=INFO
--official
--benchmarks=dpb_wordcount_benchmark
--config_override=dpb_wordcount_benchmark.dpb_service.service_type=dataflow
--dpb_dataflow_staging_location=gs://temp-storage-for-perf-tests/staging
2017-05-02 00:05:08,480 b3752e07 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-05-02 00:05:08,569 b3752e07 MainThread dpb_wordcount_benchmark(1/1) INFO Provisioning resources for benchmark dpb_wordcount_benchmark
2017-05-02 00:05:08,571 b3752e07 MainThread dpb_wordcount_benchmark(1/1) INFO Preparing benchmark dpb_wordcount_benchmark
2017-05-02 00:05:08,572 b3752e07 MainThread dpb_wordcount_benchmark(1/1) INFO Running benchmark dpb_wordcount_benchmark
2017-05-02 00:05:08,577 b3752e07 MainThread dpb_wordcount_benchmark(1/1) ERROR Error during benchmark dpb_wordcount_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 543, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 459, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/dpb_wordcount_benchmark.py",> line 162, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 115, in SubmitJob
stdout, _, _ = vm_util.IssueCommand(cmd)
File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/vm_util.py",> line 287, in IssueCommand
full_cmd = ' '.join(cmd)
TypeError: sequence item 2: expected string, NoneType found
2017-05-02 00:05:08,586 b3752e07 MainThread dpb_wordcount_benchmark(1/1) ERROR Benchmark 1/1 dpb_wordcount_benchmark (UID: dpb_wordcount_benchmark0) failed. Execution will continue.
2017-05-02 00:05:08,605 b3752e07 MainThread INFO Benchmark run statuses:
---------------------------------------------------------
Name UID Status
---------------------------------------------------------
dpb_wordcount_benchmark dpb_wordcount_benchmark0 FAILED
---------------------------------------------------------
Success rate: 0.00% (0/1)
2017-05-02 00:05:08,605 b3752e07 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/b3752e07/pkb.log
2017-05-02 00:05:08,606 b3752e07 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/b3752e07/completion_statuses.json
Build step 'Execute shell' marked build as failure