You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2018/04/29 12:01:49 UTC
Build failed in Jenkins: beam_PerformanceTests_Python #1207
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1207/display/redirect>
------------------------------------------
[...truncated 72.94 KB...]
[INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (enforce-banned-dependencies) @ beam-sdks-go-container ---
[INFO]
[INFO] --- maven-dependency-plugin:3.0.2:unpack (copy-dependency) @ beam-sdks-go-container ---
[INFO] Configured Artifact: org.apache.beam:beam-sdks-go:pkg-sources:2.5.0-SNAPSHOT:zip
[INFO] Unpacking <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/7e877a07/beam/sdks/go/target/beam-sdks-go-2.5.0-SNAPSHOT-pkg-sources.zip> to <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/7e877a07/beam/sdks/go/container/target/src/github.com/apache/beam/sdks/go> with includes "" and excludes ""
[INFO]
[INFO] --- maven-resources-plugin:3.0.2:copy-resources (copy-go-pkg-source) @ beam-sdks-go-container ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/7e877a07/beam/sdks/go/container/pkg>
[INFO]
[INFO] --- maven-resources-plugin:3.0.2:copy-resources (copy-go-cmd-source) @ beam-sdks-go-container ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO]
[INFO] --- maven-remote-resources-plugin:1.5:process (process-resource-bundles) @ beam-sdks-go-container ---
[INFO]
[INFO] --- mvn-golang-wrapper:2.1.7:get (go-get-imports) @ beam-sdks-go-container ---
[INFO] Prepared command line : bin/go get -u google.golang.org/grpc golang.org/x/oauth2/google google.golang.org/api/storage/v1
[ERROR]
[ERROR] ---------Exec.Err---------
[ERROR] # cd /home/jenkins/.mvnGoLang/.go_path/src/github.com/golang/protobuf; git pull --ff-only
[ERROR] Your configuration specifies to merge with the ref 'refs/heads/master'
[ERROR] from the remote, but no such ref was fetched.
[ERROR] package github.com/golang/protobuf/proto: exit status 1
[ERROR]
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Beam :: Parent .............................. SUCCESS [ 2.472 s]
[INFO] Apache Beam :: SDKs :: Java :: Build Tools ......... SUCCESS [ 1.962 s]
[INFO] Apache Beam :: Model ............................... SUCCESS [ 0.029 s]
[INFO] Apache Beam :: Model :: Pipeline ................... SUCCESS [ 7.329 s]
[INFO] Apache Beam :: Model :: Job Management ............. SUCCESS [ 3.223 s]
[INFO] Apache Beam :: Model :: Fn Execution ............... SUCCESS [ 3.623 s]
[INFO] Apache Beam :: SDKs ................................ SUCCESS [ 0.105 s]
[INFO] Apache Beam :: SDKs :: Go .......................... SUCCESS [ 23.420 s]
[INFO] Apache Beam :: SDKs :: Go :: Container ............. FAILURE [ 14.522 s]
[INFO] Apache Beam :: SDKs :: Java ........................ SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Core ................ SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Fn Execution ........ SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions .......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Google Cloud Platform Core SKIPPED
[INFO] Apache Beam :: Runners ............................. SKIPPED
[INFO] Apache Beam :: Runners :: Core Construction Java ... SKIPPED
[INFO] Apache Beam :: Runners :: Core Java ................ SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Harness ............. SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Container ........... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO .................. SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Amazon Web Services SKIPPED
[INFO] Apache Beam :: Runners :: Local Java Core .......... SKIPPED
[INFO] Apache Beam :: Runners :: Direct Java .............. SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: AMQP .......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Common ........ SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Cassandra ..... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch . SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: Common SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: 2.x SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: 5.x SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: XML ........... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Protobuf SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Google Cloud Platform SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Dataflow .... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: File-based-io-tests SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop Common . SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop File System SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: JDBC .......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop Input Format SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: HBase ......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: HCatalog ...... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: JMS ........... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Kafka ......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Kinesis ....... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: MongoDB ....... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: MQTT .......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Redis ......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Solr .......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Tika .......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes .... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Starter SKIPPED
[INFO] Apache Beam :: Examples ............................ SKIPPED
[INFO] Apache Beam :: Examples :: Java .................... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Examples SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Jackson SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Join library SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Sketching SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Sorter SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: SQL ... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Nexmark ............. SKIPPED
[INFO] Apache Beam :: SDKs :: Python ...................... SKIPPED
[INFO] Apache Beam :: SDKs :: Python :: Container ......... SKIPPED
[INFO] Apache Beam :: Runners :: Java Fn Execution ........ SKIPPED
[INFO] Apache Beam :: Runners :: Java Local Artifact Service SKIPPED
[INFO] Apache Beam :: Runners :: Reference ................ SKIPPED
[INFO] Apache Beam :: Runners :: Reference :: Java ........ SKIPPED
[INFO] Apache Beam :: Runners :: Reference :: Job Orchestrator SKIPPED
[INFO] Apache Beam :: Runners :: Flink .................... SKIPPED
[INFO] Apache Beam :: Runners :: Gearpump ................. SKIPPED
[INFO] Apache Beam :: Runners :: Spark .................... SKIPPED
[INFO] Apache Beam :: Runners :: Apex ..................... SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform .... SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform :: GCE metadata provisioning SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform :: GCS artifact proxy SKIPPED
[INFO] Apache Beam :: Runners :: Extensions Java .......... SKIPPED
[INFO] Apache Beam :: Runners :: Extensions Java :: Metrics SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Aggregated Javadoc .. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 58.176 s
[INFO] Finished at: 2018-04-29T12:01:39Z
[INFO] Final Memory: 105M/3203M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal com.igormaznitsa:mvn-golang-wrapper:2.1.7:get (go-get-imports) on project beam-sdks-go-container: Process exit code : 1 -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :beam-sdks-go-container
STDERR:
2018-04-29 12:01:39,612 7e877a07 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-04-29 12:01:39,621 7e877a07 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 175, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 265, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 290, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/7e877a07/beam/sdks/python/target>
2018-04-29 12:01:39,622 7e877a07 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-04-29 12:01:39,624 7e877a07 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 780, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 175, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 265, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 290, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/7e877a07/beam/sdks/python/target>
2018-04-29 12:01:39,624 7e877a07 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-04-29 12:01:39,663 7e877a07 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-04-29 12:01:39,664 7e877a07 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/7e877a07/pkb.log>
2018-04-29 12:01:39,664 7e877a07 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/7e877a07/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user shoyer@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Jenkins build is back to normal : beam_PerformanceTests_Python
#1268
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1268/display/redirect?page=changes>
Build failed in Jenkins: beam_PerformanceTests_Python #1267
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1267/display/redirect>
------------------------------------------
[...truncated 100.18 KB...]
2018-08-04 01:09:27,927 0baf5535 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code.
2018-08-04 01:09:46,098 0baf5535 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json>
2018-08-04 01:09:49,974 0baf5535 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json}> ReturnCode:1
STDOUT:
BigQuery error in load operation: Error processing job
'apache-beam-testing:bqjob_r6d1b59a3091e4529_00000165027bb68d_1': Invalid schema
update. Field timestamp has changed type from TIMESTAMP to FLOAT
STDERR: Upload complete.Waiting on bqjob_r6d1b59a3091e4529_00000165027bb68d_1 ... (0s) Current status: RUNNING Waiting on bqjob_r6d1b59a3091e4529_00000165027bb68d_1 ... (0s) Current status: DONE
2018-08-04 01:09:49,975 0baf5535 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code.
2018-08-04 01:10:12,981 0baf5535 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json>
2018-08-04 01:10:16,990 0baf5535 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json}> ReturnCode:1
STDOUT:
BigQuery error in load operation: Error processing job
'apache-beam-testing:bqjob_r4840b2454a8e3494_00000165027c1fc7_1': Invalid schema
update. Field timestamp has changed type from TIMESTAMP to FLOAT
STDERR: Upload complete.Waiting on bqjob_r4840b2454a8e3494_00000165027c1fc7_1 ... (0s) Current status: RUNNING Waiting on bqjob_r4840b2454a8e3494_00000165027c1fc7_1 ... (0s) Current status: DONE
2018-08-04 01:10:16,991 0baf5535 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code.
2018-08-04 01:10:44,182 0baf5535 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json>
2018-08-04 01:10:48,193 0baf5535 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json}> ReturnCode:1
STDOUT:
BigQuery error in load operation: Error processing job
'apache-beam-testing:bqjob_r178283251e23bd57_00000165027c99aa_1': Invalid schema
update. Field timestamp has changed type from TIMESTAMP to FLOAT
STDERR: Upload complete.Waiting on bqjob_r178283251e23bd57_00000165027c99aa_1 ... (0s) Current status: RUNNING Waiting on bqjob_r178283251e23bd57_00000165027c99aa_1 ... (0s) Current status: DONE
2018-08-04 01:10:48,194 0baf5535 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code.
2018-08-04 01:11:09,452 0baf5535 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json>
2018-08-04 01:11:13,329 0baf5535 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json}> ReturnCode:1
STDOUT:
BigQuery error in load operation: Error processing job
'apache-beam-testing:bqjob_r4b3530b4d72d9f35_00000165027cfc3b_1': Invalid schema
update. Field timestamp has changed type from TIMESTAMP to FLOAT
STDERR: Upload complete.Waiting on bqjob_r4b3530b4d72d9f35_00000165027cfc3b_1 ... (0s) Current status: RUNNING Waiting on bqjob_r4b3530b4d72d9f35_00000165027cfc3b_1 ... (0s) Current status: DONE
2018-08-04 01:11:13,330 0baf5535 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code.
2018-08-04 01:11:32,307 0baf5535 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json>
2018-08-04 01:11:36,404 0baf5535 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json}> ReturnCode:1
STDOUT:
BigQuery error in load operation: Error processing job
'apache-beam-testing:bqjob_r2dab7c4da2143862_00000165027d55b3_1': Invalid schema
update. Field timestamp has changed type from TIMESTAMP to FLOAT
STDERR: Upload complete.Waiting on bqjob_r2dab7c4da2143862_00000165027d55b3_1 ... (0s) Current status: RUNNING Waiting on bqjob_r2dab7c4da2143862_00000165027d55b3_1 ... (0s) Current status: DONE
2018-08-04 01:11:36,405 0baf5535 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code.
2018-08-04 01:11:56,492 0baf5535 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json>
2018-08-04 01:12:00,468 0baf5535 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json}> ReturnCode:1
STDOUT:
BigQuery error in load operation: Error processing job
'apache-beam-testing:bqjob_r6afc3f5f01d3da95_00000165027db417_1': Invalid schema
update. Field timestamp has changed type from TIMESTAMP to FLOAT
STDERR: Upload complete.Waiting on bqjob_r6afc3f5f01d3da95_00000165027db417_1 ... (0s) Current status: RUNNING Waiting on bqjob_r6afc3f5f01d3da95_00000165027db417_1 ... (0s) Current status: DONE
2018-08-04 01:12:00,469 0baf5535 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code.
2018-08-04 01:12:28,107 0baf5535 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json>
2018-08-04 01:12:32,126 0baf5535 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json}> ReturnCode:1
STDOUT:
BigQuery error in load operation: Error processing job
'apache-beam-testing:bqjob_r4c35d1eaaa7322ec_00000165027e2fa0_1': Invalid schema
update. Field timestamp has changed type from TIMESTAMP to FLOAT
STDERR: Upload complete.Waiting on bqjob_r4c35d1eaaa7322ec_00000165027e2fa0_1 ... (0s) Current status: RUNNING Waiting on bqjob_r4c35d1eaaa7322ec_00000165027e2fa0_1 ... (0s) Current status: DONE
2018-08-04 01:12:32,127 0baf5535 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code.
2018-08-04 01:12:59,230 0baf5535 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json>
2018-08-04 01:13:03,189 0baf5535 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json}> ReturnCode:1
STDOUT:
BigQuery error in load operation: Error processing job
'apache-beam-testing:bqjob_r56a8555cee86c30e_00000165027ea94b_1': Invalid schema
update. Field timestamp has changed type from TIMESTAMP to FLOAT
STDERR: Upload complete.Waiting on bqjob_r56a8555cee86c30e_00000165027ea94b_1 ... (0s) Current status: RUNNING Waiting on bqjob_r56a8555cee86c30e_00000165027ea94b_1 ... (0s) Current status: DONE
2018-08-04 01:13:03,189 0baf5535 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code.
2018-08-04 01:13:31,562 0baf5535 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json>
2018-08-04 01:13:35,580 0baf5535 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json}> ReturnCode:1
STDOUT:
BigQuery error in load operation: Error processing job
'apache-beam-testing:bqjob_r4a57b6221bd70572_00000165027f2756_1': Invalid schema
update. Field timestamp has changed type from TIMESTAMP to FLOAT
STDERR: Upload complete.Waiting on bqjob_r4a57b6221bd70572_00000165027f2756_1 ... (0s) Current status: RUNNING Waiting on bqjob_r4a57b6221bd70572_00000165027f2756_1 ... (0s) Current status: DONE
2018-08-04 01:13:35,580 0baf5535 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code.
2018-08-04 01:14:00,552 0baf5535 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json>
2018-08-04 01:14:04,695 0baf5535 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json}> ReturnCode:1
STDOUT:
BigQuery error in load operation: Error processing job
'apache-beam-testing:bqjob_r488e26fae3081563_00000165027f98c2_1': Invalid schema
update. Field timestamp has changed type from TIMESTAMP to FLOAT
STDERR: Upload complete.Waiting on bqjob_r488e26fae3081563_00000165027f98c2_1 ... (0s) Current status: RUNNING Waiting on bqjob_r488e26fae3081563_00000165027f98c2_1 ... (0s) Current status: DONE
2018-08-04 01:14:04,695 0baf5535 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code.
2018-08-04 01:14:21,638 0baf5535 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json>
2018-08-04 01:14:25,674 0baf5535 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json}> ReturnCode:1
STDOUT:
BigQuery error in load operation: Error processing job
'apache-beam-testing:bqjob_r4445d9fb83085a53_00000165027feb2c_1': Invalid schema
update. Field timestamp has changed type from TIMESTAMP to FLOAT
STDERR: Upload complete.Waiting on bqjob_r4445d9fb83085a53_00000165027feb2c_1 ... (0s) Current status: RUNNING Waiting on bqjob_r4445d9fb83085a53_00000165027feb2c_1 ... (0s) Current status: DONE
2018-08-04 01:14:25,674 0baf5535 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code.
2018-08-04 01:14:43,773 0baf5535 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json>
2018-08-04 01:14:47,831 0baf5535 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json}> ReturnCode:1
STDOUT:
BigQuery error in load operation: Error processing job
'apache-beam-testing:bqjob_r2fc5cc3b0eccf8_0000016502804192_1': Invalid schema
update. Field timestamp has changed type from TIMESTAMP to FLOAT
STDERR: Upload complete.Waiting on bqjob_r2fc5cc3b0eccf8_0000016502804192_1 ... (0s) Current status: RUNNING Waiting on bqjob_r2fc5cc3b0eccf8_0000016502804192_1 ... (0s) Current status: DONE
2018-08-04 01:14:47,832 0baf5535 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code.
2018-08-04 01:15:09,405 0baf5535 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json>
2018-08-04 01:15:13,393 0baf5535 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json}> ReturnCode:1
STDOUT:
BigQuery error in load operation: Error processing job
'apache-beam-testing:bqjob_r1da0a31736f5c6b5_000001650280a5c2_1': Invalid schema
update. Field timestamp has changed type from TIMESTAMP to FLOAT
STDERR: Upload complete.Waiting on bqjob_r1da0a31736f5c6b5_000001650280a5c2_1 ... (0s) Current status: RUNNING Waiting on bqjob_r1da0a31736f5c6b5_000001650280a5c2_1 ... (0s) Current status: DONE
2018-08-04 01:15:13,394 0baf5535 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code.
2018-08-04 01:15:42,871 0baf5535 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json>
2018-08-04 01:15:46,803 0baf5535 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json}> ReturnCode:1
STDOUT:
BigQuery error in load operation: Error processing job
'apache-beam-testing:bqjob_r296782e907644c9c_000001650281286f_1': Invalid schema
update. Field timestamp has changed type from TIMESTAMP to FLOAT
STDERR: Upload complete.Waiting on bqjob_r296782e907644c9c_000001650281286f_1 ... (0s) Current status: RUNNING Waiting on bqjob_r296782e907644c9c_000001650281286f_1 ... (0s) Current status: DONE
2018-08-04 01:15:46,803 0baf5535 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code.
2018-08-04 01:16:04,053 0baf5535 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json>
2018-08-04 01:16:08,105 0baf5535 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json}> ReturnCode:1
STDOUT:
BigQuery error in load operation: Error processing job
'apache-beam-testing:bqjob_r168cf30e70c68a2_0000016502817b34_1': Invalid schema
update. Field timestamp has changed type from TIMESTAMP to FLOAT
STDERR: Upload complete.Waiting on bqjob_r168cf30e70c68a2_0000016502817b34_1 ... (0s) Current status: RUNNING Waiting on bqjob_r168cf30e70c68a2_0000016502817b34_1 ... (0s) Current status: DONE
2018-08-04 01:16:08,106 0baf5535 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code.
2018-08-04 01:16:31,439 0baf5535 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json>
2018-08-04 01:16:35,400 0baf5535 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json}> ReturnCode:1
STDOUT:
BigQuery error in load operation: Error processing job
'apache-beam-testing:bqjob_r5df6a78dbcd141cd_000001650281e615_1': Invalid schema
update. Field timestamp has changed type from TIMESTAMP to FLOAT
STDERR: Upload complete.Waiting on bqjob_r5df6a78dbcd141cd_000001650281e615_1 ... (0s) Current status: RUNNING Waiting on bqjob_r5df6a78dbcd141cd_000001650281e615_1 ... (0s) Current status: DONE
2018-08-04 01:16:35,401 0baf5535 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code.
2018-08-04 01:16:55,147 0baf5535 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json>
2018-08-04 01:16:59,535 0baf5535 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json}> ReturnCode:1
STDOUT:
BigQuery error in load operation: Error processing job
'apache-beam-testing:bqjob_r6a5b38b8cff7b7cb_00000165028242d8_1': Invalid schema
update. Field timestamp has changed type from TIMESTAMP to FLOAT
STDERR: Upload complete.Waiting on bqjob_r6a5b38b8cff7b7cb_00000165028242d8_1 ... (0s) Current status: RUNNING Waiting on bqjob_r6a5b38b8cff7b7cb_00000165028242d8_1 ... (0s) Current status: DONE
2018-08-04 01:16:59,536 0baf5535 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code.
2018-08-04 01:17:18,503 0baf5535 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json>
2018-08-04 01:17:22,519 0baf5535 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json}> ReturnCode:1
STDOUT:
BigQuery error in load operation: Error processing job
'apache-beam-testing:bqjob_r540d80a10f444f7e_0000016502829e11_1': Invalid schema
update. Field timestamp has changed type from TIMESTAMP to FLOAT
STDERR: Upload complete.Waiting on bqjob_r540d80a10f444f7e_0000016502829e11_1 ... (0s) Current status: RUNNING Waiting on bqjob_r540d80a10f444f7e_0000016502829e11_1 ... (0s) Current status: DONE
2018-08-04 01:17:22,519 0baf5535 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code.
2018-08-04 01:17:46,148 0baf5535 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json>
2018-08-04 01:17:50,073 0baf5535 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json}> ReturnCode:1
STDOUT:
BigQuery error in load operation: Error processing job
'apache-beam-testing:bqjob_r41abbeef2f17231a_00000165028309f8_1': Invalid schema
update. Field timestamp has changed type from TIMESTAMP to FLOAT
STDERR: Upload complete.Waiting on bqjob_r41abbeef2f17231a_00000165028309f8_1 ... (0s) Current status: RUNNING Waiting on bqjob_r41abbeef2f17231a_00000165028309f8_1 ... (0s) Current status: DONE
2018-08-04 01:17:50,073 0baf5535 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code.
2018-08-04 01:18:14,823 0baf5535 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json>
2018-08-04 01:18:18,648 0baf5535 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json}> ReturnCode:1
STDOUT:
BigQuery error in load operation: Error processing job
'apache-beam-testing:bqjob_r1089e7f2c23201e8_00000165028379fc_1': Invalid schema
update. Field timestamp has changed type from TIMESTAMP to FLOAT
STDERR: Upload complete.Waiting on bqjob_r1089e7f2c23201e8_00000165028379fc_1 ... (0s) Current status: RUNNING Waiting on bqjob_r1089e7f2c23201e8_00000165028379fc_1 ... (0s) Current status: DONE
2018-08-04 01:18:18,649 0baf5535 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code.
2018-08-04 01:18:47,328 0baf5535 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json>
2018-08-04 01:18:51,411 0baf5535 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json}> ReturnCode:1
STDOUT:
BigQuery error in load operation: Error processing job
'apache-beam-testing:bqjob_r45ba4d1ae2d72753_000001650283f8e7_1': Invalid schema
update. Field timestamp has changed type from TIMESTAMP to FLOAT
STDERR: Upload complete.Waiting on bqjob_r45ba4d1ae2d72753_000001650283f8e7_1 ... (0s) Current status: RUNNING Waiting on bqjob_r45ba4d1ae2d72753_000001650283f8e7_1 ... (0s) Current status: DONE
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/pkb.py",> line 21, in <module>
sys.exit(Main())
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 1012, in Main
return RunBenchmarks()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 939, in RunBenchmarks
collector.PublishSamples()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/publisher.py",> line 928, in PublishSamples
publisher.PublishSamples(self.samples)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/publisher.py",> line 538, in PublishSamples
vm_util.IssueRetryableCommand(load_cmd)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/vm_util.py",> line 249, in WrappedFunction
return f(*args, **kwargs)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/vm_util.py",> line 387, in IssueRetryableCommand
'Command returned a non-zero exit code.\n')
perfkitbenchmarker.errors.CalledProcessException: Command returned a non-zero exit code.
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PerformanceTests_Python #1266
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1266/display/redirect>
------------------------------------------
GitHub pull request #6142 of commit c48115558d7af37b518f69463dba5bd7612054a5, no merge conflicts.
Setting status of c48115558d7af37b518f69463dba5bd7612054a5 to PENDING with url https://builds.apache.org/job/beam_PerformanceTests_Python/1266/ and message: 'Build started for merge commit.'
Using context: Python SDK Performance Test
[EnvInject] - Loading node environment variables.
Building remotely on beam15 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/6142/*:refs/remotes/origin/pr/6142/*
> git rev-parse refs/remotes/origin/pr/6142/merge^{commit} # timeout=10
> git rev-parse refs/remotes/origin/origin/pr/6142/merge^{commit} # timeout=10
Checking out Revision abc1bce29360cd14fc5fb6f62c1c52193d112fa4 (refs/remotes/origin/pr/6142/merge)
> git config core.sparsecheckout # timeout=10
> git checkout -f abc1bce29360cd14fc5fb6f62c1c52193d112fa4
Commit message: "Merge c48115558d7af37b518f69463dba5bd7612054a5 into 2d3ce5181ad144de9c5cc6f945a6d40f9c0ae795"
First time build. Skipping changelog.
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8333306446748541054.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker>
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3957460423414083293.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/.beam_env>
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2172882893396186781.sh
+ virtualenv <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/.beam_env> --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/.beam_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/.beam_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5967812430427483923.sh
+ virtualenv <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/.perfkit_env> --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/.perfkit_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/.perfkit_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5622511714784926622.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/.beam_env/bin/pip> install --upgrade setuptools pip grpcio-tools==1.3.5
Requirement already up-to-date: setuptools in ./src/.beam_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./src/.beam_env/lib/python2.7/site-packages (18.0)
Collecting grpcio-tools==1.3.5
Using cached https://files.pythonhosted.org/packages/05/f6/0296e29b1bac6f85d2a8556d48adf825307f73109a3c2c17fb734292db0a/grpcio_tools-1.3.5-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied, skipping upgrade: grpcio>=1.3.5 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio-tools==1.3.5) (1.13.0)
Requirement already satisfied, skipping upgrade: protobuf>=3.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio-tools==1.3.5) (3.6.0)
Requirement already satisfied, skipping upgrade: six>=1.5.2 in /usr/local/lib/python2.7/dist-packages (from grpcio>=1.3.5->grpcio-tools==1.3.5) (1.11.0)
Requirement already satisfied, skipping upgrade: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio>=1.3.5->grpcio-tools==1.3.5) (1.1.6)
Requirement already satisfied, skipping upgrade: futures>=2.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio>=1.3.5->grpcio-tools==1.3.5) (3.2.0)
Installing collected packages: grpcio-tools
Successfully installed grpcio-tools-1.3.5
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins514423414470920267.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/.perfkit_env/bin/pip> install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./PerfKitBenchmarker/.perfkit_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./PerfKitBenchmarker/.perfkit_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3743757725069306645.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker>
fatal: destination path '<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker'> already exists and is not an empty directory.
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PerformanceTests_Python #1265
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1265/display/redirect?page=changes>
Changes:
[yifanzou] [BEAM-5018] Upgrade org.tukaani:xz to 1.8
[ryan.blake.williams] Revert "Install fastavro only in linux"
[ankurgoenka] Tests for running Python on Flink.
[ankurgoenka] Fixing lint issues
[gleb] [BEAM-5050] [SQL] Fix aggregation of nulls
[yifanzou] [BEAM-3906] Automate Validation Aganist Python Wheel
[chamikara] Fixes a Javadoc error
[ankurgoenka] Adding gradle task for python streaming validates flink runner
[aaltay] [BEAM-4391] Example of distributed optimization (#5736)
[tweise] [BEAM-4778] add option to flink job server to clean staged artifacts
[pablo] Fixed wrong isRelease() call in publish
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam15 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision f96b25a947a58542189e62fe46236b63ea99ca6a (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f f96b25a947a58542189e62fe46236b63ea99ca6a
Commit message: "Fixed wrong isRelease() call in publish"
> git rev-list --no-walk f2ae73be3d3d1cb7c3d111505f832aed44bc92eb # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4635927866774441150.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6534342280852594361.sh
+ rm -rf .beam_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4523054878649429971.sh
+ rm -rf .perfkit_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2044191547514904883.sh
+ virtualenv .beam_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins869372471515896118.sh
+ virtualenv .perfkit_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3804156420611226352.sh
+ .beam_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.beam_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.beam_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5269395541421208009.sh
+ .perfkit_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.perfkit_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.perfkit_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1926765491110891651.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3145344628393015723.sh
+ .beam_env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.8.2)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.7)
Requirement already satisfied: dill<=0.2.8.2,>=0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.2.8.2)
Requirement already satisfied: fastavro==0.19.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.19.7)
Requirement already satisfied: grpcio<2,>=1.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.13.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.1.0)
Requirement already satisfied: httplib2<=0.11.3,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.11.3)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.0.0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (4.1.2)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.0)
Requirement already satisfied: pydot<1.3,>=1.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.2.4)
Requirement already satisfied: pytz<=2018.4,>=2018.3 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2018.4)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (3.12)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.6.8)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.11.0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.4)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.2.0)
Requirement already satisfied: future<1.0.0,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.16.0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.5.20)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.90.4)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (7.0.1)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.26.0)
Requirement already satisfied: proto-google-cloud-pubsub-v1==0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: nose>=1.3.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.3.7)
Collecting numpy<2,>=1.14.3 (from apache-beam==2.7.0.dev0)
Using cached https://files.pythonhosted.org/packages/85/51/ba4564ded90e093dbb6adfc3e21f99ae953d9ad56477e1b0d4a93bacf7d3/numpy-1.15.0-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.9.0)
Requirement already satisfied: setuptools>=18.0 in ./.beam_env/lib/python2.7/site-packages (from fastavro==0.19.7->apache-beam==2.7.0.dev0) (40.0.0)
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.7.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.8.0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.0.2)
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.4.2)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.2.2)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (3.4.2)
Requirement already satisfied: pyparsing>=2.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from pydot<1.3,>=1.2.0->apache-beam==2.7.0.dev0) (2.2.0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (0.14.1)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.7.0.dev0) (1.5.3)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (1.22)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (1.5)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.16)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.11.4)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.0.3)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (1.5.0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (3.8)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (2.1.0)
Installing collected packages: numpy, apache-beam
Found existing installation: numpy 1.13.3
Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'numpy'. No files were found to uninstall.
Found existing installation: apache-beam 2.7.0.dev0
Not uninstalling apache-beam at /home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_HadoopInputFormat/src/sdks/python, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'apache-beam'. No files were found to uninstall.
Running setup.py develop for apache-beam
Successfully installed apache-beam numpy-1.15.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3715548868458883605.sh
+ .beam_env/bin/python src/sdks/python/setup.py sdist --dist-dir=src/sdks/python/target
Traceback (most recent call last):
File "src/sdks/python/setup.py", line 46, in <module>
PACKAGE_VERSION = get_version()
File "src/sdks/python/setup.py", line 41, in get_version
exec(open(os.path.normpath('./apache_beam/version.py')).read(), global_names) # pylint: disable=exec-used
IOError: [Errno 2] No such file or directory: 'apache_beam/version.py'
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PerformanceTests_Python #1264
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1264/display/redirect>
------------------------------------------
[...truncated 1.73 KB...]
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3345867546547775270.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1399758868827400867.sh
+ rm -rf .beam_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6273576107848702835.sh
+ rm -rf .perfkit_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7994866195948922848.sh
+ virtualenv .beam_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins89803871673548492.sh
+ virtualenv .perfkit_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4391940347346608197.sh
+ .beam_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.beam_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.beam_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins401290153052960708.sh
+ .perfkit_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.perfkit_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.perfkit_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7704870098660130538.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins667339526735438882.sh
+ .beam_env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.8.2)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.7)
Requirement already satisfied: dill<=0.2.8.2,>=0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.2.8.2)
Requirement already satisfied: fastavro==0.19.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.19.7)
Requirement already satisfied: grpcio<2,>=1.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.13.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.1.0)
Requirement already satisfied: httplib2<=0.11.3,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.11.3)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.0.0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (4.1.2)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.0)
Requirement already satisfied: pydot<1.3,>=1.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.2.4)
Requirement already satisfied: pytz<=2018.4,>=2018.3 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2018.4)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (3.12)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.6.8)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.11.0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.4)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.2.0)
Requirement already satisfied: future<1.0.0,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.16.0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.5.20)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.90.4)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (7.0.1)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.26.0)
Requirement already satisfied: proto-google-cloud-pubsub-v1==0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: nose>=1.3.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.3.7)
Collecting numpy<2,>=1.14.3 (from apache-beam==2.7.0.dev0)
Using cached https://files.pythonhosted.org/packages/85/51/ba4564ded90e093dbb6adfc3e21f99ae953d9ad56477e1b0d4a93bacf7d3/numpy-1.15.0-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.9.0)
Requirement already satisfied: setuptools>=18.0 in ./.beam_env/lib/python2.7/site-packages (from fastavro==0.19.7->apache-beam==2.7.0.dev0) (40.0.0)
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.7.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.8.0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.0.2)
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.4.2)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.2.2)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (3.4.2)
Requirement already satisfied: pyparsing>=2.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from pydot<1.3,>=1.2.0->apache-beam==2.7.0.dev0) (2.2.0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (0.14.1)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.7.0.dev0) (1.5.3)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (1.22)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (1.5)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.16)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.11.4)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.0.3)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (1.5.0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (3.8)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (2.1.0)
Installing collected packages: numpy, apache-beam
Found existing installation: numpy 1.13.3
Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'numpy'. No files were found to uninstall.
Found existing installation: apache-beam 2.7.0.dev0
Not uninstalling apache-beam at /home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_HadoopInputFormat/src/sdks/python, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'apache-beam'. No files were found to uninstall.
Running setup.py develop for apache-beam
Successfully installed apache-beam numpy-1.15.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2076722097612132100.sh
+ .beam_env/bin/python src/sdks/python/setup.py sdist --dist-dir=src/sdks/python/target
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/local/lib/python2.7/site-packages/setuptools/dist.py>:398: UserWarning: Normalizing '2.7.0.dev' to '2.7.0.dev0'
normalized_version,
running sdist
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/gen_protos.py>:49: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
warning: cmd: standard file not found: should have one of README, README.rst, README.txt, README.md
running check
creating apache-beam-2.7.0.dev0
creating apache-beam-2.7.0.dev0/apache_beam.egg-info
copying files to apache-beam-2.7.0.dev0...
copying apache_beam.egg-info/PKG-INFO -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/SOURCES.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/dependency_links.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/entry_points.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/not-zip-safe -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/requires.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/top_level.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
Writing apache-beam-2.7.0.dev0/setup.cfg
creating src/sdks/python/target
Creating tar archive
removing 'apache-beam-2.7.0.dev0' (and everything under it)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6171094355052098590.sh
+ .perfkit_env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14)) (0.2.2)
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15)) (2.10)
Requirement already satisfied: setuptools in ./.perfkit_env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16)) (40.0.0)
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17)) (2.6.0)
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18)) (1.4)
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19)) (3.2.0)
Requirement already satisfied: PyYAML==3.12 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 20)) (3.12)
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21)) (0.8.1)
Requirement already satisfied: numpy==1.13.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22)) (1.13.3)
Requirement already satisfied: functools32 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 23)) (3.2.3.post2)
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24)) (0.5.5)
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25)) (0.3.0)
Requirement already satisfied: timeout-decorator in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 26)) (0.4.0)
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Requirement already satisfied: colorama; extra == "windows" in /home/jenkins/.local/lib/python2.7/site-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17)) (0.3.9)
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.0)
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.11.0)
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.2.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2526341026312251996.sh
+ .perfkit_env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --bigquery_table=beam_performance.pkb_results --k8s_get_retry_count=36 --k8s_get_wait_interval=10 --python_binary=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --beam_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_timeout=3600 --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output --beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it
2018-08-02 16:57:46,438 83b8abb4 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/83b8abb4/pkb.log>
2018-08-02 16:57:46,438 83b8abb4 MainThread INFO PerfKitBenchmarker version: v1.12.0-747-g7e4ea3f
2018-08-02 16:57:46,439 83b8abb4 MainThread INFO Flag values:
--beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it
--beam_it_timeout=3600
--beam_sdk=python
--k8s_get_wait_interval=10
--python_binary=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python>
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--beam_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src>
--k8s_get_retry_count=36
--benchmarks=beam_integration_benchmark
2018-08-02 16:57:46,693 83b8abb4 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-08-02 16:57:46,900 83b8abb4 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-08-02 16:57:46,905 83b8abb4 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-08-02 16:57:46,905 83b8abb4 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/gradlew> clean assemble --stacktrace --info -DintegrationTestRunner=dataflow
2018-08-02 17:03:48,932 83b8abb4 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-08-02 17:03:49,025 83b8abb4 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> setup.py nosetests --tests=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/target/apache-beam-2.7.0.dev0.tar.gz>
2018-08-02 18:03:49,033 83b8abb4 Thread-3 ERROR IssueCommand timed out after 3600 seconds. Killing command "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> setup.py nosetests --tests=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/target/apache-beam-2.7.0.dev0.tar.gz".>
2018-08-02 18:03:49,043 83b8abb4 MainThread beam_integration_benchmark(1/1) INFO Ran: {<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> setup.py nosetests --tests=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/target/apache-beam-2.7.0.dev0.tar.gz}> ReturnCode:-9
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/local/lib/python2.7/site-packages/setuptools/dist.py>:398: UserWarning: Normalizing '2.7.0.dev' to '2.7.0.dev0'
normalized_version,
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/gen_protos.py>:49: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py>:54: DeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
print('Found: %s.' % self.build_console_url(pipeline.options))
2018-08-02 18:03:49,044 83b8abb4 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 672, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 552, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-08-02 18:03:49,046 83b8abb4 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-08-02 18:03:49,049 83b8abb4 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 808, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 672, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 552, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-08-02 18:03:49,049 83b8abb4 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-08-02 18:03:49,079 83b8abb4 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-08-02 18:03:49,079 83b8abb4 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/83b8abb4/pkb.log>
2018-08-02 18:03:49,080 83b8abb4 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/83b8abb4/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PerformanceTests_Python #1263
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1263/display/redirect?page=changes>
Changes:
[rober] Avoid redundant computation when invoking ProcessElements in ParDo
[github] Update fn_test.go
[rober] De-dup pre & post invoke code in pardo.go
[timrobertson100] [BEAM-2661] Adds KuduIO
[github] Properly revert log line changes in fn_test.go
[rober] Add comparitive benchmark between cached and single use invoker.
[github] [BEAM-4852] Only read symbol table when required.
[github] Update symbols.go
[timrobertson100] [BEAM-2661] KuduIO: Add missing licenses
[ehudm] Make with_attributes kwarg optional.
[ehudm] Add decode and encode steps to streaming_wordcount
[qinyeli] Interactive Beam -- display update
[ehudm] Fix PubSubMessageMatcher not acking messages.
[ebeach] [BEAM-4862] Fixes bug in Spanner's MutationGroupEncoder by converting
[herohde] Add utilities for manipulating proto pipelines
[herohde] [BEAM-4636] Refactor Go dataflow runner for more flexible uses
[herohde] [BEAM-4813] Switch to translation from model protos to Dataflow v1beta3
[herohde] Add more Go integration tests and fix CoGBK bugs
[herohde] [BEAM-4636] Ensure unique filename for compiled worker binaries
[herohde] Handle explicit flatten
[herohde] Fix Dataflow argument bug
[herohde] CR: tweak code and fix comments
[aaltay] [BEAM-4751] fix missing pylint3 check for io subpackage (#5916)
[pablo] Add GitScm poll trigger for post-commit tests.
[relax] Enable schemas for Dataflow runner.
[tweise] [BEAM-5023] Send HarnessId in DataClient (#6066)
[relax] Fix BigQuery job retry mechanism.
[relax] Add exponential backoff on retries.
[relax] Apply code-review comment.
[tweise] [BEAM-2930] Side inputs are not yet supported in streaming mode.
[github] Don't rely on order of elements in a PCollection after GBK in
[relax] Properly handle running jobs.
[devinduan] Spelling mistakes
[relax] Convert BeamSQL to use Schemas.
[relax] Deprecate getRowCoder.
[relax] Add setSchema to remaining Table objects.
[relax] Delete a bunch of code that is no longer used.
[relax] Move utilities into schemas.utils package.
[relax] Convert Nexmark to use schemas.
[relax] remove deprecated Schema.getRowCoder
[relax] Apply spotless.
[relax] Plumb schema through DoFn.
[relax] Add helper setRowSchema to eliminatre boilerplate.
[rober] Avoid overwritting user changes to Resolver
[rober] Clean up deferedResolver
[thw] [BEAM-4842] Update Flink Runner to Flink 1.5.2
[migryz] Add retries for gpg key query logic.
[pablo] Fix scheduling for jobs
[relax] Add convenience methods for pojo and javabean schema registration.
[relax] Address code-review comments.
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision f2ae73be3d3d1cb7c3d111505f832aed44bc92eb (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f f2ae73be3d3d1cb7c3d111505f832aed44bc92eb
Commit message: "Merge pull request #6109: [BEAM-4842] Update Flink Runner to Flink 1.5.2"
> git rev-list --no-walk ff95a82e461bd8319d9733be60e75992ba90cd7c # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8260919133135875607.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3883146302874079411.sh
+ rm -rf .beam_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8573027766605549178.sh
+ rm -rf .perfkit_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7954528295858307459.sh
+ virtualenv .beam_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins117727936914663087.sh
+ virtualenv .perfkit_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4530960035642880902.sh
+ .beam_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.beam_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.beam_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5189163966448436943.sh
+ .perfkit_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.perfkit_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.perfkit_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6867585134301396263.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1463111188888185076.sh
+ .beam_env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.8.2)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.7)
Requirement already satisfied: dill<=0.2.8.2,>=0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.2.8.2)
Requirement already satisfied: grpcio<2,>=1.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.13.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.1.0)
Requirement already satisfied: httplib2<=0.11.3,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.11.3)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.0.0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (4.1.2)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.0)
Requirement already satisfied: pydot<1.3,>=1.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.2.4)
Requirement already satisfied: pytz<=2018.4,>=2018.3 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2018.4)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (3.12)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.6.8)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.11.0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.4)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.2.0)
Requirement already satisfied: future<1.0.0,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.16.0)
Requirement already satisfied: fastavro==0.19.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.19.7)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.5.20)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.90.4)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (7.0.1)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.26.0)
Requirement already satisfied: proto-google-cloud-pubsub-v1==0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: nose>=1.3.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.3.7)
Collecting numpy<2,>=1.14.3 (from apache-beam==2.7.0.dev0)
Using cached https://files.pythonhosted.org/packages/85/51/ba4564ded90e093dbb6adfc3e21f99ae953d9ad56477e1b0d4a93bacf7d3/numpy-1.15.0-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.9.0)
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.7.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.8.0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.0.2)
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.4.2)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.2.2)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (3.4.2)
Requirement already satisfied: setuptools in ./.beam_env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.7.0.dev0) (40.0.0)
Requirement already satisfied: pyparsing>=2.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from pydot<1.3,>=1.2.0->apache-beam==2.7.0.dev0) (2.2.0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (0.14.1)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.7.0.dev0) (1.5.3)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (1.22)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (1.5)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.16)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.11.4)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.0.3)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (1.5.0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (3.8)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (2.1.0)
Installing collected packages: numpy, apache-beam
Found existing installation: numpy 1.13.3
Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'numpy'. No files were found to uninstall.
Found existing installation: apache-beam 2.7.0.dev0
Not uninstalling apache-beam at /home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_Compressed_TextIOIT_HDFS/src/sdks/python, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'apache-beam'. No files were found to uninstall.
Running setup.py develop for apache-beam
Successfully installed apache-beam numpy-1.15.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6724119042722330580.sh
+ .beam_env/bin/python src/sdks/python/setup.py sdist --dist-dir=src/sdks/python/target
Traceback (most recent call last):
File "src/sdks/python/setup.py", line 46, in <module>
PACKAGE_VERSION = get_version()
File "src/sdks/python/setup.py", line 41, in get_version
exec(open(os.path.normpath('./apache_beam/version.py')).read(), global_names) # pylint: disable=exec-used
IOError: [Errno 2] No such file or directory: 'apache_beam/version.py'
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PerformanceTests_Python #1262
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1262/display/redirect>
------------------------------------------
[...truncated 2.19 KB...]
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins134203476966691273.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins266418180377408786.sh
+ rm -rf .beam_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2763990076871907715.sh
+ rm -rf .perfkit_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3751194031371478445.sh
+ virtualenv .beam_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3369103436413740422.sh
+ virtualenv .perfkit_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2139078975145550914.sh
+ .beam_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.beam_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.beam_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8720383783791910367.sh
+ .perfkit_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.perfkit_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.perfkit_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4964247970169696296.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5972620634519769960.sh
+ .beam_env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.8.2)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.7)
Requirement already satisfied: dill<=0.2.8.2,>=0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.2.8.2)
Requirement already satisfied: grpcio<2,>=1.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.13.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.1.0)
Requirement already satisfied: httplib2<=0.11.3,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.11.3)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.0.0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (4.1.2)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.0)
Requirement already satisfied: pydot<1.3,>=1.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.2.4)
Requirement already satisfied: pytz<=2018.4,>=2018.3 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2018.4)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (3.12)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.6.8)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.11.0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.4)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.2.0)
Requirement already satisfied: future<1.0.0,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.16.0)
Requirement already satisfied: fastavro==0.19.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.19.7)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.5.20)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.90.4)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (7.0.1)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.26.0)
Requirement already satisfied: proto-google-cloud-pubsub-v1==0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: nose>=1.3.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.3.7)
Collecting numpy<2,>=1.14.3 (from apache-beam==2.7.0.dev0)
Using cached https://files.pythonhosted.org/packages/85/51/ba4564ded90e093dbb6adfc3e21f99ae953d9ad56477e1b0d4a93bacf7d3/numpy-1.15.0-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.9.0)
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.7.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.8.0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.0.2)
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.4.2)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.2.2)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (3.4.2)
Requirement already satisfied: setuptools in ./.beam_env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.7.0.dev0) (40.0.0)
Requirement already satisfied: pyparsing>=2.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from pydot<1.3,>=1.2.0->apache-beam==2.7.0.dev0) (2.2.0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (0.14.1)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.7.0.dev0) (1.5.3)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (1.22)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (1.5)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.16)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.11.4)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.0.3)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (1.5.0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (3.8)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (2.1.0)
Installing collected packages: numpy, apache-beam
Found existing installation: numpy 1.13.3
Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'numpy'. No files were found to uninstall.
Found existing installation: apache-beam 2.7.0.dev0
Not uninstalling apache-beam at /home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_HadoopInputFormat/src/sdks/python, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'apache-beam'. No files were found to uninstall.
Running setup.py develop for apache-beam
Successfully installed apache-beam numpy-1.15.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins168109957690731329.sh
+ .beam_env/bin/python src/sdks/python/setup.py sdist --dist-dir=src/sdks/python/target
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/local/lib/python2.7/site-packages/setuptools/dist.py>:398: UserWarning: Normalizing '2.7.0.dev' to '2.7.0.dev0'
normalized_version,
running sdist
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/gen_protos.py>:49: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
running egg_info
creating apache_beam.egg-info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
warning: cmd: standard file not found: should have one of README, README.rst, README.txt, README.md
running check
creating apache-beam-2.7.0.dev0
creating apache-beam-2.7.0.dev0/apache_beam.egg-info
copying files to apache-beam-2.7.0.dev0...
copying apache_beam.egg-info/PKG-INFO -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/SOURCES.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/dependency_links.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/entry_points.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/not-zip-safe -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/requires.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/top_level.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
Writing apache-beam-2.7.0.dev0/setup.cfg
creating src/sdks/python/target
Creating tar archive
removing 'apache-beam-2.7.0.dev0' (and everything under it)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins9219101386029762686.sh
+ .perfkit_env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14)) (0.2.2)
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15)) (2.10)
Requirement already satisfied: setuptools in ./.perfkit_env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16)) (40.0.0)
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17)) (2.6.0)
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18)) (1.4)
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19)) (3.2.0)
Requirement already satisfied: PyYAML==3.12 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 20)) (3.12)
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21)) (0.8.1)
Requirement already satisfied: numpy==1.13.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22)) (1.13.3)
Requirement already satisfied: functools32 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 23)) (3.2.3.post2)
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24)) (0.5.5)
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25)) (0.3.0)
Requirement already satisfied: timeout-decorator in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 26)) (0.4.0)
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Requirement already satisfied: colorama; extra == "windows" in /home/jenkins/.local/lib/python2.7/site-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17)) (0.3.9)
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.0)
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.11.0)
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.2.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1829006631628808140.sh
+ .perfkit_env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --bigquery_table=beam_performance.pkb_results --k8s_get_retry_count=36 --k8s_get_wait_interval=10 --python_binary=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --beam_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_timeout=3600 --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output --beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it
2018-08-01 11:22:25,420 87a2f212 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/87a2f212/pkb.log>
2018-08-01 11:22:25,421 87a2f212 MainThread INFO PerfKitBenchmarker version: v1.12.0-747-g7e4ea3f
2018-08-01 11:22:25,422 87a2f212 MainThread INFO Flag values:
--beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it
--beam_it_timeout=3600
--beam_sdk=python
--k8s_get_wait_interval=10
--python_binary=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python>
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--beam_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src>
--k8s_get_retry_count=36
--benchmarks=beam_integration_benchmark
2018-08-01 11:22:25,728 87a2f212 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-08-01 11:22:25,960 87a2f212 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-08-01 11:22:25,964 87a2f212 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-08-01 11:22:25,964 87a2f212 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/gradlew> clean assemble --stacktrace --info -DintegrationTestRunner=dataflow
2018-08-01 11:28:25,618 87a2f212 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-08-01 11:28:25,719 87a2f212 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> setup.py nosetests --tests=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/target/apache-beam-2.7.0.dev0.tar.gz>
2018-08-01 12:28:25,727 87a2f212 Thread-3 ERROR IssueCommand timed out after 3600 seconds. Killing command "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> setup.py nosetests --tests=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/target/apache-beam-2.7.0.dev0.tar.gz".>
2018-08-01 12:28:25,737 87a2f212 MainThread beam_integration_benchmark(1/1) INFO Ran: {<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> setup.py nosetests --tests=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/target/apache-beam-2.7.0.dev0.tar.gz}> ReturnCode:-9
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/local/lib/python2.7/site-packages/setuptools/dist.py>:398: UserWarning: Normalizing '2.7.0.dev' to '2.7.0.dev0'
normalized_version,
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/gen_protos.py>:49: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py>:54: DeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
print('Found: %s.' % self.build_console_url(pipeline.options))
2018-08-01 12:28:25,738 87a2f212 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 672, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 552, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-08-01 12:28:25,739 87a2f212 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-08-01 12:28:25,741 87a2f212 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 808, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 672, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 552, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-08-01 12:28:25,742 87a2f212 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-08-01 12:28:25,796 87a2f212 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-08-01 12:28:25,797 87a2f212 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/87a2f212/pkb.log>
2018-08-01 12:28:25,797 87a2f212 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/87a2f212/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PerformanceTests_Python #1261
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1261/display/redirect?page=changes>
Changes:
[Jozef.Vilcek] Finalize writes before returning filenames via WriteFileResults
[yifanzou] [BEAM-5012] upgrade spring-expression
[aaltay] [BEAM-4747] mkdirs if they don't exist in localfilesystem (#5903)
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision ff95a82e461bd8319d9733be60e75992ba90cd7c (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f ff95a82e461bd8319d9733be60e75992ba90cd7c
Commit message: "Merge pull request #6091: [BEAM-5012] upgrade spring-expression"
> git rev-list --no-walk 7ea9c82457c96835215bf799836738e256bac1da # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8865120929856213274.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1473121459185172423.sh
+ rm -rf .beam_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3734836492793927241.sh
+ rm -rf .perfkit_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8679179533584043181.sh
+ virtualenv .beam_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2159511384415016943.sh
+ virtualenv .perfkit_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8080162836156584622.sh
+ .beam_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.beam_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.beam_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1309902400704482311.sh
+ .perfkit_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.perfkit_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.perfkit_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4478482058959712825.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1441016414319258742.sh
+ .beam_env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.8.2)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.7)
Requirement already satisfied: dill<=0.2.8.2,>=0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.2.8.2)
Requirement already satisfied: grpcio<2,>=1.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.13.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.1.0)
Requirement already satisfied: httplib2<=0.11.3,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.11.3)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.0.0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (4.1.2)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.0)
Requirement already satisfied: pydot<1.3,>=1.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.2.4)
Requirement already satisfied: pytz<=2018.4,>=2018.3 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2018.4)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (3.12)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.6.8)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.11.0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.4)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.2.0)
Requirement already satisfied: future<1.0.0,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.16.0)
Requirement already satisfied: fastavro==0.19.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.19.7)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.5.20)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.90.4)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (7.0.1)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.26.0)
Requirement already satisfied: proto-google-cloud-pubsub-v1==0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: nose>=1.3.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.3.7)
Collecting numpy<2,>=1.14.3 (from apache-beam==2.7.0.dev0)
Using cached https://files.pythonhosted.org/packages/85/51/ba4564ded90e093dbb6adfc3e21f99ae953d9ad56477e1b0d4a93bacf7d3/numpy-1.15.0-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.9.0)
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.7.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.8.0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.0.2)
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.4.2)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.2.2)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (3.4.2)
Requirement already satisfied: setuptools in ./.beam_env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.7.0.dev0) (40.0.0)
Requirement already satisfied: pyparsing>=2.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from pydot<1.3,>=1.2.0->apache-beam==2.7.0.dev0) (2.2.0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (0.14.1)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.7.0.dev0) (1.5.3)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (1.22)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (1.5)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.16)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.11.4)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.0.3)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (1.5.0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (3.8)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (2.1.0)
Installing collected packages: numpy, apache-beam
Found existing installation: numpy 1.13.3
Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'numpy'. No files were found to uninstall.
Found existing installation: apache-beam 2.7.0.dev0
Not uninstalling apache-beam at /home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_Compressed_TextIOIT_HDFS/src/sdks/python, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'apache-beam'. No files were found to uninstall.
Running setup.py develop for apache-beam
Successfully installed apache-beam numpy-1.15.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3075750881756728596.sh
+ .beam_env/bin/python src/sdks/python/setup.py sdist --dist-dir=src/sdks/python/target
Traceback (most recent call last):
File "src/sdks/python/setup.py", line 46, in <module>
PACKAGE_VERSION = get_version()
File "src/sdks/python/setup.py", line 41, in get_version
exec(open(os.path.normpath('./apache_beam/version.py')).read(), global_names) # pylint: disable=exec-used
IOError: [Errno 2] No such file or directory: 'apache_beam/version.py'
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PerformanceTests_Python #1260
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1260/display/redirect>
------------------------------------------
[...truncated 1.68 KB...]
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2607446597210156195.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2443095237927008836.sh
+ rm -rf .beam_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1189694480124475100.sh
+ rm -rf .perfkit_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2975676980433889651.sh
+ virtualenv .beam_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6197281220405623172.sh
+ virtualenv .perfkit_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3153161168204468762.sh
+ .beam_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.beam_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.beam_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3320282445925404651.sh
+ .perfkit_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.perfkit_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.perfkit_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins9169151424390991382.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins765699272845885303.sh
+ .beam_env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.8.2)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.7)
Requirement already satisfied: dill<=0.2.8.2,>=0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.2.8.2)
Requirement already satisfied: grpcio<2,>=1.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.13.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.1.0)
Requirement already satisfied: httplib2<=0.11.3,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.11.3)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.0.0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (4.1.2)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.0)
Requirement already satisfied: pydot<1.3,>=1.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.2.4)
Requirement already satisfied: pytz<=2018.4,>=2018.3 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2018.4)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (3.12)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.6.8)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.11.0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.4)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.2.0)
Requirement already satisfied: future<1.0.0,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.16.0)
Requirement already satisfied: fastavro==0.19.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.19.7)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.5.20)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.90.4)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (7.0.1)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.26.0)
Requirement already satisfied: proto-google-cloud-pubsub-v1==0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: nose>=1.3.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.3.7)
Collecting numpy<2,>=1.14.3 (from apache-beam==2.7.0.dev0)
Using cached https://files.pythonhosted.org/packages/85/51/ba4564ded90e093dbb6adfc3e21f99ae953d9ad56477e1b0d4a93bacf7d3/numpy-1.15.0-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.9.0)
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.7.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.8.0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.0.2)
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.4.2)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.2.2)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (3.4.2)
Requirement already satisfied: setuptools in ./.beam_env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.7.0.dev0) (40.0.0)
Requirement already satisfied: pyparsing>=2.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from pydot<1.3,>=1.2.0->apache-beam==2.7.0.dev0) (2.2.0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (0.14.1)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.7.0.dev0) (1.5.3)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (1.22)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (1.5)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.16)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.11.4)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.0.3)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (1.5.0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (3.8)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (2.1.0)
Installing collected packages: numpy, apache-beam
Found existing installation: numpy 1.13.3
Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'numpy'. No files were found to uninstall.
Found existing installation: apache-beam 2.7.0.dev0
Not uninstalling apache-beam at /home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_Compressed_TextIOIT_HDFS/src/sdks/python, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'apache-beam'. No files were found to uninstall.
Running setup.py develop for apache-beam
Successfully installed apache-beam numpy-1.15.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4561233640386505352.sh
+ .beam_env/bin/python src/sdks/python/setup.py sdist --dist-dir=src/sdks/python/target
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/local/lib/python2.7/site-packages/setuptools/dist.py>:398: UserWarning: Normalizing '2.7.0.dev' to '2.7.0.dev0'
normalized_version,
running sdist
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/gen_protos.py>:49: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
warning: cmd: standard file not found: should have one of README, README.rst, README.txt, README.md
running check
creating apache-beam-2.7.0.dev0
creating apache-beam-2.7.0.dev0/apache_beam.egg-info
copying files to apache-beam-2.7.0.dev0...
copying apache_beam.egg-info/PKG-INFO -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/SOURCES.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/dependency_links.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/entry_points.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/not-zip-safe -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/requires.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/top_level.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
Writing apache-beam-2.7.0.dev0/setup.cfg
creating src/sdks/python/target
Creating tar archive
removing 'apache-beam-2.7.0.dev0' (and everything under it)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4538288153756298284.sh
+ .perfkit_env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14)) (0.2.2)
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15)) (2.10)
Requirement already satisfied: setuptools in ./.perfkit_env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16)) (40.0.0)
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17)) (2.6.0)
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18)) (1.4)
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19)) (3.2.0)
Requirement already satisfied: PyYAML==3.12 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 20)) (3.12)
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21)) (0.8.1)
Requirement already satisfied: numpy==1.13.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22)) (1.13.3)
Requirement already satisfied: functools32 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 23)) (3.2.3.post2)
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24)) (0.5.5)
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25)) (0.3.0)
Requirement already satisfied: timeout-decorator in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 26)) (0.4.0)
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Requirement already satisfied: colorama; extra == "windows" in /home/jenkins/.local/lib/python2.7/site-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17)) (0.3.9)
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.0)
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.11.0)
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.2.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2064771233173343101.sh
+ .perfkit_env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --bigquery_table=beam_performance.pkb_results --k8s_get_retry_count=36 --k8s_get_wait_interval=10 --python_binary=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --beam_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_timeout=3600 --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-07-30 10:18:46,224 4159b0b0 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/4159b0b0/pkb.log>
2018-07-30 10:18:46,225 4159b0b0 MainThread INFO PerfKitBenchmarker version: v1.12.0-742-g2a71a6d
2018-07-30 10:18:46,226 4159b0b0 MainThread INFO Flag values:
--beam_it_timeout=3600
--beam_sdk=python
--k8s_get_wait_interval=10
--python_binary=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python>
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--beam_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src>
--k8s_get_retry_count=36
--benchmarks=beam_integration_benchmark
2018-07-30 10:18:46,492 4159b0b0 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-07-30 10:18:46,668 4159b0b0 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-07-30 10:18:46,672 4159b0b0 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-07-30 10:18:46,672 4159b0b0 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/gradlew> clean assemble --stacktrace --info -DintegrationTestRunner=dataflow
2018-07-30 10:24:25,112 4159b0b0 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-07-30 10:24:25,205 4159b0b0 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/target/apache-beam-2.7.0.dev0.tar.gz>
2018-07-30 11:24:25,213 4159b0b0 Thread-3 ERROR IssueCommand timed out after 3600 seconds. Killing command "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/target/apache-beam-2.7.0.dev0.tar.gz".>
2018-07-30 11:24:25,222 4159b0b0 MainThread beam_integration_benchmark(1/1) INFO Ran: {<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/target/apache-beam-2.7.0.dev0.tar.gz}> ReturnCode:-9
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/local/lib/python2.7/site-packages/setuptools/dist.py>:398: UserWarning: Normalizing '2.7.0.dev' to '2.7.0.dev0'
normalized_version,
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/gen_protos.py>:49: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py>:54: DeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
print('Found: %s.' % self.build_console_url(pipeline.options))
2018-07-30 11:24:25,223 4159b0b0 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 670, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 550, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-07-30 11:24:25,224 4159b0b0 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-07-30 11:24:25,226 4159b0b0 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 806, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 670, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 550, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-07-30 11:24:25,227 4159b0b0 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-07-30 11:24:25,243 4159b0b0 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-07-30 11:24:25,244 4159b0b0 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/4159b0b0/pkb.log>
2018-07-30 11:24:25,244 4159b0b0 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/4159b0b0/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PerformanceTests_Python #1259
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1259/display/redirect>
------------------------------------------
[...truncated 1.68 KB...]
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1960843742017956862.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5401992170880531760.sh
+ rm -rf .beam_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3905576078412088570.sh
+ rm -rf .perfkit_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2493266811912257130.sh
+ virtualenv .beam_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6936278536347873486.sh
+ virtualenv .perfkit_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins458815562325241686.sh
+ .beam_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.beam_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.beam_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2381364775078793896.sh
+ .perfkit_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.perfkit_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.perfkit_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4726826176375157776.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8334416447047675338.sh
+ .beam_env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.8.2)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.7)
Requirement already satisfied: dill<=0.2.8.2,>=0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.2.8.2)
Requirement already satisfied: grpcio<2,>=1.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.13.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.1.0)
Requirement already satisfied: httplib2<=0.11.3,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.11.3)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.0.0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (4.1.2)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.0)
Requirement already satisfied: pydot<1.3,>=1.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.2.4)
Requirement already satisfied: pytz<=2018.4,>=2018.3 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2018.4)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (3.12)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.6.8)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.11.0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.4)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.2.0)
Requirement already satisfied: future<1.0.0,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.16.0)
Requirement already satisfied: fastavro==0.19.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.19.7)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.5.20)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.90.4)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (7.0.1)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.26.0)
Requirement already satisfied: proto-google-cloud-pubsub-v1==0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: nose>=1.3.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.3.7)
Collecting numpy<2,>=1.14.3 (from apache-beam==2.7.0.dev0)
Using cached https://files.pythonhosted.org/packages/85/51/ba4564ded90e093dbb6adfc3e21f99ae953d9ad56477e1b0d4a93bacf7d3/numpy-1.15.0-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.9.0)
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.7.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.8.0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.0.2)
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.4.2)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.2.2)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (3.4.2)
Requirement already satisfied: setuptools in ./.beam_env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.7.0.dev0) (40.0.0)
Requirement already satisfied: pyparsing>=2.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from pydot<1.3,>=1.2.0->apache-beam==2.7.0.dev0) (2.2.0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (0.14.1)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.7.0.dev0) (1.5.3)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (1.22)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (1.5)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.16)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.11.4)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.0.3)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (1.5.0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (3.8)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (2.1.0)
Installing collected packages: numpy, apache-beam
Found existing installation: numpy 1.13.3
Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'numpy'. No files were found to uninstall.
Found existing installation: apache-beam 2.7.0.dev0
Not uninstalling apache-beam at /home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_Compressed_TextIOIT_HDFS/src/sdks/python, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'apache-beam'. No files were found to uninstall.
Running setup.py develop for apache-beam
Successfully installed apache-beam numpy-1.15.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4358843471068673199.sh
+ .beam_env/bin/python src/sdks/python/setup.py sdist --dist-dir=src/sdks/python/target
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/local/lib/python2.7/site-packages/setuptools/dist.py>:398: UserWarning: Normalizing '2.7.0.dev' to '2.7.0.dev0'
normalized_version,
running sdist
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/gen_protos.py>:49: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
warning: cmd: standard file not found: should have one of README, README.rst, README.txt, README.md
running check
creating apache-beam-2.7.0.dev0
creating apache-beam-2.7.0.dev0/apache_beam.egg-info
copying files to apache-beam-2.7.0.dev0...
copying apache_beam.egg-info/PKG-INFO -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/SOURCES.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/dependency_links.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/entry_points.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/not-zip-safe -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/requires.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/top_level.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
Writing apache-beam-2.7.0.dev0/setup.cfg
creating src/sdks/python/target
Creating tar archive
removing 'apache-beam-2.7.0.dev0' (and everything under it)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2274858807667533021.sh
+ .perfkit_env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14)) (0.2.2)
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15)) (2.10)
Requirement already satisfied: setuptools in ./.perfkit_env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16)) (40.0.0)
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17)) (2.6.0)
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18)) (1.4)
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19)) (3.2.0)
Requirement already satisfied: PyYAML==3.12 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 20)) (3.12)
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21)) (0.8.1)
Requirement already satisfied: numpy==1.13.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22)) (1.13.3)
Requirement already satisfied: functools32 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 23)) (3.2.3.post2)
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24)) (0.5.5)
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25)) (0.3.0)
Requirement already satisfied: timeout-decorator in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 26)) (0.4.0)
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Requirement already satisfied: colorama; extra == "windows" in /home/jenkins/.local/lib/python2.7/site-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17)) (0.3.9)
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.0)
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.11.0)
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.2.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7191424344200540075.sh
+ .perfkit_env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --bigquery_table=beam_performance.pkb_results --k8s_get_retry_count=36 --k8s_get_wait_interval=10 --python_binary=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --beam_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_timeout=2400 --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-07-30 08:41:41,842 2c549279 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/2c549279/pkb.log>
2018-07-30 08:41:41,844 2c549279 MainThread INFO PerfKitBenchmarker version: v1.12.0-742-g2a71a6d
2018-07-30 08:41:41,846 2c549279 MainThread INFO Flag values:
--beam_it_timeout=2400
--beam_sdk=python
--k8s_get_wait_interval=10
--python_binary=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python>
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--beam_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src>
--k8s_get_retry_count=36
--benchmarks=beam_integration_benchmark
2018-07-30 08:41:42,216 2c549279 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-07-30 08:41:42,393 2c549279 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-07-30 08:41:42,396 2c549279 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-07-30 08:41:42,397 2c549279 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/gradlew> clean assemble --stacktrace --info -DintegrationTestRunner=dataflow
2018-07-30 08:47:24,989 2c549279 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-07-30 08:47:25,083 2c549279 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/target/apache-beam-2.7.0.dev0.tar.gz>
2018-07-30 09:27:25,091 2c549279 Thread-3 ERROR IssueCommand timed out after 2400 seconds. Killing command "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/target/apache-beam-2.7.0.dev0.tar.gz".>
2018-07-30 09:27:25,099 2c549279 MainThread beam_integration_benchmark(1/1) INFO Ran: {<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/target/apache-beam-2.7.0.dev0.tar.gz}> ReturnCode:-9
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/local/lib/python2.7/site-packages/setuptools/dist.py>:398: UserWarning: Normalizing '2.7.0.dev' to '2.7.0.dev0'
normalized_version,
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/gen_protos.py>:49: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py>:54: DeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
print('Found: %s.' % self.build_console_url(pipeline.options))
2018-07-30 09:27:25,100 2c549279 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 670, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 550, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-07-30 09:27:25,101 2c549279 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-07-30 09:27:25,103 2c549279 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 806, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 670, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 550, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-07-30 09:27:25,104 2c549279 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-07-30 09:27:25,140 2c549279 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-07-30 09:27:25,141 2c549279 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/2c549279/pkb.log>
2018-07-30 09:27:25,141 2c549279 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/2c549279/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PerformanceTests_Python #1258
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1258/display/redirect?page=changes>
Changes:
[ehudm] Python PubSub updates:
[ehudm] Fix lint errors and clarify PubsubMessage docs.
[ehudm] Skip unit test if pubsub is not installed.
[ehudm] Fix docstring indentation.
[ehudm] Minor fixes.
[lcwik] [BEAM-4860] Ensure that shaded artifacts that are produced don't leak
[pablo] [BEAM-4818] Move the visteg plugin to the root project.
[lcwik] [BEAM-4262] Bumping bigtable versions (#6078)
[github] [BEAM-4658] Update pipeline representation in runner support libraries
[pablo] Changed hardcoded version number into env variable. Added more comments.
[aaltay] [BEAM-4076] Fix schemas on Dataflow and FnApi (#6072)
[qinyeli] Introducing version into CacheManager
[lukasz.gajowy] [BEAM-5037] Fix deserialization problem due to transient field
[lukasz.gajowy] Remove redundant static field
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 7ea9c82457c96835215bf799836738e256bac1da (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 7ea9c82457c96835215bf799836738e256bac1da
Commit message: "Merge pull request #6083: [BEAM-5037] fix synthetic options"
> git rev-list --no-walk aa2b585a5356ce3b607dd3d17a3c6ab2ce65353b # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8055711672218079793.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3631238739723495981.sh
+ rm -rf .beam_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1463782467337498506.sh
+ rm -rf .perfkit_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7762123783080025661.sh
+ virtualenv .beam_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4373957814379593676.sh
+ virtualenv .perfkit_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2690147038434867229.sh
+ .beam_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.beam_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.beam_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7250500320601567036.sh
+ .perfkit_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.perfkit_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.perfkit_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7762139428107416882.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5165480218991547042.sh
+ .beam_env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.8.2)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.7)
Requirement already satisfied: dill<=0.2.8.2,>=0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.2.8.2)
Requirement already satisfied: grpcio<2,>=1.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.13.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.1.0)
Requirement already satisfied: httplib2<=0.11.3,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.11.3)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.0.0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (4.1.2)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.0)
Requirement already satisfied: pydot<1.3,>=1.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.2.4)
Requirement already satisfied: pytz<=2018.4,>=2018.3 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2018.4)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (3.12)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.6.8)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.11.0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.4)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.2.0)
Requirement already satisfied: future<1.0.0,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.16.0)
Requirement already satisfied: fastavro==0.19.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.19.7)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.5.20)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.90.4)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (7.0.1)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.26.0)
Requirement already satisfied: proto-google-cloud-pubsub-v1==0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: nose>=1.3.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.3.7)
Collecting numpy<2,>=1.14.3 (from apache-beam==2.7.0.dev0)
Using cached https://files.pythonhosted.org/packages/85/51/ba4564ded90e093dbb6adfc3e21f99ae953d9ad56477e1b0d4a93bacf7d3/numpy-1.15.0-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.9.0)
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.7.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.8.0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.0.2)
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.4.2)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.2.2)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (3.4.2)
Requirement already satisfied: setuptools in ./.beam_env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.7.0.dev0) (40.0.0)
Requirement already satisfied: pyparsing>=2.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from pydot<1.3,>=1.2.0->apache-beam==2.7.0.dev0) (2.2.0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (0.14.1)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.7.0.dev0) (1.5.3)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (1.22)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (1.5)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.16)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.11.4)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.0.3)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (1.5.0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (3.8)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (2.1.0)
Installing collected packages: numpy, apache-beam
Found existing installation: numpy 1.13.3
Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'numpy'. No files were found to uninstall.
Found existing installation: apache-beam 2.7.0.dev0
Not uninstalling apache-beam at /home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_Compressed_TextIOIT_HDFS/src/sdks/python, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'apache-beam'. No files were found to uninstall.
Running setup.py develop for apache-beam
Successfully installed apache-beam numpy-1.15.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2566799033419822571.sh
+ .beam_env/bin/python src/sdks/python/setup.py sdist --dist-dir=src/sdks/python/target
Traceback (most recent call last):
File "src/sdks/python/setup.py", line 46, in <module>
PACKAGE_VERSION = get_version()
File "src/sdks/python/setup.py", line 41, in get_version
exec(open(os.path.normpath('./apache_beam/version.py')).read(), global_names) # pylint: disable=exec-used
IOError: [Errno 2] No such file or directory: 'apache_beam/version.py'
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PerformanceTests_Python #1257
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1257/display/redirect>
------------------------------------------
[...truncated 1.68 KB...]
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3313402882749527139.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4447330106744310889.sh
+ rm -rf .beam_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3105004998856343976.sh
+ rm -rf .perfkit_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1869377136885516741.sh
+ virtualenv .beam_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins416301535208885651.sh
+ virtualenv .perfkit_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2952868600336155078.sh
+ .beam_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.beam_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.beam_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6523644204269782559.sh
+ .perfkit_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.perfkit_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.perfkit_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins300810585250411513.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4829542257814454879.sh
+ .beam_env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.8.2)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.7)
Requirement already satisfied: dill<=0.2.8.2,>=0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.2.8.2)
Requirement already satisfied: grpcio<2,>=1.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.13.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.1.0)
Requirement already satisfied: httplib2<=0.11.3,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.11.3)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.0.0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (4.1.2)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.0)
Requirement already satisfied: pydot<1.3,>=1.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.2.4)
Requirement already satisfied: pytz<=2018.4,>=2018.3 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2018.4)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (3.12)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.6.8)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.11.0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.4)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.2.0)
Requirement already satisfied: future<1.0.0,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.16.0)
Requirement already satisfied: fastavro==0.19.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.19.7)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.5.20)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.90.4)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (7.0.1)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.26.0)
Requirement already satisfied: proto-google-cloud-pubsub-v1==0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: nose>=1.3.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.3.7)
Collecting numpy<2,>=1.14.3 (from apache-beam==2.7.0.dev0)
Using cached https://files.pythonhosted.org/packages/85/51/ba4564ded90e093dbb6adfc3e21f99ae953d9ad56477e1b0d4a93bacf7d3/numpy-1.15.0-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.9.0)
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.7.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.8.0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.0.2)
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.4.2)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.2.2)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (3.4.2)
Requirement already satisfied: setuptools in ./.beam_env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.7.0.dev0) (40.0.0)
Requirement already satisfied: pyparsing>=2.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from pydot<1.3,>=1.2.0->apache-beam==2.7.0.dev0) (2.2.0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (0.14.1)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.7.0.dev0) (1.5.3)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (1.22)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (1.5)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.16)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.11.4)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.0.3)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (1.5.0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (3.8)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (2.1.0)
Installing collected packages: numpy, apache-beam
Found existing installation: numpy 1.13.3
Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'numpy'. No files were found to uninstall.
Found existing installation: apache-beam 2.7.0.dev0
Not uninstalling apache-beam at /home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_Compressed_TextIOIT_HDFS/src/sdks/python, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'apache-beam'. No files were found to uninstall.
Running setup.py develop for apache-beam
Successfully installed apache-beam numpy-1.15.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins824765903847337789.sh
+ .beam_env/bin/python src/sdks/python/setup.py sdist --dist-dir=src/sdks/python/target
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/local/lib/python2.7/site-packages/setuptools/dist.py>:398: UserWarning: Normalizing '2.7.0.dev' to '2.7.0.dev0'
normalized_version,
running sdist
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/gen_protos.py>:49: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
warning: cmd: standard file not found: should have one of README, README.rst, README.txt, README.md
running check
creating apache-beam-2.7.0.dev0
creating apache-beam-2.7.0.dev0/apache_beam.egg-info
copying files to apache-beam-2.7.0.dev0...
copying apache_beam.egg-info/PKG-INFO -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/SOURCES.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/dependency_links.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/entry_points.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/not-zip-safe -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/requires.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/top_level.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
Writing apache-beam-2.7.0.dev0/setup.cfg
creating src/sdks/python/target
Creating tar archive
removing 'apache-beam-2.7.0.dev0' (and everything under it)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6662012102409384417.sh
+ .perfkit_env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14)) (0.2.2)
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15)) (2.10)
Requirement already satisfied: setuptools in ./.perfkit_env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16)) (40.0.0)
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17)) (2.6.0)
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18)) (1.4)
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19)) (3.2.0)
Requirement already satisfied: PyYAML==3.12 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 20)) (3.12)
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21)) (0.8.1)
Requirement already satisfied: numpy==1.13.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22)) (1.13.3)
Requirement already satisfied: functools32 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 23)) (3.2.3.post2)
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24)) (0.5.5)
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25)) (0.3.0)
Requirement already satisfied: timeout-decorator in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 26)) (0.4.0)
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Requirement already satisfied: colorama; extra == "windows" in /home/jenkins/.local/lib/python2.7/site-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17)) (0.3.9)
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.0)
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.11.0)
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.2.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6039706310566305906.sh
+ .perfkit_env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --bigquery_table=beam_performance.pkb_results --k8s_get_retry_count=36 --k8s_get_wait_interval=10 --python_binary=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --beam_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_timeout=1500 --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-07-27 16:27:40,882 0be68e21 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0be68e21/pkb.log>
2018-07-27 16:27:40,882 0be68e21 MainThread INFO PerfKitBenchmarker version: v1.12.0-735-ga3f334a
2018-07-27 16:27:40,883 0be68e21 MainThread INFO Flag values:
--beam_it_timeout=1500
--beam_sdk=python
--k8s_get_wait_interval=10
--python_binary=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python>
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--beam_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src>
--k8s_get_retry_count=36
--benchmarks=beam_integration_benchmark
2018-07-27 16:27:41,228 0be68e21 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-07-27 16:27:41,417 0be68e21 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-07-27 16:27:41,421 0be68e21 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-07-27 16:27:41,422 0be68e21 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/gradlew> clean assemble --stacktrace --info -DintegrationTestRunner=dataflow
2018-07-27 16:33:23,819 0be68e21 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-07-27 16:33:23,915 0be68e21 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/target/apache-beam-2.7.0.dev0.tar.gz>
2018-07-27 16:58:23,923 0be68e21 Thread-3 ERROR IssueCommand timed out after 1500 seconds. Killing command "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/target/apache-beam-2.7.0.dev0.tar.gz".>
2018-07-27 16:58:23,932 0be68e21 MainThread beam_integration_benchmark(1/1) INFO Ran: {<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/target/apache-beam-2.7.0.dev0.tar.gz}> ReturnCode:-9
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/local/lib/python2.7/site-packages/setuptools/dist.py>:398: UserWarning: Normalizing '2.7.0.dev' to '2.7.0.dev0'
normalized_version,
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/gen_protos.py>:49: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py>:54: DeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
print('Found: %s.' % self.build_console_url(pipeline.options))
2018-07-27 16:58:23,933 0be68e21 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 670, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 550, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-07-27 16:58:23,934 0be68e21 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-07-27 16:58:23,937 0be68e21 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 806, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 670, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 550, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-07-27 16:58:23,937 0be68e21 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-07-27 16:58:23,992 0be68e21 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-07-27 16:58:23,993 0be68e21 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0be68e21/pkb.log>
2018-07-27 16:58:23,994 0be68e21 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0be68e21/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PerformanceTests_Python #1256
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1256/display/redirect>
------------------------------------------
[...truncated 1.66 KB...]
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3539873326631408448.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8647486184674439503.sh
+ rm -rf .beam_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1424138257925272438.sh
+ rm -rf .perfkit_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5754435381585259556.sh
+ virtualenv .beam_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2154991725107398410.sh
+ virtualenv .perfkit_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2814333558721416384.sh
+ .beam_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.beam_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.beam_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3696887024533641845.sh
+ .perfkit_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.perfkit_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.perfkit_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4378314190794416191.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8949469932260524169.sh
+ .beam_env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.8.2)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.7)
Requirement already satisfied: dill<=0.2.8.2,>=0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.2.8.2)
Requirement already satisfied: grpcio<2,>=1.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.13.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.1.0)
Requirement already satisfied: httplib2<=0.11.3,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.11.3)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.0.0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (4.1.2)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.0)
Requirement already satisfied: pydot<1.3,>=1.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.2.4)
Requirement already satisfied: pytz<=2018.4,>=2018.3 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2018.4)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (3.12)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.6.8)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.11.0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.4)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.2.0)
Requirement already satisfied: future<1.0.0,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.16.0)
Requirement already satisfied: fastavro==0.19.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.19.7)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.5.20)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.90.4)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (7.0.1)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.26.0)
Requirement already satisfied: proto-google-cloud-pubsub-v1==0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: nose>=1.3.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.3.7)
Collecting numpy<2,>=1.14.3 (from apache-beam==2.7.0.dev0)
Using cached https://files.pythonhosted.org/packages/85/51/ba4564ded90e093dbb6adfc3e21f99ae953d9ad56477e1b0d4a93bacf7d3/numpy-1.15.0-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.9.0)
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.7.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.8.0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.0.2)
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.4.2)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.2.2)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (3.4.2)
Requirement already satisfied: setuptools in ./.beam_env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.7.0.dev0) (40.0.0)
Requirement already satisfied: pyparsing>=2.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from pydot<1.3,>=1.2.0->apache-beam==2.7.0.dev0) (2.2.0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (0.14.1)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.7.0.dev0) (1.5.3)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (1.22)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (1.5)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.16)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.11.4)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.0.3)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (1.5.0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (3.8)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (2.1.0)
Installing collected packages: numpy, apache-beam
Found existing installation: numpy 1.13.3
Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'numpy'. No files were found to uninstall.
Found existing installation: apache-beam 2.7.0.dev0
Not uninstalling apache-beam at /home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_Compressed_TextIOIT_HDFS/src/sdks/python, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'apache-beam'. No files were found to uninstall.
Running setup.py develop for apache-beam
Successfully installed apache-beam numpy-1.15.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8410315169384367550.sh
+ .beam_env/bin/python src/sdks/python/setup.py sdist --dist-dir=src/sdks/python/target
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/local/lib/python2.7/site-packages/setuptools/dist.py>:398: UserWarning: Normalizing '2.7.0.dev' to '2.7.0.dev0'
normalized_version,
running sdist
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/gen_protos.py>:49: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
warning: cmd: standard file not found: should have one of README, README.rst, README.txt, README.md
running check
creating apache-beam-2.7.0.dev0
creating apache-beam-2.7.0.dev0/apache_beam.egg-info
copying files to apache-beam-2.7.0.dev0...
copying apache_beam.egg-info/PKG-INFO -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/SOURCES.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/dependency_links.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/entry_points.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/not-zip-safe -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/requires.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/top_level.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
Writing apache-beam-2.7.0.dev0/setup.cfg
creating src/sdks/python/target
Creating tar archive
removing 'apache-beam-2.7.0.dev0' (and everything under it)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8857752751154454891.sh
+ .perfkit_env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14)) (0.2.2)
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15)) (2.10)
Requirement already satisfied: setuptools in ./.perfkit_env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16)) (40.0.0)
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17)) (2.6.0)
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18)) (1.4)
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19)) (3.2.0)
Requirement already satisfied: PyYAML==3.12 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 20)) (3.12)
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21)) (0.8.1)
Requirement already satisfied: numpy==1.13.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22)) (1.13.3)
Requirement already satisfied: functools32 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 23)) (3.2.3.post2)
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24)) (0.5.5)
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25)) (0.3.0)
Requirement already satisfied: timeout-decorator in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 26)) (0.4.0)
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Requirement already satisfied: colorama; extra == "windows" in /home/jenkins/.local/lib/python2.7/site-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17)) (0.3.9)
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.0)
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.11.0)
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.2.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1099949695612912981.sh
+ .perfkit_env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --bigquery_table=beam_performance.pkb_results --k8s_get_retry_count=36 --k8s_get_wait_interval=10 --python_binary=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --beam_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-07-27 16:04:45,694 d1536932 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/d1536932/pkb.log>
2018-07-27 16:04:45,695 d1536932 MainThread INFO PerfKitBenchmarker version: v1.12.0-735-ga3f334a
2018-07-27 16:04:45,696 d1536932 MainThread INFO Flag values:
--beam_sdk=python
--k8s_get_wait_interval=10
--python_binary=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python>
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--beam_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src>
--k8s_get_retry_count=36
--benchmarks=beam_integration_benchmark
2018-07-27 16:04:46,023 d1536932 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-07-27 16:04:46,220 d1536932 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-07-27 16:04:46,224 d1536932 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-07-27 16:04:46,225 d1536932 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/gradlew> clean assemble --stacktrace --info -DintegrationTestRunner=dataflow
2018-07-27 16:10:33,485 d1536932 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-07-27 16:10:33,584 d1536932 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/target/apache-beam-2.7.0.dev0.tar.gz>
2018-07-27 16:20:33,592 d1536932 Thread-3 ERROR IssueCommand timed out after 600 seconds. Killing command "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/target/apache-beam-2.7.0.dev0.tar.gz".>
2018-07-27 16:20:33,601 d1536932 MainThread beam_integration_benchmark(1/1) INFO Ran: {<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/target/apache-beam-2.7.0.dev0.tar.gz}> ReturnCode:-9
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/local/lib/python2.7/site-packages/setuptools/dist.py>:398: UserWarning: Normalizing '2.7.0.dev' to '2.7.0.dev0'
normalized_version,
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/gen_protos.py>:49: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py>:54: DeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
print('Found: %s.' % self.build_console_url(pipeline.options))
2018-07-27 16:20:33,602 d1536932 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 670, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 550, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-07-27 16:20:33,603 d1536932 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-07-27 16:20:33,605 d1536932 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 806, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 670, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 550, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-07-27 16:20:33,606 d1536932 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-07-27 16:20:33,620 d1536932 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-07-27 16:20:33,621 d1536932 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/d1536932/pkb.log>
2018-07-27 16:20:33,621 d1536932 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/d1536932/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PerformanceTests_Python #1255
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1255/display/redirect?page=changes>
Changes:
[kedin] [SQL] Remove explicit timestamp parsing from PubsubIOJsonTable
[kedin] [SQL] Wrap Pubsub payload into a message schema
[pablo] Change reading sideinput experimental flag into new pattern.
[mariagh] Call assert_that per window via reify_windows flag
[github] [BEAM-4268] Improving the separation between Metrics API and Execution
[aromanenko.dev] [BEAM-4048] Refactor COMBINE mode for reading/writing from/to Pub/Sub
[matthias] Futurize metrics subpackage
[pablo] Adding context switch to operations missing it.
[sidhom] [BEAM-2597] Flink batch ExecutableStage operator
[thw] Fix compile error in RemoteExecutionTest.
[klk] Move Release Guide to committers section
[klk] Move committer section to the bottom (we can find it :-)
[tgroh] Suppress more flaky tests.
[klk] Streamline the contribution section
[tgroh] Add a FlattenRunnerTest for a multi-consumed input
[tgroh] Use ListMultimap when creating Transform Runners
[tgroh] Migrate ImpulseEvaluatorFactory
[tgroh] Migrate FlattenEvaluatorFactoryTest
[tgroh] Migrate GroupByKeyOnly to the RR
[tgroh] Use ModelCoders in LengthPrefixUnknownCoders
[ekirpichov] [BEAM-3515] Portable translation of SplittableProcessKeyed
[ekirpichov] Fixes compile error in FlinkExecutableStageFunctionTest
[ekirpichov] Sickbays flaky test
[robbe.sneyders] Add py27-lint3 test to gradle.build
[klk] Google has scrapped research, now only AI
[iemejia] Fix error-prone warnings for HCatalogIO
[iemejia] Fix error-prone warnings and multiple cleanups based on static analysis
[lcwik] [BEAM-4280] Prevent DirectStreamObserver from blocking indefinitely if
[tgroh] Enable MapFnRunner to provide full WindowedValues
[tgroh] Implement AssignWindows in the Java SDK Harness
[tgroh] Add a static constructor to GrpcStateService
[tgroh] Log when Handler Delegation Throws
[kirpichov] fromQuery().withTemplateCompatibility() should do the same thing as
[klk] Temporarily suppress validation of ai.google links
[iemejia] Fix error-prone warnings for HBaseIO
[timrobertson100] Fix error-prone warnings for SolrIO
[timrobertson100] Fix error-prone warnings for ElasticsearchIO
[timrobertson100] Fix error-prone and some static analysis warnings in
[mergebot] Add link to portability support sheet
[timrobertson100] Fix error-prone and some static analysis warnings in XmlIO
[pablo] Fix error-prone warnings for file-based-io-tests
[kedin] [SQL] Add integration test for PubsubIO JSON -> SQL
[kedin] [SQL] Fix typos, add rewindowing and retriggering to TestPubsubSignal
[apilloud] [BEAM-3983] Add utils for converting to BigQuery types
[apilloud] [BEAM-3983][SQL] Add BigQuery table provider
[apilloud] [BEAM-4248] Copy enums from com.google.cloud
[kedin] Add SQL postCommit to main javaPostCommit
[wcn] Fix log statement to mark sink as generator of info.
[timrobertson100] Fix error-prone warnings for io/hadoop-common
[mairbek] Make Cloud Spanner transaction id public. Allows to build non-library
[daniel.o.programmer] [BEAM-4277] Adding unshaded jars used by Dataflow to poms.
[tgroh] Revert "Log when Handler Delegation Throws"
[tgroh] Log when Handler Delegation Throws
[tgroh] squash! Revert "Log when Handler Delegation Throws"
[pablo] [BEAM-4249] Fix failing validates container test
[valentyn] Add a Python 3 entry to the list of ongoing projects.
[lcwik] [BEAM-4127] Flink streaming translation for portable pipeline. (#5358)
[herohde] [BEAM-3826] Add missing package comments
[kedin] [SQL] Add support for DQL to PubsubJsonTable
[mergebot] Add Go SDK quickstart and minimal documentation page
[mergebot] Add Go to the Wordcount example
[mergebot] Fix mailing list links and docker image
[pablo] Add a dependency on future.
[timrobertson100] Fix error-prone and some static analysis warnings in
[lukasz.gajowy] [BEAM-4140] Utilize beam_location perfkit flag
[github] Update index.md
[kpeterson] Adding a ValueProvider interface.
[klk] Reorganize SQL documentation into sections
[swegner] Upgrade Byte Buddy and FindBugs annotations dependencies.
[swegner] Fix ErrorProne warnings
[swegner] Remove isTriggerSpecified from WindowingStategy equality checks.
[kpeterson] Fixing checkstyle error.
[lcwik] Add window mapping transform.
[lcwik] Use the coder from the side input PCollection definition by removing the
[lcwik] Remove garbage collection from PValueCache
[pablo] Fixing incorrect assert in dependency_test.py
[kedin] [SQL] Add integration test for Pubsub Json Table DLQ
[github] [BEAM-4300] Fix ValidatesRunner tests in Python: run with same mechanism
[tgroh] Add an IdGenerator class to java/fn-execution
[ringwalt] Improve the TaggedOutput error when a tuple is passed as a tag.
[lukasz.gajowy] [BEAM-214] Add ParquetIO
[lukasz.gajowy] [BEAM-214] Add gradle build in ParquetIO
[lukasz.gajowy] [BEAM-214] Refactore the reads to leverage FileIO
[lukasz.gajowy] [BEAM-214] Use Parquet 1.10.0 version in ParquetIO.Sink
[lukasz.gajowy] [BEAM-214] Add ParquetIOIT
[lukasz.gajowy] [BEAM-214] Update ReadFn to an non deprecated, newest Parquet API
[lukasz.gajowy] [BEAM-214] Simplify the IO by removing redundant PTransforms
[lukasz.gajowy] [BEAM-214] Implement writing slices of bytes in BeamOutputStream
[apilloud] [BEAM-3733] [SQL] Use saffron.properties for charset
[pablo] [BEAM-4331] Disable integrationTest task caching
[tgroh] Add a new DockerEnvironmentFactory Constructor
[pablo] Fixing Seed Job
[samuelw] [BEAM-3776] Fix issue with merging late windows where a watermark hold
[pablo] Add future dependency to the Dockerfile
[apilloud] [SQL] Add jdbc:beam:
[apilloud] [SQL] Use CalciteConnection as context
[axelmagn] Create jobsubmission package for job management.
[axelmagn] Add Job Invocation Interfaces
[axelmagn] Add in-memory implementation of JobService
[pablo] Remove stale TODO comments for adding archetype to java precommit
[pablo] Upgrade Gradle shadowJar plugin to latest.
[github] Mention adding a reviewer in PR template. (#5397)
[kedin] Fix Maven build
[rober] Update generated protos in Go SDK
[herohde] Invoke Go Dataflow integration tests from post-commit
[herohde] Add the Go SDK to the README
[timrobertson100] [BEAM-4342] Enforce ErrorProne analysis in hadoop IO, and static
[timrobertson100] [BEAM-4340] Enforce ErrorProne analysis in file-based-io-tests
[timrobertson100] [BEAM-4341] Enforce ErrorProne analysis in google-cloud-platform IO
[timrobertson100] [BEAM-4335] Enforce ErrorProne analysis in amazon-web-services IO
[timrobertson100] [BEAM-4339] Enforce ErrorProne analysis in elasticsearch IO
[timrobertson100] [BEAM-4355] Enforce ErrorProne analysis in XML IO
[timrobertson100] [BEAM-4338] Enforce ErrorProne analysis in common IO
[timrobertson100] [BEAM-4337] Enforce ErrorProne analysis in cassandra IO
[timrobertson100] [BEAM-4355] Reduces scope of findbugs annotations to build time only
[timrobertson100] [BEAM-4353] Enforce ErrorProne analysis in solr IO
[timrobertson100] [BEAM-4345] Enforce ErrorProne analysis in JDBC IO
[timrobertson100] [BEAM-4336] Enforce ErrorProne analysis in AMQP IO
[timrobertson100] [BEAM-4347] Enforce ErrorProne analysis in kafka IO
[dariusz.aniszewski] [BEAM-4041] Increase timeout for getting K8s LoadBalancer external IP
[timrobertson100] [BEAM-4346] Enforce ErrorProne analysis in JMS IO
[timrobertson100] [BEAM-4352] Enforce ErrorProne analysis in Redis IO
[lcwik] [BEAM-4271] Support side inputs for ExecutableStage and provide runner
[daniel.o.programmer] [BEAM-2937] Add new Combine URNs.
[tgroh] Update worker_id Documentation
[robbe.sneyders] Futurize internal subpackage
[dariusz.aniszewski] fixed typo
[mariagh] Add validation for streaming wordcount (streaming_wordcount_debugging)
[pablo] Increasing the concurrent test execution count (#5408)
[tgroh] Add a Default RemoteEnvironment Implementation
[sidhom] Require explicit IdGenerator for all DockerEnvironmentFactory
[sidhom] [BEAM-4148] Add pluggable UrlFactory to ServerFactory
[sidhom] [BEAM-4267] Docker-based JobBundleFactory
[tgroh] Add a Window-only State Context
[lcwik] Address Ben's comments.
[github] Make hash function in Coder base class more conservative.
[github] updated right documentation for HCatalogIO.write
[sidhom] Fix DataStreamsTest checkstyle error
[ankurgoenka] Updating python container version to beam-master-20180518
[wcn] Fix the test to properly run.
[ankurgoenka] Reverting fnapi container version
[ankurgoenka] Copyting dependency.py and dependency_test.py
[ankurgoenka] reformatting code
[ankurgoenka] Introducing classes
[ankurgoenka] gruping public and non public methods
[ankurgoenka] Renaming method and updating reference for file_copy and file_download
[ankurgoenka] Removing Google specific code and unused code in stager.
[ankurgoenka] Removing reference to Google/GCS/Dataflow and restructuring a bit of
[ankurgoenka] Fixing Stager Tests
[ankurgoenka] Making dependency.py use stager.py and fixing dependency_test.py test
[ankurgoenka] Moving stager to portability
[ankurgoenka] Removing stage_job_resources from dependencies.py and other minor
[ankurgoenka] Merging filehandler in stager
[ankurgoenka] renaming GCSStager to _ParameterizedStager
[ankurgoenka] Making a few methods static and moving code around
[ankurgoenka] Applying changes from rebase
[szewinho] HDFS large cluster configuration. Jenkins job updated to use large
[ankurgoenka] Fixing lint
[iemejia] [BEAM-4343] Enforce ErrorProne analysis in HBaseIO
[iemejia] [BEAM-4344] Enforce ErrorProne analysis in the HCatalogIO
[tgroh] Add a BundleFactoryOutputReceiverFactory
[tgroh] Add a DirectJobBundleFactory
[tgroh] Fix Copyright Headers in sdks/java/fn-execution
[yifanzou] rollback BEAM-3256 add archetype testing to gradle build PreCommits
[ankurgoenka] Adding ArtifactStagingFileHandler to push files to staging server
[ankurgoenka] Adding filesystem based artifact service to ULR job service
[ankurgoenka] Adding context manager to ArtifactStagingFileHandler
[ankurgoenka] Converting ArtifactServiceFileHandler to PortableStager based on
[ankurgoenka] Simplifying numeric constants
[ekirpichov] [BEAM-4375] Replaces usage of JUnit runners that do not work well with
[matthias] Futurize options subpackage
[szewinho] [BEAM-4065] Basic performance tests analysis added.
[szewinho] Fixed typo in env name, added unit tests for create report logic, slack
[szewinho] Jenkins job will run tests before executing script. Extended testing of
[timrobertson100] [BEAM-4347] Improve ErrorProne analysis in kafka IO
[timrobertson100] [BEAM-4348] Enforce ErrorProne analysis in kinesis IO
[timrobertson100] [BEAM-4349] Enforce ErrorProne analysis in mongodb IO
[timrobertson100] [BEAM-4351] Enforce ErrorProne analysis in mqtt IO
[lcwik] [BEAM-4060] Add the pom.xml and pom.properties file to the artifact when
[github] [BEAM-1755] Add a directory with build-specific scripts to clear up the
[ankurgoenka] Logging and naming changes
[ankurgoenka] Enhancing test cases
[tgroh] Add an InProcessEnvironmentFactory
[timrobertson100] [BEAM-4348] Removes double-checked locking to aid readability
[ankurgoenka] Updating the dostring
[tgroh] Update InProcessManagedChannelFactory
[tgroh] Add a RemoteStageEvaluatorFactory
[timrobertson100] [BEAM-4354] Enforce ErrorProne analysis in tika IO
[iemejia] Fix build with Maven (Again!)
[lukasz.gajowy] [BEAM-214] Apply reviewer's suggestions
[timrobertson100] [BEAM-4347] Minor changes following PR review
[timrobertson100] [BEAM-4347] Correction of typo following PR review
[tgroh] Use WindowOnlyContext in ReduceFnContextFactory
[tgroh] Update GroupAlsoByWindowEvaluatorFactory
[tgroh] Move InProcessEnvironmentFactory to java-fn-execution
[klk] Autoformat sdks/java/extensions/sql
[klk] Add spotless config to global gradle file
[klk] Activate spotless for SQL module
[boyuanz] Solve BEAM-4393: Missing copyright notices for shaded packages
[iemejia] Remove repeated GCPIO dependency in SQL module (Again!)
[iemejia] Enforce error-prone in ParquetIO, fix missing javadoc and refine deps
[katarzyna.kucharczyk] Replaced created datetime to jenkins build id in kubernetes namespace.
[schroederc] Use ElementEncoders for beam.Create
[boyuanz] Change bulid rules of sdks/java/harness to make harness as a uber-jar
[swegner] Remove DoFnTester from testing guide.
[robertwb] Slowly ramp up wait time in state sampler.
[tgroh] Move the Local Artifact Services to the DirectRunner
[tgroh] Migrate the ReferenceRunnerJobServer to runners-direct
[tgroh] Fix License Headers
[schroederc] Add test case for proto oneof fields in create_test.go
[tgroh] Add a test to show Flatten Execution
[apilloud] [SQL] Fix transitive dependencies
[apilloud] [SQL] Run tests against actual jar
[boyuanz] [BEAM-4393]: Copy root repo's LICENSE & NOTICE into shadowJar
[herohde] Tweak Go integration test driver
[tgroh] Update Docker Command
[iemejia] [BEAM-4306] Enforce ErrorProne analysis in apex runner
[daniel.o.programmer] [BEAM-4406] Updating Java portable environment major version.
[daniel.o.programmer] [BEAM-4406] Updating Python portable environment major version.
[ehudm] Add a custom _url_dirname for local filesystems.
[tgroh] Add a Portable TransformEvaluatorRegistry
[tgroh] Consistent Parameter Order in Evaluator Factories
[tgroh] Make ImmutableExecutableStage Public
[mergebot] Fix typo in becoming-a-committer docs
[iemejia] Fix broken build (again third time this week yay!)
[timrobertson100] [BEAM-4361] Document usage of HBase TableSnapshotInputFormat
[jbonofre] Fix Maven parent version in ParquetIO
[iemejia] [BEAM-4312] Enforce ErrorProne analysis in gearpump runner
[mergebot] Add Apache Kudu IO to the planned IOs table
[mergebot] Add next Java LTS to the WIP section of the contribution guide
[mergebot] Add PubSubIO to Python IO table row
[iemejia] [BEAM-4307] Enforce ErrorProne analysis in
[mergebot] Change ParquetIO status in built-in I/O section
[iemejia] [BEAM-4308] Enforce ErrorProne analysis in runners-core-java
[ammons] Say that merging windows are not supported in Distinct.java.
[lcwik] [BEAM-4411] Update Jackson version to latest to pick up security fixes.
[kenn] Implement UNNEST operator for SQL
[iemejia] [BEAM-4317] Enforce ErrorProne analysis in runners/reference/java
[iemejia] Refine dependencies, make new ones explicit and minor maven plugins
[iemejia] [BEAM-4323] Enforce ErrorProne analysis in sketching extensions
[iemejia] [BEAM-4324] Enforce ErrorProne analysis in sorter extensions
[iemejia] [BEAM-4321] Enforce ErrorProne analysis in join-library extensions
[kenn] Add equals, hashCode, and structuralValue to Nexmark model objects
[thw] [BEAM-4297] Streaming executable stage translation and operator for
[thw] [BEAM-4297] Add serialization test and make output mapping stable.
[lcwik] Add link to design docs and template to the contribution guide.
[timrobertson100] [BEAM-4389] Enable partial updates in ElasticsearchIO
[kenn] Force load of Calcite JDBC driver (avoids internal error)
[kenn] Explicitly load Beam JDBC driver before test
[iemejia] [BEAM-3813] Support (de)serialization of S3 encryption options via JSON
[pablo] [BEAM-4426]: Addressed DataflowDistributionAccumulatorTest failure on
[aaltay] Support installing Beam SDK from a wheel distribution in SDK containers.
[cademarkegard] fix errorprone annotations and enable failOnWarning
[lukasz.gajowy] Add Performance Testing to "Work in Progress" section
[iemejia] Update postgres version to 42.2.2 (Java 8)
[coheigea] Update README to include virtualenv in the build instructions
[iemejia] [BEAM-4356] Enforce ErrorProne analysis in nexmark
[kenn] Finish RowType -> Schema rename
[kenn] Add convenience aliases for nullary type constructors
[kenn] Add MAP field support to Schema.Builder
[kenn] Add static factories for array, row, map types
[kenn] Schema fields are non-null by default
[kenn] TestUtils.RowsBuilder to use FieldType instead of TypeName
[kenn] Clean up CalciteUtils conversions a bit
[pablo] enable failOnWarning for build-tools
[apilloud] Run everything with shadowJar
[apilloud] guava is on kinesis public API
[apilloud] Fix direct runner shadow config
[kenn] Add more field conveniences to Schema.Builder
[kenn] Remove extraneous RowSqlTypes
[pablo] [BEAM-4322] Enforce ErrorProne analysis in protobuf extensions project
[apilloud] Run test on shadowJar on a flag
[kenn] Remove Schema.TypeName.type()
[kenn] Remove unsafe builder methods from FieldType
[apilloud] [SQL] Build mega jar for JDBC
[apilloud] [SQL] JDBC Class Loader IT
[ankurgoenka] proto changes to support artifact_staging_id
[ankurgoenka] Doc changes
[ankurgoenka] Renaming artifact_staging_id to staging_session_token
[ankurgoenka] Fixing python code for staging_session_token proto changes
[marek.simunek] [BEAM-4368] add euphoria java 8 dsl documentation section
[coheigea] Avoid fully qualified class names, when there is an existing (static)
[coheigea] Fixing ErrorProne MissingOverrides warnings
[kedin] [SQL] Fix PubsubsJsonIT
[apilloud] [SQL] Init compiler factory with JDBC class loader
[pablo] clean generated files prior to lint
[ankurgoenka] Fixing java code for staging_session_token proto changes
[ankurgoenka] Go proto update
[amaliujia] Add LIKE operator to Beam SQL.
[kedin] [SQL] Replace planner.compilePipeline() with sqlEnv.parseQuery()
[kedin] [SQL] Add sqlEnv.executeDdl()
[kedin] [SQL] Make planner package-private
[kedin] [SQL] Add factory methods to BeamSqlEnv
[kedin] [SQL] Rename ReadOnlyTableProvider
[coheigea] Fixing some ErrorProne warnings
[github] print() is a function in Python 3
[cademarkegard] [BEAM-4303] Enforce ErrorProne analysis in examples project
[boyuanz] Address BEAM-4328: :beam-sdks-java-io-google-cloud-platform:test failure
[kedin] [SQL] Wrap SQL parsing exceptions in ParseException
[tgroh] Add an abstraction for State and Timers
[tgroh] DirectRunner Cleanups
[tgroh] Link up the Portable DirectRunner
[tgroh] Reuse ID Generators across Environments
[tgroh] Use the Current Users Container as the Environment
[aromanenko.dev] [BEAM-4421] Fix for issue with reading s3 files using ParquetIO
[github] Fix flaky comparison in log_handler_test.py
[tgroh] Use the Current Users Container as the Environment
[tgroh] Enable Docker in the PortableDirectRunner
[tgroh] Move DirectJobBundleFactory to fn-execution
[tgroh] Cleanups to the PortableDirectRunner
[tgroh] Cleanups and Wiring for the ReferenceRunnerJobService
[amyrvold] [BEAM-4253] Update dataflow worker to fix ParDoTest fails in Dataflow
[kedin] [SQL] Add ParseException to sqlEnv.explain()
[kedin] [SQL] Add integration tests for BigQuery writes
[kedin] Add TestBigQuery rule
[tgroh] Use an UnsupportedArtifactRetrievalService
[altay] Move assert_that's new keyword argument to the end
[jhsueh] [BEAM-4320] Enforce ErrorProne analysis in jackson extensions project
[jiangkai] introduce optimized logic plan
[jiangkai] update
[jiangkai] fix unit tests failed
[iemejia] [BEAM-4310] Enforce ErrorProne analysis in
[jiangkai] address comments
[relax] Now that Dataflow is updated, we can finish updating tests and usages to
[szewinho] [BEAM-3214] Add integration test for HBaseIO.
[iemejia] [BEAM-4137] Remove MongoDB specific options from
[ankurgoenka] Code review comments fixes
[ajamato] Add new metrics protos based on s.apache.org/beam-fn-api-metrics
[jiangkai] separate rules into sets
[amaliujia] Address comments: 1. Use SqlFunction.like to implement like operator. 2.
[apilloud] [SQL] Use AutoService for jdbc
[apilloud] [SQL] AutoService for TableProvider
[apilloud] [SQL] Don't cache IT results
[apilloud] [SQL] Test that TextIO and Direct Runner work
[apilloud] [SQL] Add context class loader hack
[rober] [BEAM-4276] Add combiner lifting support to Go SDK
[rober] fixup! Address comments.
[herohde] [BEAM-4474] Ensure unbounded Dataflow jobs are submitted as streaming
[herohde] Fix build break due to artifact staging change
[relax] Add a test with ReferenceRunner
[aromanenko.dev] Added design docs page
[aromanenko.dev] Added first intro sentence.
[aromanenko.dev] Moved one doc to other section and added several new docs
[aromanenko.dev] Added more docs
[github] [BEAM-4480] Fixed deprecated method invoking
[lcwik] [BEAM-4481] Remove duplicate definitions of dependencies.
[herohde] [BEAM-4475] Add container boot code to Go precommit
[herohde] [BEAM-4475] Add Go submodule build to Go precommit
[axelmagn] Fix ProvisionInfo in DockerJobBundleFactory
[github] Improve default value for experiments set in RuntimeValueProvider
[cademarkegard] [BEAM-4326] Enforce ErrorProne analysis in the fn-execution project
[alan] [BEAM-4423] Mark pull requests stale after 60 days; close 7 days after
[jbonofre] Moving to 2.6.0-SNAPSHOT on master branch
[lukasz.gajowy] [BEAM-4415] Add jenkins job for HDFS ParquetIOIT
[lukasz.gajowy] [BEAM-4137] Extract file-related options from IOTestPipelineOptions
[lukasz.gajowy] [BEAM-4137] Extract postgres related options from IOTestPipelineOptions
[timrobertson100] [BEAM-4311] Enforce ErrorProne analysis in Flink runner
[github] [BEAM-4517] Add maven status in README
[jiangkai] add expression boolean casting
[coheigea] Removing extraneous whitespace around the equals operator
[katarzyna.kucharczyk] Changed a namespace to parsed job name with current job number.
[jiangkai] add boolean type for agg function
[jiangkai] support EXISTS operator
[github] Update Environments.java
[github] Remove unneeded collection import.
[github] Combine immutability type fixes. (#3)
[github] fixup!
[lcwik] BEAM-3876 avoid NPE if checkpoint is null in an unbounded source
[lcwik] testing npe fix and exception rethrow in unit tests
[lcwik] Update https://github.com/apache/beam/pull/4894 to correspond with
[github] [BEAM-4276] Add missing scope line
[jiangkai] explain plan unit tests
[lcwik] [BEAM-4517] Add PyPI status badge.
[aljoscha.krettek] [BEAM-3905] Update Flink Runner to Flink 1.5.0
[kedin] [SQL] Manually read rows in TestBigQuery
[robertwb] [BEAM-3981] Cleanup of coders futurization.
[robertwb] Use ByteCountingOutputStream for estimate_size.
[kedin] Add polling assertion support to TestBigQuery
[kedin] [SQL] Add Pubsub to BigQuery E2E integration test
[sidhom] Add reference to ArtifactSourcePool implementation
[sidhom] Clean up FlinkExecutableStageFunction and tests
[sidhom] [BEAM-4523] Implement batch flink executable stage context
[kedin] [SQL] Delete BeamSql
[kedin] [SQL] Rename QueryTransform to SqlTransform
[mergebot] Add SQL area to ongoing projects
[robertwb] Reject unknown arguments in WindowInto.
[mergebot] [BEAM-4488] How to install Python SDK from source [BEAM-4489] Document
[mergebot] More changes
[mergebot] updates after review
[mergebot] Updating release guide to use Gradle commands.
[mergebot] Addressing comments. Improving commands.
[mergebot] Addressing comments. Improving commands.
[mergebot] Update release guide for publishing source zip from GitHub
[mergebot] Addressing TODOs
[mergebot] Address comments.
[mergebot] Improving.
[mergebot] Add more updates
[mergebot] More updates
[mergebot] Addressed Alan's comments
[mergebot] Add instructions about how to stage python binaries
[mergebot] copy renamed zi[ file to sdks/python/staging subdir
[mergebot] Update gradle publish with --no-parallel --no-daemon
[kenn] Suppress immutability warning caused by LoadingCache in enum instance
[kenn] Make BeamSqlExpression for operators, use it for string operators
[ekirpichov] Add retrieval_token to GetManifestRequest
[ekirpichov] Refactor code that accesses staging_token
[ekirpichov] Regenerate go protobufs
[kenn] Replace project and filter with calc
[kenn] Disable CalcRemoveRule
[szewinho] Improve consistency of hadoop and hbase scripts, fix shellcheck issues
[iemejia] Fix build (missed auto service for SQL) and model-fn-execution on Flink
[iemejia] [BEAM-4318] Enforce ErrorProne analysis in Spark runner
[wcn] Encode position ids.
[robertwb] Explain why module namespace is modified in Cython codepath.
[apilloud] [SQL] Make getString more generic
[ankurgoenka] Beam File System based Artifact Staging Service
[ankurgoenka] BeamFileSystemArtifactStagingService Tests
[ankurgoenka] Enhancing test case and error handling.
[ankurgoenka] Enhancing testcases and incorporating review comments.
[ankurgoenka] Review comments Fix
[ankurgoenka] Rebasing
[apilloud] [SQL] Support unquoted table types
[herohde] Remove new Go precommit dependencies due to failures
[jiangkai] address conflicts and comments
[jiangkai] remove volcano plan function
[klk] [SQL] Separate assertions for each expression in ExpressionChecker
[klk] Remove StringOperatorsTest in favor of SQL syntax tests
[mergebot] Add Python Kafka to list of in-progress I/O transforms.
[pablo] Upgrade Gradle to version 4.8
[ehudm] Remove with_attributes keyword from ReadFromPubSub.
[xiliu] [BEAM-3079] Update samza-runner with more features and improvements
[xiliu] Add missing apache headers
[swegner] Use new DSL methods for generated POM metadata
[lcwik] [BEAM-4540] Migrate junit/hamcrest to provided scope.
[swegner] [BEAM-925] Fix FindBugs violations in Dataflow Runner and re-enable
[swegner] [BEAM-4313] Fix ErrorProne warnings and enable validation in Dataflow
[ankurgoenka] Followup PR for #5591
[swegner] Add support for pre-commit trigger paths
[swegner] Set path triggers for existing pre-commit test jobs
[rober] Add graph.Fn encoding functions.
[robinyq] Defer calling formatTimestamp() to achieve better performance
[ccy] [BEAM-4549] Use per-pipeline unique ids for side inputs in
[altay] Explain experimental/deprecated from users perspective.
[jiangkai] call SqlFunctions in numeric type conversion
[github] Update Dockerfile for new setup.py changes.
[robbe.sneyders] Futurize utils subpackage
[iemejia] [BEAM-4551] Update spark runner to Spark version 2.3.1
[iemejia] Fix maven build error on sdks/java/io/google-cloud-platform module
[robinyq] Throws exception directly instead of calling checkState()
[github] Remove GPL findbugs dependency (#5609)
[lcwik] Clean-up BeamFnDataBufferingOutboundObserver since Dataflow has migrated
[jiangkai] unit test for BuildExpression literal casting
[aaltay] Futurize portability subpackage (#5385)
[altay] Futurize unpackaged files
[altay] resolved six.string_types equivalency
[altay] Futurize testing subpackage
[altay] Futurize tools subpackage
[altay] Remove old_div
[swegner] Revert "Merge pull request #5611: [BEAM-4445] Filter pre-commit
[robertwb] [BEAM-4546] Implement hot key fanout for combiners.
[robertwb] Use discaring mode for first level of combine.
[echauchot] [BEAM-4283] Export nexmark query execution times to BigQuery
[echauchot] [BEAM-4283] Make BigQueryIO Services classes public and experimental and
[echauchot] [BEAM-4283] Fix PerfsToBigQueryTest and serialization issue in
[echauchot] Add Etienne Chauchot's ongoing work to the contribute page.
[fvoichick] Fixed link to nonexistent IsBounded.apply method
[fvoichick] Fixed documentation of default window size in example code
[fvoichick] Fixed inappropriate Javadoc comments
[aaltay] [BEAM-4333] Add integration tests for python mobile games (#5630)
[jiangkai] ignore explain unit test
[kedin] [SQL] Add INSERT tests to JdbcDriverTest
[kedin] [SQL] Add complex type test to JdbcDriverTest
[kedin] [SQL] Make field names case-sensitive
[robertwb] Suppress warning.
[robertwb] Implement BeamFileSystemArtifactSource
[robertwb] Add job submission capabilities to Flink runner.
[robertwb] Update to new JobInvoker/Invocation interface.
[robertwb] Remove references to Flink distributed file cache.
[robertwb] A bit more documentation.
[robertwb] Address comments on Flink code.
[kirpichov] Removed some dead code, fixed some warnings
[kirpichov] Adds tracing of watermarks in direct runner
[kirpichov] Fixes typo in class name
[kirpichov] Simplifies GBK expansion code a bit
[kirpichov] Fixes bug: Processes values arriving for a GBK key sequentially
[kirpichov] Introduces BundleProgressHandler.
[swegner] Fix ErrorProne violations in direct runner and enable failOnWarnings
[robertwb] [BEAM-4565] Fix hot key fanout in the face of combiner lifting.
[robertwb] Remove trailing whitespace.
[kirpichov] [BEAM-4473,BEAM-4281] Makes ReferenceRunner use a Direct stream observer
[ccy] [BEAM-4549] Revert #5623
[kirpichov] [BEAM-4291] Implements distributed artifact retrieval
[kirpichov] Deletes ArtifactSource
[thw] Fix Flink runner Maven build.
[ccy] [BEAM-4549] Fix side inputs for streaming pipelines on Dataflow
[mergebot] [BEAM-2852] Update Nexmark documentation with Kafka support
[mergebot] [BEAM-4479] Fixed document for Coder
[swegner] Fix ErrorProne violations in fn harness and enable failOnWarnings
[swegner] fixup! Fix ErrorProne violations in fn harness and enable failOnWarnings
[pablo] Improving state sampler Cython annotations.
[pablo] Adding msecs tracking to side inputs iterator
[pablo] Adding the sampling thread to side input microbench.
[pablo] Adding flag to mask change
[pablo] Fixing all lint raising-format-tuple warnings.
[github] Make build_rules.gradle version of shadow match build.gradle
[swegner] Temporarily exclude website sources from RAT
[swegner] fixup! Fix ErrorProne violations in fn harness and enable failOnWarnings
[ehudm] Increase Jenkins build retention from 14 to 30 days.
[ankurgoenka] Fixing concurrency bug in BeamFileSystemArtifactServicesTest
[iemejia] Fix Maven build 7
[iemejia] [BEAM-4570] Use pagination in
[thw] Fix ValidatesRunner_Apex #5604
[iemejia] [BEAM-4399] Get Cassandra options out of IOTestPipelineOptions
[iemejia] [BEAM-4399] Refactor CassandraIOIT to be writeThenRead
[iemejia] [BEAM-4399] Update Cassandra driver to version 3.5.0
[kenn] Fix name of heuristic JAVA_CLASS_TO_TYPENAME
[kenn] [SQL] Allow explicit return type in ExpressionChecker
[kenn] DSL-level tests for standard operators
[kenn] Add test that all supported operators are tested
[kenn] [SQL] Make parse failure slightly more actionable
[pablo] Upgrading python dependency on httplib2
[kenn] Fix case sensitivity in JdbcDriverTest
[ehudm] Jupyter notebook for monitoring precommit job times
[lcwik] [BEAM-4576] Add hamcrest dependencies to this scope that was moved to
[mergebot] Add link to @RequiresStableInput design doc
[matthias] Futurize examples subpackage
[apilloud] [SQL] Add parse code for SET
[apilloud] [SQL] Plumb PipelineOptions through BeamRelNode
[apilloud] [SQL] Plumb PipelineOptions through BeamSqlCli
[apilloud] [SQL] Remove unused BeamIOType
[ccy] [BEAM-4582] Fix streaming Create transform on Dataflow
[kirpichov] Propagates artifact retrieval token in Flink runner and to the Java
[ankurgoenka] load beam fileSystems in FnHarness
[axelmagn] Implement new staging token contract in java SDK
[robertwb] Add Impulse primitive to Python SDK
[robertwb] Expand Read to Impulse | ParDo(Split) | ParDo(Read) in Python SDK
[robertwb] Fix default docker image name.
[kirpichov] Add an AddHarnessIdInterceptor
[kirpichov] Remove the runners/reference InProcessManagedChannelFactory
[kirpichov] Allow ManagedChannelFactories to attach Interceptors
[kirpichov] Populate and Use Harness IDs in the Java Harness
[apilloud] [SQL] Set application name to BeamSql
[apilloud] [SQL] Correctly represent pipeline
[apilloud] [SQL] Cleanup table IO functions
[apilloud] [SQL] Simplify overly verbose names
[apilloud] [SQL] Add each Calcite node to Beam graph once
[apilloud] [SQL] Seperate parse from pipeline construction
[swegner] fixup! Fix ErrorProne violations in fn harness and enable failOnWarnings
[kirpichov] Fixes checkstyle
[chamikara] Fixes broken links for downloading source releases.
[lcwik] [BEAM-4592] Make Dataflow understand kind:varint as a well known since
[chamikara] Fixes null pointer exception on empty pubsub message data (#5016)
[kirpichov] A couple more trivial changes extracted from #5662
[lcwik] Update Dataflow container images
[katarzyna.kucharczyk] Created static retry method which accepts retriable method, amount of
[robertwb] Update impulse-based read.
[kenn] Make BeamRelNode take a PCollectionList<Row> always
[rober] Fix build break from PR 5676, which modified method signatures.
[migryz] Implemented working prototype for PostCommit jobs
[migryz] Update post commit jobs to utilize new helper methods
[ankurgoenka] Fix cache references in portable flink runner
[robertwb] [BEAM-4150] Extract IO coders from the RemoteGrpcPort.
[pablo] Adding ignore parameters for gitattributes
[swegner] Remove submodule
[robertwb] Fix and test global combine with fanout.
[migryz] Fix typo in _ghprb suffix
[migryz] Update suffix to be PullRequest
[ccy] Revert #5689 to fix build
[ankurgoenka] Remove unused import
[github] Make suffix shorter
[kirpichov] Make ImmutableExecutableStage constructors public
[kirpichov] [BEAM-4285] Implement Flink batch side input handler
[ankurgoenka] Fixing key lookup in FlinkBatchSideInputHandlerFactory.SideInputHandler
[ekirpichov] Introduces PipelineValidator that checks the well-formedness of a
[cademarkegard] [BEAM-4325] Enforce ErrorProne analysis in the SQL project
[echauchot] [BEAM-4283] Fix naming of the BigQuery fields
[coheigea] Removing some null checks, where we already know that the variable in
[iemejia] [BEAM-3314] Set correctly host and port on RedisIO
[iemejia] [BEAM-3314] Fix error-prone warnings and add extra test for Read
[matthias] Futurize direct runner
[matthias] Futurize dataflow,experimental,job,portability and test subpackages of
[matthias] Futurize complete runners subpackage
[matthias] Cleanup futurize runners
[lcwik] Revert "Revert #5689 to fix build"
[github] Add migration notice to apache/beam website README.me
[robertwb] Always consider Read to be a primitive for dataflow pipelines.
[migryz] Point gearpump to master
[robertwb] [BEAM-4605] Remove subtransform of GBK.
[robbe.sneyders] Futurize typehints subpackage
[kenn] Migrate build_rules.gradle to buildSrc plugin
[amaliujia] support unbounded limit.
[aaltay] Fix backlog bytes reported by CountingSource. (#5719)
[mairbek] Fixed utf8 encoding in Cloud Spanner mutation encoder
[robertwb] Don't require a staged SDK for the Python portable worker.
[pablo] Enabling gradle build to receive extra buildscript repositories
[kenn] Remove build_rules.gradle
[daniel.o.programmer] [BEAM-2937] Comment Combine Grouped Values URN.
[daniel.o.programmer] [BEAM-2973] Moving CombineValues URN to different enum.
[robertwb] Add back old code path for dataflow.
[klk] Fix errors in BeamModulePlugin perf test section
[mergebot] Adds a dependencies guide to Beam Website
[iemejia] Fix maven compile for the 8th time (yay!)
[iemejia] Fix errorprone warnings on extensions/google-cloud-platform-core module
[iemejia] Enable errorprone globally
[klk] Fix Gearpump validatesRunner gradle config
[robertwb] Stage empty manifest if a staging session is given.
[migryz] [BEAM-4559] Add latest post-commit tests status to GH PR
[sidhom] Add registrars for the portable runner
[sidhom] Add endpoint logging for PortableRunner
[github] dataflow_runner.py: use the pbegin's pipeline
[github] materialize_test.go: delint: non-idiomatic block
[klk] Fix docker gated code paths
[altay] Add instructions about how to perform release validations
[ehudm] Parallelize Java precommits
[scott] Spread jenkins load by randomizing daily cron schedule
[amaliujia] logical operator tests
[kirpichov] Adds support for SDF in ULR and the Java SDK.
[klk] Allow specifying delimiter in TextIO
[robbe.sneyders] Address PR comments
[klk] Migrate some date expressions to operators
[klk] Use label "stale" when closing stale PRs
[kenn] Add support for "lines" format text tables
[kenn] Beam SQL shell prompt is now 'BeamSQL>'
[kenn] Override SqlLine.properties to customize welcome and help messages
[pablo] Improving documentation for Metrics results
[xiliu] Finalize Samza Runner with gradle and findbugs issues
[amaliujia] add comparison operator test
[ekirpichov] Address easy review comments
[xiliu] Fix a unit test timeout and minor issues in pom.xml
[kirpichov] Handles memoization of encoded key/window in StateAccessor
[kirpichov] Simplifies API of WireCoders and LengthPrefixUnknownCoders.
[sidhom] [BEAM-4615] Flink job server wrapper and shadow jar
[apilloud] [SQL] Don't wait on Unbounded PCollections
[kirpichov] Address comments
[xiliu] [BEAM-4640] Samza runner postcommit ValidatesRunner job
[aaltay] [BEAM-4333] Add integration tests for python mobile games (#5747)
[jiangkai] [BEAM-4602][BEAM-4598] Date type comparison
[github] Cythonizing a few hot calls (#5733)
[daniel.o.programmer] [BEAM-3708] Adding Combine component implementations to Java SDK
[iemejia] Fix maven build part 9
[xiliu] Remove unused previous names
[daniel.o.programmer] Quick fix, removing commented out code.
[altay] Publish 2.5.0 release
[altay] Change download link for 2.4.0 version to use the archive link
[altay] improvements to fanout
[yifanzou] [BEAM-4302] Add Jenkins job to check Beam dependencies and generate
[ekirpichov] Fixes Flink ValidatesRunner and a compile error
[scott] Split pre-commit tests into separate jobs based on trigger condition.
[scott] Filter pre-commit triggering based on touched files
[scott] Rename setPostCommit to setAutoJob
[amyrvold] [BEAM-4645] Cleanup docker images on failure
[amaliujia] replace byte[] with String as Map key.
[scott] Fix ErrorProne warnings in sdks-java-core project
[scott] Enable ErrorProne analysis for all Java projects.
[scott] Update Findbugs annotations dependency to version with
[scott] Enable failOnWarnings for all projects.
[pablo] Removing debug string from UserScore.java
[scott] Add checkstyle validation to ban usage of @SuppressFBWarnings.
[amyrvold] [BEAM-3261] Fix pipeline failures by waiting for logging server to be
[github] Revert "improvements to fanout"
[klk] Apply rename of JobBuilder to Samza VR job
[xiliu] [BEAM-4641] Samza runner postcommit status in PR template
[kenn] Add bundle-ready SQL shell using application plugin
[relax] Use ByteBuddy to generate a coder class for a specific Schema.
[relax] Subclass proper generic type, removing need for extra typecast.
[relax] Address code-review comments.
[scott] Rename refactored Precommit source files to remove GradleBuild suffix.
[herohde] Run gofmt
[kenn] Reduce stdout spam from BeamModulePlugin
[scott] Update Pre-commit job names tracked in precommit_job_times.ipynb
[lcwik] [BEAM-4661] Define well known timer URN.
[herohde] [BEAM-4635] Avoid cross-compiling worker binary if linux-amd64
[klk] Increase website job timeout to 100 by default
[scott] Add Apache license headers
[ankurgoenka] Adding flinkMarterUrl as an option
[robertwb] Give the root transform in Python a non-empty name.
[herohde] [BEAM-4635] Do not remove worker binary on Go Dataflow staging
[amaliujia] Improve BigQuery write integration tests and support TIMESTAMP SQL type
[ryan.blake.williams] [BEAM-2810] use fastavro-blocks in Avro IO
[klk] Fix accidental license header javadoc comments
[klk] Fix HTML error in ApproximateQuantiles
[klk] Turn on spotless java everywhere
[aaltay] [BEAM-4606] Pin pytz version (#5751)
[scott] Update pre-commit filter to include migrated build rules
[klk] Do not email individuals about failures in Jenkins
[klk] Pipeline options in JDBC URI; default userAgent=BeamSQL
[klk] Spotless Java everywhere!
[klk] Simplify checkstyle now that spotless is in play
[klk] Temporarily disable JavadocParagraph and LineLength checkstyle
[scott] Remove blanket website from rat and add appropriate exclusions.
[github] improvements to fanout (#5800)
[herohde] Do not launch containers with transient semi_persistent_dir. Fails on
[herohde] Go-on-Flink: treat artifact staging EOF error as ok
[herohde] Go-on-Flink: make transform UniqueNames _globally_ unique
[herohde] Go-on-Flink: remove internal java runner class hack
[herohde] Go-on-Flink: fix swapped arguments in artifact staging
[herohde] Go-on-Flink: use clearer error if the artifact retrieval token or
[herohde] Go-on-Flink: fix Go to actually send worker_id gRPC header
[lukasz.gajowy] Remove mvn leftovers from jenkins IOIT job definitions
[klk] Rearrange checkArgument to help findbugs
[chamikara] Reduces the size of pretty string of BigQuery jobs
[mariagh] Add Python Nexmark suite
[herohde] CR: Don't enforce non-empty control worker_id yet
[xiliu] Make Samza runner in lexicographical order
[robertwb] [BEAM-4285] Extend side input handlers to handle multiple access
[relax] Rename RowType -> Schema and move to new schemas package.
[relax] Add classes for SchemaRegistry and SchemaProvider.
[relax] Add SchemaCoder, some more detail in SchemaProvider/Registery, and
[relax] Update to comment.
[relax] Integrate Schemas into PCollection Coder inference.
[relax] Add setSchema convenience method.
[relax] Fix compilation errors.
[relax] Refactor Schema: Schema now is a fully-specified based on primitive
[relax] Implement SchemaCoder and fix up schema classes.
[relax] Flesh out FieldAccessDescriptor.
[relax] Start plumbing through DoFn.
[robertwb] Rename several MultimapSideInput* classes to more generic SideInput*.
[relax] Start editing signatures
[relax] Finish plumbing schemas through.
[relax] Add test for schema PCollection.
[relax] Add withSchema to Create.java
[relax] Plumb schemas through OutputReceiver.
[relax] Add an implementation of SchemaProvider.
[relax] Add SchemaRegistry and a test for schema inference. Change schema
[relax] Add unit test for SchemaRegistry.
[relax] Add FieldAccessDescriptorTest
[relax] Add test for Create changes.
[relax] Plumb FieldAccess through.
[relax] Make sure everything is marked experimental with Kind.SCHEMAS
[relax] Add graph-construction time verification of schemas.
[relax] Fix build and style failures.
[relax] Address code-review comments.
[relax] Fixup after merge.
[relax] Apply spotless autoformat.
[lcwik] [BEAM-4659] Add well known timer coder to Java SDK.
[kirpichov] Makes fuser materialize PCollections used as side inputs
[relax] Fix compilation error introduced by merge.
[ankurgoenka] Uploading job server jar when running on remote flink environment
[ankurgoenka] Passing files to stage on flink explicitly while creating the
[ankurgoenka] Spotless
[robertwb] Remove obsolete comment about multimap fitting into memory.
[relax] Fix CheckStyle errors.
[ankurgoenka] Job service additionally need google cloud dependency to use GCS
[relax] Fix Apex runner.
[relax] Fix more breakages.
[relax] run spotless.
[kirpichov] Makes Python PortableRunner usable via PipelineOptions
[amaliujia] Remove unused system.out.println
[ankurgoenka] TODO to add more file systems
[kirpichov] Brings back hack for embedded FnApiRunner
[robertwb] [BEAM-3883] Stage files in the portability runner.
[robertwb] Fix tests due to api change.
[kirpichov] Changes portableWordCount gradle task to be runnable at master
[ankurgoenka] Adding non standard side input urn to flink
[relax] Fix issues.
[relax] Fix Spark issue.
[kirpichov] Fixes spotless errors at master
[mariagh] Allow streaming mobile examples to read from either a topic or a
[relax] Address code-review comments and fix a failing test.
[relax] Fix Apex runner.
[echauchot] [BEAM-2850] Add Nexmark PostCommit runs for spark, flink and direct
[relax] Change ParDoSchemaTest to ValidatesRunner, and exclude unsupported
[relax] Make worker changes backwards compatible with DataflowRunner. Once
[mariagh] Address review comments
[amaliujia] Support BigQuery read from SQL.
[relax] Add another backwards-compatible method.
[ekirpichov] Makes FileIO.match watermark advance even without new files
[pablo] Test case for providing experiments to pipeline
[kirpichov] [BEAM-4689] Reverts change of SDF key type
[daniel.o.programmer] [BEAM-3708] Adding grouping table to Precombine step.
[daniel.o.programmer] [BEAM-3708] Simplifying precombine grouping tables.
[klk] Add integrationTest task for GCP IO module
[klk] PubsubIO: Always use GCP project from PipelineOptions for subscriptions
[klk] Use BagState in PubsubSignal since it doesn't use SetState methods
[klk] Replace println with logging
[klk] More logging when PubsubJsonIT fails
[klk] Add start signal to TestPubsubSignal
[klk] Use futures in PubsubJsonIT LIMIT test
[klk] Integration test of Pubsub public dataset
[klk] Use test PipelineOptions in PubsubJsonIT LIMIT test
[klk] Sickbay PubsubReadIT on Dataflow
[klk] Fix import order
[mariagh] Add query 1 & 2
[mariagh] Clean up
[mariagh] Cleaup
[mariagh] Fix linter errors
[mariagh] Address review commnets; Reuse topic for queries
[github] Return timeout value to 180
[relax] Fix build.
[relax] Add SchemaProviderRegistrar class.
[klk] Revert accidental clobber of MinimalWordCount.java
[relax] Exclude schema tests from Samza runner.
[klk] Update BigQueryAvroUtils to support BigQuery's NUMERIC type.
[relax] Add DefaultSchema and fix failures turned up by plugins.
[klk] Convert Beam Row to Avatica Row in BeamEnumerableCollector
[kenn] sqlline dep to 1.4.0
[kenn] Fix broken javadoc
[kenn] Remove more extraneous printlns from build
[kenn] Remove aliased tables in Nexmark SQL query 5
[lcwik] Revert "Enabling gradle build to receive extra buildscript repositories"
[lcwik] Don't enable plugins which require you to be online when run in
[kenn] fixup! Remove aliased tables in Nexmark SQL query 5
[lcwik] [BEAM-4126] Delete Maven build files.
[kenn] Simplify/fix windowing check in BeamSortRel
[robertwb] Fix global hot key fanout setting.
[scott] Increase concurrency in ValidateRunner execution
[lcwik] [BEAM-3971, BEAM-4284] Remove fromProto for Pipeline and PTransform
[robinyqiu] Support @RequiresStableInput on Dataflow runner in Java SDK
[kenn] Run SQL operator tests through JDBC as well
[kenn] Test retrieval of TIMESTAMP via JDBC
[kenn] Instantiate $SUM0 as a SUM operator
[ankurgoenka] Read worker harness image name from pipeline option for portable runners
[ankurgoenka] Spotless
[kenn] Instantiate "+" as DATETIME_PLUS
[ankurgoenka] Support only a single environment in SdkComponents
[ankurgoenka] Renaming createEnvironment to createOrGetDefaultEnvironment.
[lcwik] [BEAM-4716] Remove findbugs declarations in build.gradle files since it
[relax] Address review commemnts.
[ccy] [BEAM-4007] Fix TODO style in typehints.py
[relax] Add schema inference from POJO.
[relax] Add BYTES type to schema, and POJO inference.
[relax] Add automatic getter/setter inference from POJOS. * Recursive POJOs
[relax] Handle setter for ByteBuffer fields. Setter will be called with a byte[]
[relax] Split Row into separate subclass for RowWithStorage and RowWithGetters.
[relax] Introduce FieldValueGetterFactory and FieldValueSetterFactory.
[relax] Add support for nested, array, and map types.
[relax] Refactor getter-based SchemaProvider into a common base class.
[relax] Refactor getter/setter conversion utilities into a library. This cleans
[relax] Refactor Schema inference from POJOs so much of the code can be reused
[relax] Add encoding byte BYTES type.
[relax] Make FieldValueGetter and FieldValueSetter generic types again.
[relax] Start adding JavaBean getters and setters.
[relax] Make sure that field order is stable, and make tests robust to
[relax] Simplify convertArray.
[amaliujia] support decimal type in reading from PubSub JSON
[ankurgoenka] Fix artifact retrival file system
[jiangkai] [BEAM-4723] Support date type plus interval
[ankurgoenka] Create SDKComponents with pipeline options
[echauchot] [BEAM-4707] Add timestamp field to Nexmark tables and use explicit mode
[lukasz.gajowy] [BEAM-4137] Divide separate PipelineOptions for elasticsearch tests
[lukasz.gajowy] [BEAM-4137] Remove unused options from IOTestPipelineOptions
[katarzyna.kucharczyk] Added integration test configuration to beam-examples gradle. Fixed
[markliu] Move gradle-spotless plugin to buildscript
[rober] Add a simple benchmark to the ParDo execution node.
[robertwb] Use main output for cold keys.
[pablo] Unpin GCP test requirement
[lcwik] [BEAM-4654] Translate timers as PCollections within the Java SDK.
[robertwb] Use local GCP credentials, if any, in local docker environment.
[cclauss] [BEAM-1251] Modernize Python 2 code to get ready for Python 3
[robertwb] Split bounded sources in larger than 1k chunks.
[cclauss] [BEAM-1251] Upgrade from buffer to memoryview (again)
[thw] Use the beam:option:value:v1 as the portable pipeline options
[thw] [BEAM-4733] Pass pipeline options from Python portable runner to job
[matthias] removed unnecessary install_aliases, import io and unicode imports
[thw] Fix Flink portable streaming translation executable stage output
[echauchot] [BEAM-4707] insert timestamp to bigQuery as seconds as expected by
[echauchot] [BEAM-4671] Skip MetricsPusherTest validates runner test for runners
[matthias] Futurize io subpackage
[matthias] incorporated all feedback for futurize io subpackage
[robertwb] object() takes no parameters
[amaliujia] remove decimal support in BigQuery SQL read.
[iemejia] [BEAM-4016] Invoke Setup and TearDown on SplitRestrictionFn and
[ankurgoenka] Initialization BeamFileSystem in FlinkExecutableStageFunction
[ankurgoenka] [BEAM-4644] Use SLF4J in ExecutableStageDoFnOperator
[ccy] Revert #5878 to unbreak Python postcommit
[ankurgoenka] Pull the docker image before starting docker container.
[iemejia] Rename COMBINE_TRANSFORM_URN to the more proper
[ccy] [BEAM-4003] Fix missing iteritems import
[relax] Address code-review comments.
[relax] Address code-review comments.
[github] Avoid silent error when calling unimplemented function.
[github] Avoid silent error when calling unimplemented function.
[relax] Add better validation to isSetter isGetter
[ekirpichov] Infer boundedness for SDF application at application time
[ekirpichov] [BEAM-4745] Revert "[BEAM-4016] Invoke Setup and TearDown on
[markliu] Remove :buildSrc from settings.gradle since it's not a subproject
[pablo] Logging relies on StateSampler for context
[ccy] [BEAM-4594] Beam Python state and timers user-facing API
[ccy] [BEAM-4593] Remove refcounts from the Python SDK
[ankurgoenka] FileSystems should throw exception if filesystem for schema is not found
[ankurgoenka] Make schema pattern stricter
[ccy] [BEAM-1251] Revert #5887 to unbreak Python PostCommit
[pablo] Improving signing of all published artifacts and adding test publication
[ccy] [BEAM-1251] Upgrade snappy and use a memoryview
[pablo] Removing extra publication
[cclauss] [BEAM-3761] Define cmp() in Python 3
[cclauss] [BEAM-3959] Add Python 3 undefined names to flake8
[amyrvold] [BEAM-3457] Upgrade version of gogradle and add examples and containers
[apilloud] [SQL] Inject JDBC rules through Hook.
[lcwik] [BEAM-4481, BEAM-4484] Start vendoring portability dependencies to not
[apilloud] [SQL] Plumb pipelineOptions through IOSinkRel
[lcwik] [BEAM-4653] Add support to the Java SDK harness to execute timers.
[apilloud] [SQL] Default timezone is UTC
[lcwik] [BEAM-2915] Add support for handling bag user state to the
[amaliujia] add testArithmeticOperator
[altay] Wait on connection channel to be ready
[ekirpichov] Bumps Dataflow worker container version
[ekirpichov] [BEAM-4745, BEAM-4016] Reintroduces Setup and TearDown on
[amaliujia] simplify unbounded limit
[iemejia] [BEAM-4020] Add an HBaseIO read implementation based on SDF
[iemejia] [BEAM-4020] Add HBaseIO.readAll() based on SDF
[lukasz.gajowy] [BEAM-4691] Rename common_job_properties
[aromanenko.dev] [BEAM-4622] Makes required to call Beam SQL expressions validation
[aromanenko.dev] Check number of arguments at first
[lcwik] [BEAM-4764] Fix missing reference to project to be able to refer to the
[lcwik] [BEAM-4764] Fix java precommit to ensure that any checks in model/* are
[apilloud] [SQL] Apply JDBC rules globally
[rober] Add a small benchmark for metrics
[ankurgoenka] Warn on docker image pull fail
[robinyqiu] Fix bug in java mobile gaming example
[lcwik] [BEAM-4744] Enable parallel publishing/release
[qinyeli] Interactive runner and corresponding tests
[github] [BEAM-4648] Remove experimental Python RPC DirectRunner (#5777)
[s.lebedev] [BEAM-1251] Replace NameError-driven dispatch with ``past``
[altay] Install fastavro only in linux
[pablo] [BEAM-4718]Run gradle build before publish
[pablo] Add CODEOWNERS file (#5909)
[pablo] Full tracking of Batch side inputs. Performance tests show no regression
[kirpichov] Adds license header to CODEOWNERS
[kirpichov] Adds a naive implementation of bounded SDFs
[echauchot] Move echauchot from runner/core to runner/core/metrics in CODEOWNERS
[github] [BEAM-4752] Add support for newer dill dependency (#5931)
[elliottb] Fix the expected encoding of BigQuery's NUMERIC type when reading from
[ankurgoenka] Adding tet cases for matchResouces
[kirpichov] Makes SplittableDoFnTest exercise both bounded and unbounded SDFs.
[github] [BEAM-4752] Add dill compatibility for older versions of dill (#5949)
[ankurgoenka] Ignore generated vendor files in python container build
[kirpichov] Supports bounded SDF in all runners.
[relax] Add new Convert transform for schemas.
[relax] Fix two bugs: * proper handling of nullable fields. * Row stores
[relax] Add tests.
[relax] Fix broken test.
[relax] Add @Nullable annotation.
[github] [BEAM-4432] Adding Sources to produce Synthetic output for Batch
[ryan.blake.williams] send PutArtifactResponse in BeamFileSystemArtifactStagingService
[apilloud] [BEAM-4774] Add Nexmark SQL to postcommits
[aaltay] [BEAM-2810] fastavro integration test (#5862)
[kirpichov] Improve error messages on Spark and Flink SDF translation
[qinyeli] Interactive Beam -- yields Read Trans after apply()
[apilloud] [SQL] Actually run JDBC Jar Test
[apilloud] [SQL] Wrap PipelineOptions with correct class loader
[kirpichov] Converts BoundedReadFromUnboundedSource to a DoFn
[boyuanz] Automate release branch cut process
[yifanzou] [BEAM-4801] add dependency owners file
[lcwik] [BEAM-4769] Fix issue when performing coder inference for wildcard
[lcwik] [BEAM-4744] Fix runners/google-cloud-dataflow-java/examples-streaming
[ankurgoenka] Fixing Test case for beam Artifact Staging Service
[boyuanz] Fix javadoc broken
[daniel.o.programmer] [BEAM-3711] Enabling combiner lifting in Dataflow Runner.
[ankurgoenka] Fix for Flaky test BeamFileSystemArtifactServicesTest
[lcwik] [BEAM-4418] Fix duplicate source set declaration conflict.
[pablo] Add coders microbenchmark. (#5565)
[melissapa] [BEAM-3769] Fix broken JIRA links in README
[pablo] Moving to 2.7.0-SNAPSHOT on master branch.
[thw] [BEAM-4791] Portable Flink runner integration test.
[thw] Use shadowTest configuration since we don't plan to shade away the SDK
[thw] [BEAM-4791] Move executor to Before/After and use
[github] Increase the complete time out
[github] Minor javadoc fix up.
[amaliujia] BEAM-4820: Add timeout to Future.get when waiting for Pubsub messages
[github] Adding @angoenka to CODEOWNERS
[boyuanz] Fix javadoc build broken in synthetic for release branch
[garrettjonesgoogle] Bumping google cloud versions
[thw] Add JIRA reference.
[lcwik] Address PR comments.
[pablo] Fix bugs in 'cut release branch step'
[lcwik] Address PR comments.
[amaliujia] [BEAM-4806] improve unit tests for SqlLineMain
[qinyeli] Added a round trip through proto in Interactive runner.
[github] Add version bound to numpy.
[echauchot] CodeOwners: Narrow down extensions-java to metrics and assign echauchot
[jxlewis] Added ClientConfiguration to AwsOptions and S3FileSystem
[jxlewis] Updated documentation, added BEAM-4828 to In-Progress
[github] Fix indentation.
[github] SQS -> Amazon SQS
[boyuanz] Added '--continue' switches into nightly build
[supercclank] Set BatchPath on google.api.services.storage.Storage Storage Builder
[kirpichov] Converts SolrIO away from BoundedSource
[supercclank] Change to using java.nio.file.Paths over string concatenation for batch
[apilloud] [BEAM-4761] Run Nexmark on Dataflow
[github] [BEAM-4832] Fix concurrent race in datamgr.go
[github] Update datamgr.go
[carlos.alonso] Adds the BigQueryInsertError model class
[carlos.alonso] Adds BigQueryInsertErrorCoder and tests
[carlos.alonso] Adds extended errors info to WriteResult
[carlos.alonso] Makes streaming methods generic but limited to the implemented error
[carlos.alonso] Removes ErrorContainers auxiliary class. Adds BigQueryServicesImpl test
[carlos.alonso] Makes BigQueryIO.Write to be configurable to decide which kind of errors
[carlos.alonso] Removes unneeded coder registration
[carlos.alonso] Simplifies BigQueryInsertError codification
[carlos.alonso] A few style improvements
[katarzyna.kucharczyk] Changed order of installing perfkit requirements and beam requirements
[thw] Remove dead link from README.
[amyrvold] [BEAM-4831] Ignore failures during :beam-sdks-go:vet to allow ./gradlew
[aaltay] Add automation script for verifying release build (#5973)
[ekirpichov] Avoids use of discouraged Duration(long) ctor
[qinyeli] Interactive Runner -- keep_alive
[thw] [BEAM-4842] Update Flink Runner to Flink 1.5.1
[daniel.o.programmer] [BEAM-3709] Implementing new combine URNs in python.
[aaltay] Add bash script to automate "Preparation for GPG" (#6015)
[github] Fixing log message
[lukasz.gajowy] [BEAM-4845] Make BigQueryServices and FakeDatasetService public to fix
[amyrvold] Fix [BEAM-4847] by manually setting max workers and jvm memory and add
[relax] Enable schemas for Apex runner.
[relax] Enable Schemas on samza runner.
[relax] Enable schemas for Flink runner.
[relax] Enable schemas for Spark.
[relax] Fix bugs.
[relax] Fix Apex breakage.
[ehudm] Remove CODEOWNERs.
[kedin] [SQL] Enable running BeamSqlLine from gradle
[thw] [BEAM-4847] Reduce Gradle JVM Xmx to 4g to fix Jenkins build failures.
[aaltay] Automate 'Start a snapshot build' step in beam release guide (#6042)
[lukasz.gajowy] [BEAM-4838] Add dockerfile for standalone Jenkins. Plugins included.
[dariusz.aniszewski] separate virtualenvs for Beam and PerfKit
[lcwik] [BEAM-4866] Fix missing licenses.
[pablo] Removing scoped metrics container
[pablo] Remove old style metrics context management
[garrettjonesgoogle] Bumping versions that were missed in #5988
[altay] Remove reference to dataflow-distribution.properties
[lcwik] [BEAM-4176] Initial implementation for running portable runner tests
[pablo] Fix Java Nightly Snapshot Failures
[lcwik] [BEAM-4629] Output the names of the failing licenses as part of the
[aaltay] [BEAM-4859] Enable Python VR tests in streaming in postcommit task
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision aa2b585a5356ce3b607dd3d17a3c6ab2ce65353b (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f aa2b585a5356ce3b607dd3d17a3c6ab2ce65353b
Commit message: "Merge pull request #6056: [BEAM-4836] separate virtualenvs for Beam and PerfKit"
> git rev-list --no-walk 0892e23940a584a170e2668d86b7bdc50c03e024 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins266031542697616565.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2118418907451248094.sh
+ rm -rf .beam_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1013385759992617214.sh
+ rm -rf .perfkit_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1440131868588494871.sh
+ virtualenv .beam_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3203404953492079456.sh
+ virtualenv .perfkit_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3388230821306939726.sh
+ .beam_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.beam_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.beam_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8681682376553589136.sh
+ .perfkit_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.perfkit_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.perfkit_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1014785923945432727.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7488451273964902903.sh
+ .beam_env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.8.2)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.7)
Requirement already satisfied: dill<=0.2.8.2,>=0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.2.8.2)
Requirement already satisfied: grpcio<2,>=1.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.13.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.1.0)
Requirement already satisfied: httplib2<=0.11.3,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.11.3)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.0.0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (4.1.2)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.0)
Requirement already satisfied: pydot<1.3,>=1.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.2.4)
Requirement already satisfied: pytz<=2018.4,>=2018.3 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2018.4)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (3.12)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.6.8)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.11.0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.4)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.2.0)
Requirement already satisfied: future<1.0.0,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.16.0)
Requirement already satisfied: fastavro==0.19.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.19.7)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.5.20)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.90.4)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (7.0.1)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.26.0)
Requirement already satisfied: proto-google-cloud-pubsub-v1==0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: nose>=1.3.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.3.7)
Collecting numpy<2,>=1.14.3 (from apache-beam==2.7.0.dev0)
Using cached https://files.pythonhosted.org/packages/85/51/ba4564ded90e093dbb6adfc3e21f99ae953d9ad56477e1b0d4a93bacf7d3/numpy-1.15.0-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.9.0)
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.7.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.8.0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.0.2)
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.4.2)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.2.2)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (3.4.2)
Requirement already satisfied: setuptools in ./.beam_env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.7.0.dev0) (40.0.0)
Requirement already satisfied: pyparsing>=2.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from pydot<1.3,>=1.2.0->apache-beam==2.7.0.dev0) (2.2.0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (0.14.1)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.7.0.dev0) (1.5.3)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (1.22)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (1.5)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.16)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.11.4)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.0.3)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (1.5.0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (3.8)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (2.1.0)
Installing collected packages: numpy, apache-beam
Found existing installation: numpy 1.13.3
Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'numpy'. No files were found to uninstall.
Found existing installation: apache-beam 2.7.0.dev0
Not uninstalling apache-beam at /home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_Compressed_TextIOIT_HDFS/src/sdks/python, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'apache-beam'. No files were found to uninstall.
Running setup.py develop for apache-beam
Successfully installed apache-beam numpy-1.15.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7216896748234028238.sh
+ .beam_env/bin/python src/sdks/python/setup.py sdist --dist-dir=src/sdks/python/target
Traceback (most recent call last):
File "src/sdks/python/setup.py", line 46, in <module>
PACKAGE_VERSION = get_version()
File "src/sdks/python/setup.py", line 41, in get_version
exec(open(os.path.normpath('./apache_beam/version.py')).read(), global_names) # pylint: disable=exec-used
IOError: [Errno 2] No such file or directory: 'apache_beam/version.py'
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PerformanceTests_Python #1254
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1254/display/redirect>
------------------------------------------
[...truncated 1.72 KB...]
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2255626873956628632.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins9165999473588564426.sh
+ rm -rf .beam_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins9063788179791268742.sh
+ rm -rf .perfkit_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5884993017406052429.sh
+ virtualenv .beam_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1342281336343959987.sh
+ virtualenv .perfkit_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3118154934927439155.sh
+ .beam_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.beam_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.beam_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6504331171475556563.sh
+ .perfkit_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.perfkit_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.perfkit_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4209305034972557352.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5351899519785124003.sh
+ .beam_env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.8.2)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.7)
Requirement already satisfied: dill<=0.2.8.2,>=0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.2.8.2)
Requirement already satisfied: grpcio<2,>=1.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.13.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.1.0)
Requirement already satisfied: httplib2<=0.11.3,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.11.3)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.0.0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (4.1.2)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.0)
Requirement already satisfied: pydot<1.3,>=1.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.2.4)
Requirement already satisfied: pytz<=2018.4,>=2018.3 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2018.4)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (3.12)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.6.8)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.11.0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.4)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.2.0)
Requirement already satisfied: future<1.0.0,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.16.0)
Requirement already satisfied: fastavro==0.19.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.19.7)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.5.20)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.90.4)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (7.0.1)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.26.0)
Requirement already satisfied: proto-google-cloud-pubsub-v1==0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: nose>=1.3.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.3.7)
Collecting numpy<2,>=1.14.3 (from apache-beam==2.7.0.dev0)
Using cached https://files.pythonhosted.org/packages/85/51/ba4564ded90e093dbb6adfc3e21f99ae953d9ad56477e1b0d4a93bacf7d3/numpy-1.15.0-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.9.0)
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.7.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.8.0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.0.2)
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.4.2)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.2.2)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (3.4.2)
Requirement already satisfied: setuptools in ./.beam_env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.7.0.dev0) (40.0.0)
Requirement already satisfied: pyparsing>=2.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from pydot<1.3,>=1.2.0->apache-beam==2.7.0.dev0) (2.2.0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (0.14.1)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.7.0.dev0) (1.5.3)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (1.22)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (1.5)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.16)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.11.4)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.0.3)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (1.5.0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (3.8)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (2.1.0)
Installing collected packages: numpy, apache-beam
Found existing installation: numpy 1.13.3
Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'numpy'. No files were found to uninstall.
Found existing installation: apache-beam 2.7.0.dev0
Not uninstalling apache-beam at /home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_Compressed_TextIOIT_HDFS/src/sdks/python, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'apache-beam'. No files were found to uninstall.
Running setup.py develop for apache-beam
Successfully installed apache-beam numpy-1.15.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1563680810598338892.sh
+ .beam_env/bin/python src/sdks/python/setup.py sdist --dist-dir=src/sdks/python/target
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/local/lib/python2.7/site-packages/setuptools/dist.py>:398: UserWarning: Normalizing '2.7.0.dev' to '2.7.0.dev0'
normalized_version,
running sdist
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/gen_protos.py>:49: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
running egg_info
creating apache_beam.egg-info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
warning: cmd: standard file not found: should have one of README, README.rst, README.txt, README.md
running check
creating apache-beam-2.7.0.dev0
creating apache-beam-2.7.0.dev0/apache_beam.egg-info
copying files to apache-beam-2.7.0.dev0...
copying apache_beam.egg-info/PKG-INFO -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/SOURCES.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/dependency_links.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/entry_points.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/not-zip-safe -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/requires.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/top_level.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
Writing apache-beam-2.7.0.dev0/setup.cfg
creating src/sdks/python/target
Creating tar archive
removing 'apache-beam-2.7.0.dev0' (and everything under it)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3779598970039815852.sh
+ .perfkit_env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14)) (0.2.2)
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15)) (2.10)
Requirement already satisfied: setuptools in ./.perfkit_env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16)) (40.0.0)
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17)) (2.6.0)
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18)) (1.4)
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19)) (3.2.0)
Requirement already satisfied: PyYAML==3.12 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 20)) (3.12)
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21)) (0.8.1)
Requirement already satisfied: numpy==1.13.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22)) (1.13.3)
Requirement already satisfied: functools32 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 23)) (3.2.3.post2)
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24)) (0.5.5)
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25)) (0.3.0)
Requirement already satisfied: timeout-decorator in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 26)) (0.4.0)
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Requirement already satisfied: colorama; extra == "windows" in /home/jenkins/.local/lib/python2.7/site-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17)) (0.3.9)
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.0)
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.11.0)
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.2.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7173993871705149011.sh
+ .perfkit_env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --bigquery_table=beam_performance.pkb_results --k8s_get_retry_count=36 --k8s_get_wait_interval=10 --python_binary=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --beam_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-07-26 10:46:45,580 b302cf0c MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/b302cf0c/pkb.log>
2018-07-26 10:46:45,580 b302cf0c MainThread INFO PerfKitBenchmarker version: v1.12.0-729-ga00bd87
2018-07-26 10:46:45,581 b302cf0c MainThread INFO Flag values:
--beam_sdk=python
--k8s_get_wait_interval=10
--python_binary=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python>
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--beam_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src>
--k8s_get_retry_count=36
--benchmarks=beam_integration_benchmark
2018-07-26 10:46:45,875 b302cf0c MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-07-26 10:46:46,063 b302cf0c MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-07-26 10:46:46,067 b302cf0c MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-07-26 10:46:46,068 b302cf0c MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/gradlew> clean assemble --stacktrace --info -DintegrationTestRunner=dataflow
2018-07-26 10:52:45,185 b302cf0c MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-07-26 10:52:45,278 b302cf0c MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/target/apache-beam-2.7.0.dev0.tar.gz>
2018-07-26 11:02:45,286 b302cf0c Thread-3 ERROR IssueCommand timed out after 600 seconds. Killing command "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/target/apache-beam-2.7.0.dev0.tar.gz".>
2018-07-26 11:02:45,296 b302cf0c MainThread beam_integration_benchmark(1/1) INFO Ran: {<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/target/apache-beam-2.7.0.dev0.tar.gz}> ReturnCode:-9
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/local/lib/python2.7/site-packages/setuptools/dist.py>:398: UserWarning: Normalizing '2.7.0.dev' to '2.7.0.dev0'
normalized_version,
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/gen_protos.py>:49: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py>:54: DeprecationWarning: options is deprecated since First stable release.. References to <pipeline>.options will not be supported
print('Found: %s.' % self.build_console_url(pipeline.options))
2018-07-26 11:02:45,297 b302cf0c MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 668, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 548, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-07-26 11:02:45,298 b302cf0c MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-07-26 11:02:45,301 b302cf0c MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 804, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 668, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 548, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-07-26 11:02:45,301 b302cf0c MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-07-26 11:02:45,317 b302cf0c MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-07-26 11:02:45,318 b302cf0c MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/b302cf0c/pkb.log>
2018-07-26 11:02:45,318 b302cf0c MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/b302cf0c/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PerformanceTests_Python #1253
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1253/display/redirect>
------------------------------------------
GitHub pull request #6056 of commit d951271cb4412facc746bf3aab4b9dcf86259472, no merge conflicts.
Setting status of d951271cb4412facc746bf3aab4b9dcf86259472 to PENDING with url https://builds.apache.org/job/beam_PerformanceTests_Python/1253/ and message: 'Build started for merge commit.'
Using context: Python SDK Performance Test
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
Cloning the remote Git repository
Cloning repository https://github.com/apache/beam.git
> git init <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src> # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/*
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
> git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/6056/*:refs/remotes/origin/pr/6056/*
> git rev-parse refs/remotes/origin/pr/6056/merge^{commit} # timeout=10
> git rev-parse refs/remotes/origin/origin/pr/6056/merge^{commit} # timeout=10
Checking out Revision f5b091a50c6bfa3e7f10e361eab9d4d107763ddc (refs/remotes/origin/pr/6056/merge)
> git config core.sparsecheckout # timeout=10
> git checkout -f f5b091a50c6bfa3e7f10e361eab9d4d107763ddc
Commit message: "Merge d951271cb4412facc746bf3aab4b9dcf86259472 into f2a1fd4307929aa938067058205a5ff79dc0207f"
First time build. Skipping changelog.
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4613836979666378994.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7257108433714162759.sh
+ rm -rf .beam_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6164015999892067355.sh
+ rm -rf .perfkit_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8665091684199027555.sh
+ virtualenv .beam_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5604433107216637169.sh
+ virtualenv .perfkit_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6086223940262032725.sh
+ .beam_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.beam_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.beam_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8326416354300863133.sh
+ .perfkit_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.perfkit_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.perfkit_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6909867195755979876.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1403573749653444305.sh
+ .beam_env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.8.2)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.7)
Requirement already satisfied: dill<=0.2.8.2,>=0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.2.8.2)
Requirement already satisfied: grpcio<2,>=1.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.13.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.1.0)
Requirement already satisfied: httplib2<=0.11.3,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.11.3)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.0.0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (4.1.2)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.0)
Requirement already satisfied: pydot<1.3,>=1.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.2.4)
Requirement already satisfied: pytz<=2018.4,>=2018.3 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2018.4)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (3.12)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.6.8)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.11.0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.4)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.2.0)
Requirement already satisfied: future<1.0.0,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.16.0)
Requirement already satisfied: fastavro==0.19.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.19.7)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.5.20)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.90.4)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (7.0.1)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.26.0)
Requirement already satisfied: proto-google-cloud-pubsub-v1==0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: nose>=1.3.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.3.7)
Collecting numpy<2,>=1.14.3 (from apache-beam==2.7.0.dev0)
Using cached https://files.pythonhosted.org/packages/85/51/ba4564ded90e093dbb6adfc3e21f99ae953d9ad56477e1b0d4a93bacf7d3/numpy-1.15.0-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.9.0)
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.7.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.8.0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.0.2)
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.4.2)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.2.2)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (3.4.2)
Requirement already satisfied: setuptools in ./.beam_env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.7.0.dev0) (40.0.0)
Requirement already satisfied: pyparsing>=2.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from pydot<1.3,>=1.2.0->apache-beam==2.7.0.dev0) (2.2.0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (0.14.1)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.7.0.dev0) (1.5.3)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (1.22)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (1.5)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.16)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.11.4)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.0.3)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (1.5.0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (3.8)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (2.1.0)
Installing collected packages: numpy, apache-beam
Found existing installation: numpy 1.13.3
Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'numpy'. No files were found to uninstall.
Found existing installation: apache-beam 2.7.0.dev0
Not uninstalling apache-beam at /home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_Compressed_TextIOIT_HDFS/src/sdks/python, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'apache-beam'. No files were found to uninstall.
Running setup.py develop for apache-beam
Successfully installed apache-beam numpy-1.15.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8809190521784436377.sh
+ .perfkit_env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14)) (0.2.2)
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15)) (2.10)
Requirement already satisfied: setuptools in ./.perfkit_env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16)) (40.0.0)
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17)) (2.6.0)
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18)) (1.4)
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19)) (3.2.0)
Requirement already satisfied: PyYAML==3.12 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 20)) (3.12)
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21)) (0.8.1)
Requirement already satisfied: numpy==1.13.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22)) (1.13.3)
Requirement already satisfied: functools32 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 23)) (3.2.3.post2)
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24)) (0.5.5)
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25)) (0.3.0)
Requirement already satisfied: timeout-decorator in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 26)) (0.4.0)
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Requirement already satisfied: colorama; extra == "windows" in /home/jenkins/.local/lib/python2.7/site-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17)) (0.3.9)
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.0)
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.11.0)
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.2.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1458877980335576090.sh
+ .perfkit_env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --bigquery_table=beam_performance.pkb_results --k8s_get_retry_count=36 --k8s_get_wait_interval=10 --python_binary=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --beam_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-07-25 15:38:58,755 2f8cc01e MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/2f8cc01e/pkb.log>
2018-07-25 15:38:58,756 2f8cc01e MainThread INFO PerfKitBenchmarker version: v1.12.0-729-ga00bd87
2018-07-25 15:38:58,757 2f8cc01e MainThread INFO Flag values:
--beam_sdk=python
--k8s_get_wait_interval=10
--python_binary=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python>
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--beam_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src>
--k8s_get_retry_count=36
--benchmarks=beam_integration_benchmark
2018-07-25 15:38:59,028 2f8cc01e MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-07-25 15:38:59,215 2f8cc01e MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-07-25 15:38:59,223 2f8cc01e MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-07-25 15:38:59,224 2f8cc01e MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/gradlew> clean assemble --stacktrace --info -DintegrationTestRunner=dataflow
2018-07-25 15:45:10,639 2f8cc01e MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-07-25 15:45:10,735 2f8cc01e MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 668, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 548, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 198, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 274, in _BuildPythonCommand
raise RuntimeError('No python binary is found')
RuntimeError: No python binary is found
2018-07-25 15:45:10,736 2f8cc01e MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-07-25 15:45:10,739 2f8cc01e MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 804, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 668, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 548, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 198, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 274, in _BuildPythonCommand
raise RuntimeError('No python binary is found')
RuntimeError: No python binary is found
2018-07-25 15:45:10,739 2f8cc01e MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-07-25 15:45:10,794 2f8cc01e MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-07-25 15:45:10,795 2f8cc01e MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/2f8cc01e/pkb.log>
2018-07-25 15:45:10,795 2f8cc01e MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/2f8cc01e/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PerformanceTests_Python #1252
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1252/display/redirect>
------------------------------------------
[...truncated 4.68 KB...]
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins147823451126947701.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/aa/063eca6a416f397bd99552c534c6d11d57f58f2e94c14780f3bbf818c4cf/monotonic-1.5-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.5 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3533660477905184729.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-10 12:00:52,567 422a7ff4 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/422a7ff4/pkb.log>
2018-05-10 12:00:52,568 422a7ff4 MainThread INFO PerfKitBenchmarker version: v1.12.0-584-g9260fab
2018-05-10 12:00:52,569 422a7ff4 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-10 12:00:52,941 422a7ff4 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-10 12:00:53,095 422a7ff4 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-10 12:00:53,123 422a7ff4 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-10 12:00:53,124 422a7ff4 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-10 12:00:58,281 422a7ff4 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/422a7ff4/beam/gradlew> clean install -xcheck --stacktrace -DintegrationTestRunner=dataflow
2018-05-10 12:05:21,135 422a7ff4 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-10 12:05:21,146 422a7ff4 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/422a7ff4/beam/sdks/python/target>
2018-05-10 12:05:21,148 422a7ff4 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-10 12:05:21,149 422a7ff4 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 781, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/422a7ff4/beam/sdks/python/target>
2018-05-10 12:05:21,150 422a7ff4 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-10 12:05:21,195 422a7ff4 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-10 12:05:21,196 422a7ff4 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/422a7ff4/pkb.log>
2018-05-10 12:05:21,197 422a7ff4 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/422a7ff4/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user jasonkuster@google.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user schroederc@google.com
Not sending mail to unregistered user shoyer@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1251
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1251/display/redirect?page=changes>
Changes:
[apilloud] [SQL] Use Lex.JAVA and drop toLowerCase in DDL
[apilloud] [SQL] Planner is private within BeamQueryPlanner
[github] Porting Python PostCommit to Gradle. (#5146)
------------------------------------------
[...truncated 4.68 KB...]
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6452696876120264024.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/aa/063eca6a416f397bd99552c534c6d11d57f58f2e94c14780f3bbf818c4cf/monotonic-1.5-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.5 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4374926398290278148.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-10 06:00:42,785 bb421231 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/bb421231/pkb.log>
2018-05-10 06:00:42,786 bb421231 MainThread INFO PerfKitBenchmarker version: v1.12.0-584-g9260fab
2018-05-10 06:00:42,787 bb421231 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-10 06:00:43,099 bb421231 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-10 06:00:43,264 bb421231 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-10 06:00:43,268 bb421231 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-10 06:00:43,269 bb421231 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-10 06:00:48,431 bb421231 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/bb421231/beam/gradlew> clean install -xcheck --stacktrace -DintegrationTestRunner=dataflow
2018-05-10 06:04:55,789 bb421231 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-10 06:04:55,799 bb421231 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/bb421231/beam/sdks/python/target>
2018-05-10 06:04:55,801 bb421231 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-10 06:04:55,803 bb421231 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 781, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/bb421231/beam/sdks/python/target>
2018-05-10 06:04:55,803 bb421231 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-10 06:04:55,841 bb421231 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-10 06:04:55,842 bb421231 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/bb421231/pkb.log>
2018-05-10 06:04:55,842 bb421231 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/bb421231/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user jasonkuster@google.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user schroederc@google.com
Not sending mail to unregistered user shoyer@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1250
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1250/display/redirect>
------------------------------------------
[...truncated 5.08 KB...]
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1792401711501177360.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/aa/063eca6a416f397bd99552c534c6d11d57f58f2e94c14780f3bbf818c4cf/monotonic-1.5-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.5 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2757522060705429708.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-10 01:04:58,668 155f91e5 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/155f91e5/pkb.log>
2018-05-10 01:04:58,668 155f91e5 MainThread INFO PerfKitBenchmarker version: v1.12.0-584-g9260fab
2018-05-10 01:04:58,669 155f91e5 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-10 01:04:58,965 155f91e5 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-10 01:04:59,114 155f91e5 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-10 01:04:59,118 155f91e5 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-10 01:04:59,118 155f91e5 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-10 01:05:03,970 155f91e5 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/155f91e5/beam/gradlew> clean install -xcheck --stacktrace -DintegrationTestRunner=dataflow
2018-05-10 01:08:57,843 155f91e5 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-10 01:08:57,858 155f91e5 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/155f91e5/beam/sdks/python/target>
2018-05-10 01:08:57,859 155f91e5 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-10 01:08:57,861 155f91e5 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 781, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/155f91e5/beam/sdks/python/target>
2018-05-10 01:08:57,862 155f91e5 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-10 01:08:57,897 155f91e5 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-10 01:08:57,898 155f91e5 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/155f91e5/pkb.log>
2018-05-10 01:08:57,898 155f91e5 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/155f91e5/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user jasonkuster@google.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user schroederc@google.com
Not sending mail to unregistered user shoyer@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1249
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1249/display/redirect?page=changes>
Changes:
[apilloud] [BEAM-4251] Fix broken javadoc ref
[apilloud] [SQL] Simplify planner usage, fix assertions
[apilloud] [SQL] Cleanup BeamAggregation rule and rel to fix
[apilloud] [SQL] Fix assertion failures in test
[kirpichov] [BEAM-4242] Fixes O(n) complexity of Wait.on() due to combiner lifting
[iemejia] Fix warning on repeated core-construction-java dependency on
[aaltay] Nexmark pubsub benchmarks. (#5247)
------------------------------------------
[...truncated 4.66 KB...]
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2730193946477290219.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/aa/063eca6a416f397bd99552c534c6d11d57f58f2e94c14780f3bbf818c4cf/monotonic-1.5-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.5 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5647444987526960164.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-10 00:00:38,955 c2fa1eca MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/c2fa1eca/pkb.log>
2018-05-10 00:00:38,955 c2fa1eca MainThread INFO PerfKitBenchmarker version: v1.12.0-584-g9260fab
2018-05-10 00:00:38,956 c2fa1eca MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-10 00:00:39,389 c2fa1eca MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-10 00:00:39,557 c2fa1eca MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-10 00:00:39,561 c2fa1eca MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-10 00:00:39,562 c2fa1eca MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-10 00:00:44,692 c2fa1eca MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/c2fa1eca/beam/gradlew> clean install -xcheck --stacktrace -DintegrationTestRunner=dataflow
2018-05-10 00:05:18,331 c2fa1eca MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-10 00:05:18,342 c2fa1eca MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/c2fa1eca/beam/sdks/python/target>
2018-05-10 00:05:18,344 c2fa1eca MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-10 00:05:18,346 c2fa1eca MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 781, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/c2fa1eca/beam/sdks/python/target>
2018-05-10 00:05:18,346 c2fa1eca MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-10 00:05:18,395 c2fa1eca MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-10 00:05:18,396 c2fa1eca MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/c2fa1eca/pkb.log>
2018-05-10 00:05:18,397 c2fa1eca MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/c2fa1eca/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user jasonkuster@google.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user schroederc@google.com
Not sending mail to unregistered user shoyer@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1248
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1248/display/redirect?page=changes>
Changes:
[kedin] [SQL] Change DDL parser to create Beam Schema types instead of Calcite
[kedin] [SQL] Support arrays of primitive types in DDL
[kedin] [SQL] Parse Row fields
[kedin] [SQL] Remove Column. Replace with Schema
[kedin] [SQL] Validate Table schemas in BeamSqlCliTest
[kedin] [SQL] Add support for angular brackets array definition
[kedin] [SQL] Add support for Maps in DDL
[kedin] [SQL] Add support for angular brackets Row syntax
[kedin] [SQL] Fix rebase conflicts
[yifanzou] update machine numbers to 1..16 for inventory tests
[kedin] [SQL] Add QuickCheck and tests for DDL schema verification
[kedin] [SQL] Remove support for postfix array declaration in DDL
[Pablo] StateSampler knows the execution thread it tracks.
[github] Revert "Enable githubCommitNotifier for post commits"
------------------------------------------
[...truncated 4.69 KB...]
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins122492978219254013.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/aa/063eca6a416f397bd99552c534c6d11d57f58f2e94c14780f3bbf818c4cf/monotonic-1.5-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.5 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8769515154695154695.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-09 19:09:58,835 6da85548 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/6da85548/pkb.log>
2018-05-09 19:09:58,836 6da85548 MainThread INFO PerfKitBenchmarker version: v1.12.0-580-g36cfc82
2018-05-09 19:09:58,837 6da85548 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-09 19:09:59,177 6da85548 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-09 19:09:59,352 6da85548 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-09 19:09:59,356 6da85548 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-09 19:09:59,357 6da85548 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-09 19:10:04,310 6da85548 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/6da85548/beam/gradlew> clean install -xcheck --stacktrace -DintegrationTestRunner=dataflow
2018-05-09 19:14:02,936 6da85548 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-09 19:14:02,946 6da85548 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/6da85548/beam/sdks/python/target>
2018-05-09 19:14:02,948 6da85548 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-09 19:14:02,950 6da85548 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 781, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/6da85548/beam/sdks/python/target>
2018-05-09 19:14:02,950 6da85548 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-09 19:14:02,960 6da85548 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-09 19:14:02,961 6da85548 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/6da85548/pkb.log>
2018-05-09 19:14:02,961 6da85548 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/6da85548/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user jasonkuster@google.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user schroederc@google.com
Not sending mail to unregistered user shoyer@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1247
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1247/display/redirect>
------------------------------------------
[...truncated 9.21 KB...]
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/aa/063eca6a416f397bd99552c534c6d11d57f58f2e94c14780f3bbf818c4cf/monotonic-1.5-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.5 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5460359802169586681.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-09 12:00:37,807 99050ccc MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/99050ccc/pkb.log>
2018-05-09 12:00:37,808 99050ccc MainThread INFO PerfKitBenchmarker version: v1.12.0-580-g36cfc82
2018-05-09 12:00:37,809 99050ccc MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-09 12:00:38,178 99050ccc MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-09 12:00:38,345 99050ccc MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-09 12:00:38,349 99050ccc MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-09 12:00:38,349 99050ccc MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-09 12:00:43,641 99050ccc MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/99050ccc/beam/gradlew> clean install -xcheck --stacktrace -DintegrationTestRunner=dataflow
2018-05-09 12:04:59,730 99050ccc MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-09 12:04:59,744 99050ccc MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/99050ccc/beam/sdks/python/target>
2018-05-09 12:04:59,746 99050ccc MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-09 12:04:59,749 99050ccc MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 781, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/99050ccc/beam/sdks/python/target>
2018-05-09 12:04:59,749 99050ccc MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-09 12:04:59,765 99050ccc MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-09 12:04:59,766 99050ccc MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/99050ccc/pkb.log>
2018-05-09 12:04:59,766 99050ccc MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/99050ccc/completion_statuses.json>
Build step 'Execute shell' marked build as failure
[Set GitHub commit status (universal)] ERROR on repos [GHRepository@86299f5[description=Apache Beam,homepage=,name=beam,fork=false,size=58754,milestones={},language=Java,commits={},source=<null>,parent=<null>,responseHeaderFields={null=[HTTP/1.1 200 OK], Access-Control-Allow-Origin=[*], Access-Control-Expose-Headers=[ETag, Link, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval], Cache-Control=[private, max-age=60, s-maxage=60], Content-Encoding=[gzip], Content-Security-Policy=[default-src 'none'], Content-Type=[application/json; charset=utf-8], Date=[Wed, 09 May 2018 12:05:00 GMT], ETag=[W/"c9c57ab721dc1f578ccd914bb574f9f9"], Last-Modified=[Wed, 09 May 2018 10:15:34 GMT], OkHttp-Received-Millis=[1525867500544], OkHttp-Response-Source=[NETWORK 200], OkHttp-Selected-Protocol=[http/1.1], OkHttp-Sent-Millis=[1525867500393], Referrer-Policy=[origin-when-cross-origin, strict-origin-when-cross-origin], Server=[GitHub.com], Status=[200 OK], Strict-Transport-Security=[max-age=31536000; includeSubdomains; preload], Transfer-Encoding=[chunked], Vary=[Accept, Authorization, Cookie, X-GitHub-OTP], X-Accepted-OAuth-Scopes=[repo], X-Content-Type-Options=[nosniff], X-Frame-Options=[deny], X-GitHub-Media-Type=[github.v3; format=json], X-GitHub-Request-Id=[948A:4F42:12D28AE:2634ECF:5AF2E3B2], X-OAuth-Scopes=[admin:repo_hook, repo, repo:status], X-RateLimit-Limit=[5000], X-RateLimit-Remaining=[4947], X-RateLimit-Reset=[1525867519], X-Runtime-rack=[0.068357], X-XSS-Protection=[1; mode=block]},url=https://api.github.com/repos/apache/beam,id=50904245]] (sha:60f90c8) with context:beam_PerformanceTests_Python
Setting commit status on GitHub for https://github.com/apache/beam/commit/60f90c8dcb229c35a82c7be15e64a89678bae058
ERROR: Build step failed with exception
java.io.FileNotFoundException: https://api.github.com/repos/apache/beam/statuses/60f90c8dcb229c35a82c7be15e64a89678bae058
at com.squareup.okhttp.internal.huc.HttpURLConnectionImpl.getInputStream(HttpURLConnectionImpl.java:243)
at com.squareup.okhttp.internal.huc.DelegatingHttpsURLConnection.getInputStream(DelegatingHttpsURLConnection.java:210)
at com.squareup.okhttp.internal.huc.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:25)
at org.kohsuke.github.Requester.parse(Requester.java:612)
at org.kohsuke.github.Requester.parse(Requester.java:594)
at org.kohsuke.github.Requester._to(Requester.java:272)
Caused: org.kohsuke.github.GHFileNotFoundException: {"message":"Not Found","documentation_url":"https://developer.github.com/v3/repos/statuses/#create-a-status"}
at org.kohsuke.github.Requester.handleApiError(Requester.java:686)
at org.kohsuke.github.Requester._to(Requester.java:293)
at org.kohsuke.github.Requester.to(Requester.java:234)
at org.kohsuke.github.GHRepository.createCommitStatus(GHRepository.java:1075)
at org.jenkinsci.plugins.github.status.GitHubCommitStatusSetter.perform(GitHubCommitStatusSetter.java:160)
Caused: org.jenkinsci.plugins.github.common.CombineErrorHandler$ErrorHandlingException
at org.jenkinsci.plugins.github.common.CombineErrorHandler.handle(CombineErrorHandler.java:74)
at org.jenkinsci.plugins.github.status.GitHubCommitStatusSetter.perform(GitHubCommitStatusSetter.java:164)
at com.cloudbees.jenkins.GitHubCommitNotifier.perform(GitHubCommitNotifier.java:151)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
at hudson.model.Run.execute(Run.java:1749)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Build step 'Set build status on GitHub commit [deprecated]' marked build as failure
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user jasonkuster@google.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user schroederc@google.com
Not sending mail to unregistered user shoyer@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1246
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1246/display/redirect?page=changes>
Changes:
[Pablo] Make experiments as set attr of RuntimeValueProvider
------------------------------------------
[...truncated 9.21 KB...]
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/aa/063eca6a416f397bd99552c534c6d11d57f58f2e94c14780f3bbf818c4cf/monotonic-1.5-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.5 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8165186307560750003.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-09 06:00:37,363 fc32181a MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/fc32181a/pkb.log>
2018-05-09 06:00:37,364 fc32181a MainThread INFO PerfKitBenchmarker version: v1.12.0-580-g36cfc82
2018-05-09 06:00:37,365 fc32181a MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-09 06:00:37,762 fc32181a MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-09 06:00:37,929 fc32181a MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-09 06:00:37,933 fc32181a MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-09 06:00:37,934 fc32181a MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-09 06:00:43,390 fc32181a MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/fc32181a/beam/gradlew> clean install -xcheck --stacktrace -DintegrationTestRunner=dataflow
2018-05-09 06:05:35,136 fc32181a MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-09 06:05:35,150 fc32181a MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/fc32181a/beam/sdks/python/target>
2018-05-09 06:05:35,151 fc32181a MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-09 06:05:35,154 fc32181a MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 781, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/fc32181a/beam/sdks/python/target>
2018-05-09 06:05:35,155 fc32181a MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-09 06:05:35,165 fc32181a MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-09 06:05:35,166 fc32181a MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/fc32181a/pkb.log>
2018-05-09 06:05:35,166 fc32181a MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/fc32181a/completion_statuses.json>
Build step 'Execute shell' marked build as failure
[Set GitHub commit status (universal)] ERROR on repos [GHRepository@3e9a4f53[description=Apache Beam,homepage=,name=beam,fork=false,size=58754,milestones={},language=Java,commits={},source=<null>,parent=<null>,responseHeaderFields={null=[HTTP/1.1 200 OK], Access-Control-Allow-Origin=[*], Access-Control-Expose-Headers=[ETag, Link, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval], Cache-Control=[private, max-age=60, s-maxage=60], Content-Encoding=[gzip], Content-Security-Policy=[default-src 'none'], Content-Type=[application/json; charset=utf-8], Date=[Wed, 09 May 2018 06:05:35 GMT], ETag=[W/"b325fcee1bcdcb6673ffb3a642ac3060"], Last-Modified=[Wed, 09 May 2018 00:26:13 GMT], OkHttp-Received-Millis=[1525845935908], OkHttp-Response-Source=[NETWORK 200], OkHttp-Selected-Protocol=[http/1.1], OkHttp-Sent-Millis=[1525845935761], Referrer-Policy=[origin-when-cross-origin, strict-origin-when-cross-origin], Server=[GitHub.com], Status=[200 OK], Strict-Transport-Security=[max-age=31536000; includeSubdomains; preload], Transfer-Encoding=[chunked], Vary=[Accept, Authorization, Cookie, X-GitHub-OTP], X-Accepted-OAuth-Scopes=[repo], X-Content-Type-Options=[nosniff], X-Frame-Options=[deny], X-GitHub-Media-Type=[github.v3; format=json], X-GitHub-Request-Id=[DE20:4F40:989E8C:14F5A45:5AF28F9A], X-OAuth-Scopes=[admin:repo_hook, repo, repo:status], X-RateLimit-Limit=[5000], X-RateLimit-Remaining=[4967], X-RateLimit-Reset=[1525848688], X-Runtime-rack=[0.051818], X-XSS-Protection=[1; mode=block]},url=https://api.github.com/repos/apache/beam,id=50904245]] (sha:60f90c8) with context:beam_PerformanceTests_Python
Setting commit status on GitHub for https://github.com/apache/beam/commit/60f90c8dcb229c35a82c7be15e64a89678bae058
ERROR: Build step failed with exception
java.io.FileNotFoundException: https://api.github.com/repos/apache/beam/statuses/60f90c8dcb229c35a82c7be15e64a89678bae058
at com.squareup.okhttp.internal.huc.HttpURLConnectionImpl.getInputStream(HttpURLConnectionImpl.java:243)
at com.squareup.okhttp.internal.huc.DelegatingHttpsURLConnection.getInputStream(DelegatingHttpsURLConnection.java:210)
at com.squareup.okhttp.internal.huc.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:25)
at org.kohsuke.github.Requester.parse(Requester.java:612)
at org.kohsuke.github.Requester.parse(Requester.java:594)
at org.kohsuke.github.Requester._to(Requester.java:272)
Caused: org.kohsuke.github.GHFileNotFoundException: {"message":"Not Found","documentation_url":"https://developer.github.com/v3/repos/statuses/#create-a-status"}
at org.kohsuke.github.Requester.handleApiError(Requester.java:686)
at org.kohsuke.github.Requester._to(Requester.java:293)
at org.kohsuke.github.Requester.to(Requester.java:234)
at org.kohsuke.github.GHRepository.createCommitStatus(GHRepository.java:1075)
at org.jenkinsci.plugins.github.status.GitHubCommitStatusSetter.perform(GitHubCommitStatusSetter.java:160)
Caused: org.jenkinsci.plugins.github.common.CombineErrorHandler$ErrorHandlingException
at org.jenkinsci.plugins.github.common.CombineErrorHandler.handle(CombineErrorHandler.java:74)
at org.jenkinsci.plugins.github.status.GitHubCommitStatusSetter.perform(GitHubCommitStatusSetter.java:164)
at com.cloudbees.jenkins.GitHubCommitNotifier.perform(GitHubCommitNotifier.java:151)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
at hudson.model.Run.execute(Run.java:1749)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Build step 'Set build status on GitHub commit [deprecated]' marked build as failure
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user jasonkuster@google.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user schroederc@google.com
Not sending mail to unregistered user shoyer@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1245
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1245/display/redirect?page=changes>
Changes:
[github] Explictly delcare globals defined elsewhere
[mairbek] Introduced SpannerWriteResult that
[mairbek] Addressed comments
[mairbek] Happy checkstyle
[github] Adding a microbenchmark for side input iterables. (#5294)
[apilloud] Enable githubCommitNotifier for post commits
[tgroh] Migrate the `portable` subpackage to Portability
------------------------------------------
[...truncated 5.12 KB...]
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1693927279099916115.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/aa/063eca6a416f397bd99552c534c6d11d57f58f2e94c14780f3bbf818c4cf/monotonic-1.5-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.5 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1423956943309251479.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-09 00:22:47,596 487ca9f1 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/487ca9f1/pkb.log>
2018-05-09 00:22:47,596 487ca9f1 MainThread INFO PerfKitBenchmarker version: v1.12.0-580-g36cfc82
2018-05-09 00:22:47,597 487ca9f1 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-09 00:22:47,873 487ca9f1 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-09 00:22:48,031 487ca9f1 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-09 00:22:48,035 487ca9f1 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-09 00:22:48,035 487ca9f1 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-09 00:22:53,199 487ca9f1 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/487ca9f1/beam/gradlew> clean install -xcheck --stacktrace -DintegrationTestRunner=dataflow
2018-05-09 00:26:48,573 487ca9f1 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-09 00:26:48,584 487ca9f1 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/487ca9f1/beam/sdks/python/target>
2018-05-09 00:26:48,585 487ca9f1 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-09 00:26:48,588 487ca9f1 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 781, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/487ca9f1/beam/sdks/python/target>
2018-05-09 00:26:48,588 487ca9f1 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-09 00:26:48,630 487ca9f1 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-09 00:26:48,631 487ca9f1 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/487ca9f1/pkb.log>
2018-05-09 00:26:48,631 487ca9f1 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/487ca9f1/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user jasonkuster@google.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user schroederc@google.com
Not sending mail to unregistered user shoyer@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1244
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1244/display/redirect?page=changes>
Changes:
[schroederc] Move textio.FileSystem and its registry to a separate package
[schroederc] Move filesystem implementations under beam/io/filesystem package
[wcn] Fix missing return in datastore connector.
[swegner] Create ValidatesRunner task for DirectRunner.
[swegner] Convert some tests from ValidatesRunner -> NeedsRunner
[apilloud] [SQL] Cleanout unneeded sqlEnv
[apilloud] [SQL] Hide details of BeamSqlEnv from rel test
[apilloud] [SQL] Simplify TableProvider interface
[apilloud] [SQL] Return map from TableProvider.getTables
[apilloud] [SQL] Add tables via TableStore in CalciteSchema
[apilloud] [SQL] Move Create and Drop Table inline with ddl
[tgroh] Suppress Thread Leak Test
[tgroh] Rename ExecutorServiceParlallelExecutorTest#test
------------------------------------------
[...truncated 4.74 KB...]
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4320043418822733879.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/aa/063eca6a416f397bd99552c534c6d11d57f58f2e94c14780f3bbf818c4cf/monotonic-1.5-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.5 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1124193938476074160.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-08 18:00:37,274 663fc479 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/663fc479/pkb.log>
2018-05-08 18:00:37,275 663fc479 MainThread INFO PerfKitBenchmarker version: v1.12.0-580-g36cfc82
2018-05-08 18:00:37,275 663fc479 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-08 18:00:37,484 663fc479 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-08 18:00:37,631 663fc479 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-08 18:00:37,635 663fc479 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-08 18:00:37,635 663fc479 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-08 18:00:42,535 663fc479 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/663fc479/beam/gradlew> clean install -xcheck --stacktrace -DintegrationTestRunner=dataflow
2018-05-08 18:04:53,410 663fc479 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-08 18:04:53,420 663fc479 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/663fc479/beam/sdks/python/target>
2018-05-08 18:04:53,421 663fc479 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-08 18:04:53,422 663fc479 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 781, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/663fc479/beam/sdks/python/target>
2018-05-08 18:04:53,423 663fc479 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-08 18:04:53,448 663fc479 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-08 18:04:53,449 663fc479 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/663fc479/pkb.log>
2018-05-08 18:04:53,449 663fc479 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/663fc479/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user jasonkuster@google.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user shoyer@google.com
Not sending mail to unregistered user schroederc@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1243
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1243/display/redirect>
------------------------------------------
[...truncated 4.52 KB...]
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins477075854833080956.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/aa/063eca6a416f397bd99552c534c6d11d57f58f2e94c14780f3bbf818c4cf/monotonic-1.5-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.5 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5332734419671954790.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-08 12:00:43,852 9097e4d9 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/9097e4d9/pkb.log>
2018-05-08 12:00:43,853 9097e4d9 MainThread INFO PerfKitBenchmarker version: v1.12.0-578-g1836465
2018-05-08 12:00:43,854 9097e4d9 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-08 12:00:44,083 9097e4d9 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-08 12:00:44,245 9097e4d9 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-08 12:00:44,249 9097e4d9 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-08 12:00:44,249 9097e4d9 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-08 12:00:49,441 9097e4d9 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/9097e4d9/beam/gradlew> clean install -xcheck --stacktrace -DintegrationTestRunner=dataflow
2018-05-08 12:05:02,447 9097e4d9 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-08 12:05:02,457 9097e4d9 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/9097e4d9/beam/sdks/python/target>
2018-05-08 12:05:02,458 9097e4d9 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-08 12:05:02,460 9097e4d9 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 781, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/9097e4d9/beam/sdks/python/target>
2018-05-08 12:05:02,460 9097e4d9 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-08 12:05:02,488 9097e4d9 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-08 12:05:02,489 9097e4d9 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/9097e4d9/pkb.log>
2018-05-08 12:05:02,490 9097e4d9 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/9097e4d9/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user jasonkuster@google.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user shoyer@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1242
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1242/display/redirect?page=changes>
Changes:
[daniel.o.programmer] Change to docstring in filesystem.py
[github] Moving Python PostCommit into a Gradle task. (#5289)
------------------------------------------
[...truncated 4.53 KB...]
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3370853666288046184.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/aa/063eca6a416f397bd99552c534c6d11d57f58f2e94c14780f3bbf818c4cf/monotonic-1.5-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.5 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins261813762147085139.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-08 06:00:38,217 469eec03 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/469eec03/pkb.log>
2018-05-08 06:00:38,218 469eec03 MainThread INFO PerfKitBenchmarker version: v1.12.0-578-g1836465
2018-05-08 06:00:38,219 469eec03 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-08 06:00:38,621 469eec03 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-08 06:00:38,779 469eec03 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-08 06:00:38,782 469eec03 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-08 06:00:38,783 469eec03 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-08 06:00:44,181 469eec03 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/469eec03/beam/gradlew> clean install -xcheck --stacktrace -DintegrationTestRunner=dataflow
2018-05-08 06:05:28,127 469eec03 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-08 06:05:28,136 469eec03 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/469eec03/beam/sdks/python/target>
2018-05-08 06:05:28,137 469eec03 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-08 06:05:28,139 469eec03 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 781, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/469eec03/beam/sdks/python/target>
2018-05-08 06:05:28,139 469eec03 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-08 06:05:28,198 469eec03 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-08 06:05:28,199 469eec03 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/469eec03/pkb.log>
2018-05-08 06:05:28,199 469eec03 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/469eec03/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user jasonkuster@google.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user shoyer@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1241
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1241/display/redirect?page=changes>
Changes:
[thw] [BEAM-4246] Remove Maven build directory dependency in Apex runner.
[thw] Upgrade Apex version to 3.7.0 to get rid of poisened netlet 1.3.0
------------------------------------------
[...truncated 4.55 KB...]
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4753164093500792055.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/aa/063eca6a416f397bd99552c534c6d11d57f58f2e94c14780f3bbf818c4cf/monotonic-1.5-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.5 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6459374300070367387.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-08 00:00:40,684 74b2afca MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/74b2afca/pkb.log>
2018-05-08 00:00:40,684 74b2afca MainThread INFO PerfKitBenchmarker version: v1.12.0-578-g1836465
2018-05-08 00:00:40,685 74b2afca MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-08 00:00:41,194 74b2afca MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-08 00:00:41,381 74b2afca MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-08 00:00:41,385 74b2afca MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-08 00:00:41,386 74b2afca MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-08 00:00:47,591 74b2afca MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/74b2afca/beam/gradlew> clean install -xcheck --stacktrace -DintegrationTestRunner=dataflow
2018-05-08 00:10:08,648 74b2afca MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-08 00:10:08,662 74b2afca MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/74b2afca/beam/sdks/python/target>
2018-05-08 00:10:08,664 74b2afca MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-08 00:10:08,666 74b2afca MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 781, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/74b2afca/beam/sdks/python/target>
2018-05-08 00:10:08,667 74b2afca MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-08 00:10:08,705 74b2afca MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-08 00:10:08,706 74b2afca MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/74b2afca/pkb.log>
2018-05-08 00:10:08,707 74b2afca MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/74b2afca/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user jasonkuster@google.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user shoyer@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1240
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1240/display/redirect>
------------------------------------------
[...truncated 4.54 KB...]
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4661909573531510816.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/aa/063eca6a416f397bd99552c534c6d11d57f58f2e94c14780f3bbf818c4cf/monotonic-1.5-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.5 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins14739166260006659.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-07 18:01:02,030 f72fcf8f MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/f72fcf8f/pkb.log>
2018-05-07 18:01:02,031 f72fcf8f MainThread INFO PerfKitBenchmarker version: v1.12.0-578-g1836465
2018-05-07 18:01:02,033 f72fcf8f MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-07 18:01:02,453 f72fcf8f MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-07 18:01:02,707 f72fcf8f MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-07 18:01:02,711 f72fcf8f MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-07 18:01:02,712 f72fcf8f MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-07 18:01:09,683 f72fcf8f MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/f72fcf8f/beam/gradlew> clean install -xcheck --stacktrace -DintegrationTestRunner=dataflow
2018-05-07 18:06:44,809 f72fcf8f MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-07 18:06:44,822 f72fcf8f MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/f72fcf8f/beam/sdks/python/target>
2018-05-07 18:06:44,824 f72fcf8f MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-07 18:06:44,826 f72fcf8f MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 781, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/f72fcf8f/beam/sdks/python/target>
2018-05-07 18:06:44,827 f72fcf8f MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-07 18:06:44,884 f72fcf8f MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-07 18:06:44,885 f72fcf8f MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/f72fcf8f/pkb.log>
2018-05-07 18:06:44,886 f72fcf8f MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/f72fcf8f/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user jasonkuster@google.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user shoyer@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1239
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1239/display/redirect>
------------------------------------------
[...truncated 4.54 KB...]
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4998656870473128651.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/aa/063eca6a416f397bd99552c534c6d11d57f58f2e94c14780f3bbf818c4cf/monotonic-1.5-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.5 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1042977088663791323.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-07 12:00:38,770 065bb00f MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/065bb00f/pkb.log>
2018-05-07 12:00:38,770 065bb00f MainThread INFO PerfKitBenchmarker version: v1.12.0-577-g0d6a9aa
2018-05-07 12:00:38,771 065bb00f MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-07 12:00:39,017 065bb00f MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-07 12:00:39,295 065bb00f MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-07 12:00:39,302 065bb00f MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-07 12:00:39,303 065bb00f MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-07 12:00:45,149 065bb00f MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/065bb00f/beam/gradlew> clean install -xcheck --stacktrace -DintegrationTestRunner=dataflow
2018-05-07 12:05:52,049 065bb00f MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-07 12:05:52,062 065bb00f MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/065bb00f/beam/sdks/python/target>
2018-05-07 12:05:52,063 065bb00f MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-07 12:05:52,065 065bb00f MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 781, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/065bb00f/beam/sdks/python/target>
2018-05-07 12:05:52,066 065bb00f MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-07 12:05:52,105 065bb00f MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-07 12:05:52,106 065bb00f MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/065bb00f/pkb.log>
2018-05-07 12:05:52,107 065bb00f MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/065bb00f/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user jasonkuster@google.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user shoyer@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1238
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1238/display/redirect>
------------------------------------------
[...truncated 4.54 KB...]
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2000152233350451449.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/aa/063eca6a416f397bd99552c534c6d11d57f58f2e94c14780f3bbf818c4cf/monotonic-1.5-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.5 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins9107637541295296419.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-07 06:00:41,294 8eb482dc MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/8eb482dc/pkb.log>
2018-05-07 06:00:41,294 8eb482dc MainThread INFO PerfKitBenchmarker version: v1.12.0-577-g0d6a9aa
2018-05-07 06:00:41,295 8eb482dc MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-07 06:00:41,501 8eb482dc MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-07 06:00:41,664 8eb482dc MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-07 06:00:41,668 8eb482dc MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-07 06:00:41,669 8eb482dc MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-07 06:00:46,971 8eb482dc MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/8eb482dc/beam/gradlew> clean install -xcheck --stacktrace -DintegrationTestRunner=dataflow
2018-05-07 06:04:55,495 8eb482dc MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-07 06:04:55,506 8eb482dc MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/8eb482dc/beam/sdks/python/target>
2018-05-07 06:04:55,507 8eb482dc MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-07 06:04:55,509 8eb482dc MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 781, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/8eb482dc/beam/sdks/python/target>
2018-05-07 06:04:55,509 8eb482dc MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-07 06:04:55,527 8eb482dc MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-07 06:04:55,528 8eb482dc MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/8eb482dc/pkb.log>
2018-05-07 06:04:55,529 8eb482dc MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/8eb482dc/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user jasonkuster@google.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user shoyer@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1237
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1237/display/redirect>
------------------------------------------
[...truncated 4.54 KB...]
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7554523155013107655.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/aa/063eca6a416f397bd99552c534c6d11d57f58f2e94c14780f3bbf818c4cf/monotonic-1.5-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.5 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3655198860506791688.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-07 00:00:35,661 b22c5959 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/b22c5959/pkb.log>
2018-05-07 00:00:35,661 b22c5959 MainThread INFO PerfKitBenchmarker version: v1.12.0-577-g0d6a9aa
2018-05-07 00:00:35,662 b22c5959 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-07 00:00:36,042 b22c5959 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-07 00:00:36,232 b22c5959 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-07 00:00:36,235 b22c5959 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-07 00:00:36,236 b22c5959 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-07 00:00:41,922 b22c5959 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/b22c5959/beam/gradlew> clean install -xcheck --stacktrace -DintegrationTestRunner=dataflow
2018-05-07 00:06:13,506 b22c5959 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-07 00:06:13,520 b22c5959 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/b22c5959/beam/sdks/python/target>
2018-05-07 00:06:13,522 b22c5959 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-07 00:06:13,525 b22c5959 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 781, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/b22c5959/beam/sdks/python/target>
2018-05-07 00:06:13,526 b22c5959 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-07 00:06:13,558 b22c5959 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-07 00:06:13,559 b22c5959 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/b22c5959/pkb.log>
2018-05-07 00:06:13,560 b22c5959 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/b22c5959/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user jasonkuster@google.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user shoyer@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1236
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1236/display/redirect>
------------------------------------------
[...truncated 4.54 KB...]
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4485151787855792818.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/aa/063eca6a416f397bd99552c534c6d11d57f58f2e94c14780f3bbf818c4cf/monotonic-1.5-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.5 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5147281323271792884.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-06 18:00:41,791 37dc18da MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/37dc18da/pkb.log>
2018-05-06 18:00:41,792 37dc18da MainThread INFO PerfKitBenchmarker version: v1.12.0-577-g0d6a9aa
2018-05-06 18:00:41,793 37dc18da MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-06 18:00:42,113 37dc18da MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-06 18:00:42,332 37dc18da MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-06 18:00:42,337 37dc18da MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-06 18:00:42,338 37dc18da MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-06 18:00:48,161 37dc18da MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/37dc18da/beam/gradlew> clean install -xcheck --stacktrace -DintegrationTestRunner=dataflow
2018-05-06 18:06:43,404 37dc18da MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-06 18:06:43,416 37dc18da MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/37dc18da/beam/sdks/python/target>
2018-05-06 18:06:43,418 37dc18da MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-06 18:06:43,420 37dc18da MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 781, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/37dc18da/beam/sdks/python/target>
2018-05-06 18:06:43,421 37dc18da MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-06 18:06:43,436 37dc18da MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-06 18:06:43,437 37dc18da MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/37dc18da/pkb.log>
2018-05-06 18:06:43,437 37dc18da MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/37dc18da/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user jasonkuster@google.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user shoyer@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1235
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1235/display/redirect>
------------------------------------------
[...truncated 4.54 KB...]
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4200929610893318071.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/aa/063eca6a416f397bd99552c534c6d11d57f58f2e94c14780f3bbf818c4cf/monotonic-1.5-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.5 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1217229940661743471.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-06 12:00:37,795 e296e3cd MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e296e3cd/pkb.log>
2018-05-06 12:00:37,796 e296e3cd MainThread INFO PerfKitBenchmarker version: v1.12.0-577-g0d6a9aa
2018-05-06 12:00:37,797 e296e3cd MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-06 12:00:37,995 e296e3cd MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-06 12:00:38,166 e296e3cd MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-06 12:00:38,169 e296e3cd MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-06 12:00:38,170 e296e3cd MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-06 12:00:43,481 e296e3cd MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e296e3cd/beam/gradlew> clean install -xcheck --stacktrace -DintegrationTestRunner=dataflow
2018-05-06 12:05:17,129 e296e3cd MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-06 12:05:17,141 e296e3cd MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e296e3cd/beam/sdks/python/target>
2018-05-06 12:05:17,142 e296e3cd MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-06 12:05:17,144 e296e3cd MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 781, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e296e3cd/beam/sdks/python/target>
2018-05-06 12:05:17,145 e296e3cd MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-06 12:05:17,193 e296e3cd MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-06 12:05:17,194 e296e3cd MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e296e3cd/pkb.log>
2018-05-06 12:05:17,195 e296e3cd MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e296e3cd/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user jasonkuster@google.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user shoyer@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1234
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1234/display/redirect>
------------------------------------------
[...truncated 4.54 KB...]
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5934797366180056880.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/aa/063eca6a416f397bd99552c534c6d11d57f58f2e94c14780f3bbf818c4cf/monotonic-1.5-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.5 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4496668234875030015.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-06 06:00:33,170 a572d4d3 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/a572d4d3/pkb.log>
2018-05-06 06:00:33,170 a572d4d3 MainThread INFO PerfKitBenchmarker version: v1.12.0-577-g0d6a9aa
2018-05-06 06:00:33,171 a572d4d3 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-06 06:00:33,607 a572d4d3 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-06 06:00:33,771 a572d4d3 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-06 06:00:33,774 a572d4d3 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-06 06:00:33,775 a572d4d3 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-06 06:00:38,921 a572d4d3 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/a572d4d3/beam/gradlew> clean install -xcheck --stacktrace -DintegrationTestRunner=dataflow
2018-05-06 06:05:14,371 a572d4d3 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-06 06:05:14,381 a572d4d3 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/a572d4d3/beam/sdks/python/target>
2018-05-06 06:05:14,383 a572d4d3 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-06 06:05:14,384 a572d4d3 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 781, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/a572d4d3/beam/sdks/python/target>
2018-05-06 06:05:14,385 a572d4d3 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-06 06:05:14,443 a572d4d3 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-06 06:05:14,444 a572d4d3 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/a572d4d3/pkb.log>
2018-05-06 06:05:14,444 a572d4d3 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/a572d4d3/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user jasonkuster@google.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user shoyer@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1233
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1233/display/redirect>
------------------------------------------
[...truncated 4.54 KB...]
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins238767545631536820.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/aa/063eca6a416f397bd99552c534c6d11d57f58f2e94c14780f3bbf818c4cf/monotonic-1.5-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.5 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1651119597549796430.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-06 00:00:34,879 f0963711 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/f0963711/pkb.log>
2018-05-06 00:00:34,880 f0963711 MainThread INFO PerfKitBenchmarker version: v1.12.0-577-g0d6a9aa
2018-05-06 00:00:34,882 f0963711 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-06 00:00:35,138 f0963711 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-06 00:00:35,315 f0963711 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-06 00:00:35,318 f0963711 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-06 00:00:35,319 f0963711 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-06 00:00:41,315 f0963711 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/f0963711/beam/gradlew> clean install -xcheck --stacktrace -DintegrationTestRunner=dataflow
2018-05-06 00:05:59,334 f0963711 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-06 00:05:59,347 f0963711 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/f0963711/beam/sdks/python/target>
2018-05-06 00:05:59,349 f0963711 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-06 00:05:59,351 f0963711 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 781, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/f0963711/beam/sdks/python/target>
2018-05-06 00:05:59,352 f0963711 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-06 00:05:59,407 f0963711 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-06 00:05:59,408 f0963711 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/f0963711/pkb.log>
2018-05-06 00:05:59,408 f0963711 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/f0963711/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user jasonkuster@google.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user shoyer@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1232
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1232/display/redirect>
------------------------------------------
[...truncated 4.54 KB...]
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6226459328513373971.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/aa/063eca6a416f397bd99552c534c6d11d57f58f2e94c14780f3bbf818c4cf/monotonic-1.5-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.5 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7432409411430361103.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-05 18:02:22,058 0edebfde MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0edebfde/pkb.log>
2018-05-05 18:02:22,059 0edebfde MainThread INFO PerfKitBenchmarker version: v1.12.0-577-g0d6a9aa
2018-05-05 18:02:22,060 0edebfde MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-05 18:02:22,337 0edebfde MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-05 18:02:22,505 0edebfde MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-05 18:02:22,509 0edebfde MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-05 18:02:22,509 0edebfde MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-05 18:02:28,461 0edebfde MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0edebfde/beam/gradlew> clean install -xcheck --stacktrace -DintegrationTestRunner=dataflow
2018-05-05 18:07:40,899 0edebfde MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-05 18:07:40,909 0edebfde MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0edebfde/beam/sdks/python/target>
2018-05-05 18:07:40,911 0edebfde MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-05 18:07:40,913 0edebfde MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 781, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0edebfde/beam/sdks/python/target>
2018-05-05 18:07:40,914 0edebfde MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-05 18:07:40,930 0edebfde MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-05 18:07:40,931 0edebfde MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0edebfde/pkb.log>
2018-05-05 18:07:40,932 0edebfde MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0edebfde/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user jasonkuster@google.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user shoyer@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1231
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1231/display/redirect?page=changes>
Changes:
[apilloud] Revert "Merge pull request #5220: [BEAM-3983][SQL] Add BigQuery table
[apilloud] Revert "Merge pull request #4947: [BEAM-3983] Add utils for converting
------------------------------------------
[...truncated 4.54 KB...]
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3294619523682552139.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/aa/063eca6a416f397bd99552c534c6d11d57f58f2e94c14780f3bbf818c4cf/monotonic-1.5-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.5 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7187232489825388123.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-05 12:04:52,705 7d7d4d49 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/7d7d4d49/pkb.log>
2018-05-05 12:04:52,705 7d7d4d49 MainThread INFO PerfKitBenchmarker version: v1.12.0-577-g0d6a9aa
2018-05-05 12:04:52,706 7d7d4d49 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-05 12:04:53,198 7d7d4d49 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-05 12:04:53,370 7d7d4d49 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-05 12:04:53,374 7d7d4d49 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-05 12:04:53,374 7d7d4d49 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-05 12:04:58,462 7d7d4d49 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/7d7d4d49/beam/gradlew> clean install -xcheck --stacktrace -DintegrationTestRunner=dataflow
2018-05-05 12:09:49,399 7d7d4d49 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-05 12:09:49,409 7d7d4d49 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/7d7d4d49/beam/sdks/python/target>
2018-05-05 12:09:49,411 7d7d4d49 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-05 12:09:49,413 7d7d4d49 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 781, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/7d7d4d49/beam/sdks/python/target>
2018-05-05 12:09:49,413 7d7d4d49 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-05 12:09:49,443 7d7d4d49 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-05 12:09:49,443 7d7d4d49 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/7d7d4d49/pkb.log>
2018-05-05 12:09:49,444 7d7d4d49 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/7d7d4d49/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user jasonkuster@google.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user shoyer@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1230
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1230/display/redirect?page=changes>
Changes:
[coheigea] Combine nested if statements
------------------------------------------
[...truncated 4.49 KB...]
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7079395939582489318.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/aa/063eca6a416f397bd99552c534c6d11d57f58f2e94c14780f3bbf818c4cf/monotonic-1.5-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.5 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2685674630389068120.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-05 06:02:25,650 180360c0 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/180360c0/pkb.log>
2018-05-05 06:02:25,651 180360c0 MainThread INFO PerfKitBenchmarker version: v1.12.0-577-g0d6a9aa
2018-05-05 06:02:25,652 180360c0 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-05 06:02:25,990 180360c0 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-05 06:02:26,201 180360c0 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-05 06:02:26,206 180360c0 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-05 06:02:26,206 180360c0 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-05 06:02:31,567 180360c0 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/180360c0/beam/gradlew> clean install -xcheck --stacktrace -DintegrationTestRunner=dataflow
2018-05-05 06:06:23,434 180360c0 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-05 06:06:23,442 180360c0 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/180360c0/beam/sdks/python/target>
2018-05-05 06:06:23,443 180360c0 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-05 06:06:23,445 180360c0 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 781, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/180360c0/beam/sdks/python/target>
2018-05-05 06:06:23,445 180360c0 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-05 06:06:23,477 180360c0 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-05 06:06:23,478 180360c0 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/180360c0/pkb.log>
2018-05-05 06:06:23,478 180360c0 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/180360c0/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user jasonkuster@google.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user shoyer@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1229
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1229/display/redirect?page=changes>
Changes:
[mariagh] Update CoGroupByKey docstring
[sidhom] Use wildcard generics rather than Object in portable translator
[sidhom] [BEAM-4231] Runner utility for coder instantiation
------------------------------------------
[...truncated 4.51 KB...]
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins75125069342373121.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/aa/063eca6a416f397bd99552c534c6d11d57f58f2e94c14780f3bbf818c4cf/monotonic-1.5-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.5 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4842221025192894472.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-05 00:09:14,174 6bbbab9e MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/6bbbab9e/pkb.log>
2018-05-05 00:09:14,175 6bbbab9e MainThread INFO PerfKitBenchmarker version: v1.12.0-577-g0d6a9aa
2018-05-05 00:09:14,176 6bbbab9e MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-05 00:09:14,554 6bbbab9e MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-05 00:09:14,821 6bbbab9e MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-05 00:09:14,827 6bbbab9e MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-05 00:09:14,828 6bbbab9e MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-05 00:09:21,768 6bbbab9e MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/6bbbab9e/beam/gradlew> clean install -xcheck --stacktrace -DintegrationTestRunner=dataflow
2018-05-05 00:15:23,678 6bbbab9e MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-05 00:15:23,696 6bbbab9e MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/6bbbab9e/beam/sdks/python/target>
2018-05-05 00:15:23,698 6bbbab9e MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-05 00:15:23,701 6bbbab9e MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 781, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/6bbbab9e/beam/sdks/python/target>
2018-05-05 00:15:23,701 6bbbab9e MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-05 00:15:23,735 6bbbab9e MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-05 00:15:23,736 6bbbab9e MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/6bbbab9e/pkb.log>
2018-05-05 00:15:23,738 6bbbab9e MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/6bbbab9e/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user jasonkuster@google.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user shoyer@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1228
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1228/display/redirect?page=changes>
Changes:
[swegner] Remove Gradle worker cap.
[swegner] Re-enable additional Jenkins worker parallelism for Dataflow
[swegner] Adjust Jenkins Gradle memory usage based on max-workers
[swegner] Hard-code jenkins machines total memory until we find a working API
------------------------------------------
[...truncated 4.49 KB...]
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6915006597034856160.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/aa/063eca6a416f397bd99552c534c6d11d57f58f2e94c14780f3bbf818c4cf/monotonic-1.5-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.5 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6503675509702015280.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-04 18:04:17,845 8c2cc485 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/8c2cc485/pkb.log>
2018-05-04 18:04:17,846 8c2cc485 MainThread INFO PerfKitBenchmarker version: v1.12.0-573-gb2a0a68
2018-05-04 18:04:17,847 8c2cc485 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-04 18:04:18,272 8c2cc485 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-04 18:04:18,536 8c2cc485 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-04 18:04:18,541 8c2cc485 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-04 18:04:18,542 8c2cc485 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-04 18:04:25,240 8c2cc485 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/8c2cc485/beam/gradlew> clean install -xcheck --stacktrace -DintegrationTestRunner=dataflow
2018-05-04 18:09:36,636 8c2cc485 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-04 18:09:36,649 8c2cc485 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/8c2cc485/beam/sdks/python/target>
2018-05-04 18:09:36,651 8c2cc485 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-04 18:09:36,653 8c2cc485 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 780, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/8c2cc485/beam/sdks/python/target>
2018-05-04 18:09:36,653 8c2cc485 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-04 18:09:36,685 8c2cc485 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-04 18:09:36,686 8c2cc485 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/8c2cc485/pkb.log>
2018-05-04 18:09:36,686 8c2cc485 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/8c2cc485/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user jasonkuster@google.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user shoyer@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1227
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1227/display/redirect?page=changes>
Changes:
[iemejia] Make MetricsHttpSinkTest more concise and fix typo
------------------------------------------
[...truncated 4.52 KB...]
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5206099886400342274.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/aa/063eca6a416f397bd99552c534c6d11d57f58f2e94c14780f3bbf818c4cf/monotonic-1.5-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.5 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1882754806044232706.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-04 12:01:04,296 fa13bb7d MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/fa13bb7d/pkb.log>
2018-05-04 12:01:04,297 fa13bb7d MainThread INFO PerfKitBenchmarker version: v1.12.0-573-gb2a0a68
2018-05-04 12:01:04,297 fa13bb7d MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-04 12:01:04,788 fa13bb7d MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-04 12:01:04,948 fa13bb7d MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-04 12:01:04,952 fa13bb7d MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-04 12:01:04,952 fa13bb7d MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-04 12:01:11,002 fa13bb7d MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/fa13bb7d/beam/gradlew> clean install -xcheck --stacktrace -DintegrationTestRunner=dataflow
2018-05-04 12:05:41,317 fa13bb7d MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-04 12:05:41,324 fa13bb7d MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/fa13bb7d/beam/sdks/python/target>
2018-05-04 12:05:41,325 fa13bb7d MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-04 12:05:41,327 fa13bb7d MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 780, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/fa13bb7d/beam/sdks/python/target>
2018-05-04 12:05:41,327 fa13bb7d MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-04 12:05:41,381 fa13bb7d MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-04 12:05:41,382 fa13bb7d MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/fa13bb7d/pkb.log>
2018-05-04 12:05:41,382 fa13bb7d MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/fa13bb7d/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user jasonkuster@google.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user shoyer@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1226
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1226/display/redirect?page=changes>
Changes:
[tgroh] Add an Unsupported StateRequestHandler
[apilloud] [SQL] Use the same javacc and fmpp as calcite
[apilloud] [SQL] BeamQueryPlanner uses BeamSqlParserImpl
------------------------------------------
[...truncated 4.55 KB...]
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2383670572846156876.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/aa/063eca6a416f397bd99552c534c6d11d57f58f2e94c14780f3bbf818c4cf/monotonic-1.5-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.5 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3837318280685648430.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-04 06:00:45,206 38c943f1 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/38c943f1/pkb.log>
2018-05-04 06:00:45,207 38c943f1 MainThread INFO PerfKitBenchmarker version: v1.12.0-573-gb2a0a68
2018-05-04 06:00:45,207 38c943f1 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-04 06:00:45,501 38c943f1 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-04 06:00:45,658 38c943f1 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-04 06:00:45,661 38c943f1 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-04 06:00:45,662 38c943f1 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-04 06:00:50,828 38c943f1 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/38c943f1/beam/gradlew> clean install -xcheck --stacktrace -DintegrationTestRunner=dataflow
2018-05-04 06:05:16,462 38c943f1 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-04 06:05:16,470 38c943f1 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/38c943f1/beam/sdks/python/target>
2018-05-04 06:05:16,472 38c943f1 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-04 06:05:16,473 38c943f1 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 780, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/38c943f1/beam/sdks/python/target>
2018-05-04 06:05:16,474 38c943f1 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-04 06:05:16,501 38c943f1 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-04 06:05:16,502 38c943f1 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/38c943f1/pkb.log>
2018-05-04 06:05:16,502 38c943f1 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/38c943f1/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user jasonkuster@google.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user shoyer@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1225
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1225/display/redirect?page=changes>
Changes:
[apilloud] [SQL] Add BeamEnumerableConverter
[mairbek] Templatize host name in SpannerIO
[coheigea] A few small fixes to catch a few regressions - put String literals
[sidhom] Move SyntheticNodes to top-level construction package and rename
[sidhom] Pull Flink execution environment utilities into static methods
[sidhom] Refactor Runner/SDK wire coder creation into a shared utility
[sidhom] [BEAM-3972] Translate portable batch pipelines by proto
[sidhom] FIXUP: Use ModelCoders to create proto coders and use WireCoders to
[sidhom] FIXUP: Use RunnerApi in WireCoders to prepare for coder instantiation
[tgroh] Add additional methods to QueryablePipeline
[tgroh] Expand the ExecutableGraph interface
[tgroh] Add ProtoOverrides
[jasonkuster] Add labels to Go SDK Dataflow Runner.
[jasonkuster] Add verbose error message if unmarshaling fails for label flag.
------------------------------------------
[...truncated 4.55 KB...]
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins816417449346650119.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/aa/063eca6a416f397bd99552c534c6d11d57f58f2e94c14780f3bbf818c4cf/monotonic-1.5-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.5 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4561375221741013585.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-04 00:00:50,218 e9647bc1 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e9647bc1/pkb.log>
2018-05-04 00:00:50,219 e9647bc1 MainThread INFO PerfKitBenchmarker version: v1.12.0-573-gb2a0a68
2018-05-04 00:00:50,219 e9647bc1 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-04 00:00:50,477 e9647bc1 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-04 00:00:50,643 e9647bc1 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-04 00:00:50,647 e9647bc1 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-04 00:00:50,647 e9647bc1 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-04 00:00:55,795 e9647bc1 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e9647bc1/beam/gradlew> clean install -xcheck --stacktrace -DintegrationTestRunner=dataflow
2018-05-04 00:05:30,908 e9647bc1 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-04 00:05:30,917 e9647bc1 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e9647bc1/beam/sdks/python/target>
2018-05-04 00:05:30,918 e9647bc1 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-04 00:05:30,920 e9647bc1 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 780, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e9647bc1/beam/sdks/python/target>
2018-05-04 00:05:30,920 e9647bc1 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-04 00:05:30,940 e9647bc1 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-04 00:05:30,941 e9647bc1 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e9647bc1/pkb.log>
2018-05-04 00:05:30,941 e9647bc1 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e9647bc1/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user jasonkuster@google.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user shoyer@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1224
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1224/display/redirect>
------------------------------------------
[...truncated 4.46 KB...]
Using cached https://files.pythonhosted.org/packages/a2/71/8273a7eeed0aff6a854237ab5453bc9aa67deb49df4832801c21f0ff3782/contextlib2-0.5.5-py2.py3-none-any.whl
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins603971570518380326.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9f/ff/68860dec8b90424028ca54df8b9fc8cd1c47ef58710bd5e591ec8c619243/monotonic-1.4-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.4 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7785240956336259259.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-03 18:00:47,479 e3be401d MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e3be401d/pkb.log>
2018-05-03 18:00:47,479 e3be401d MainThread INFO PerfKitBenchmarker version: v1.12.0-567-g59dbd5d
2018-05-03 18:00:47,480 e3be401d MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-03 18:00:47,977 e3be401d MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-03 18:00:48,134 e3be401d MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-03 18:00:48,137 e3be401d MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-03 18:00:48,138 e3be401d MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-03 18:00:53,456 e3be401d MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e3be401d/beam/gradlew> clean install -xcheck --stacktrace -DintegrationTestRunner=dataflow
2018-05-03 18:05:30,260 e3be401d MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-03 18:05:30,270 e3be401d MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e3be401d/beam/sdks/python/target>
2018-05-03 18:05:30,271 e3be401d MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-03 18:05:30,273 e3be401d MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 780, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e3be401d/beam/sdks/python/target>
2018-05-03 18:05:30,273 e3be401d MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-03 18:05:30,329 e3be401d MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-03 18:05:30,329 e3be401d MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e3be401d/pkb.log>
2018-05-03 18:05:30,330 e3be401d MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e3be401d/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user shoyer@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1223
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1223/display/redirect?page=changes>
Changes:
[echauchot] [BEAM-4138] Support runners that do not support committed metrics in
------------------------------------------
[...truncated 4.46 KB...]
Using cached https://files.pythonhosted.org/packages/a2/71/8273a7eeed0aff6a854237ab5453bc9aa67deb49df4832801c21f0ff3782/contextlib2-0.5.5-py2.py3-none-any.whl
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3976313595851427305.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9f/ff/68860dec8b90424028ca54df8b9fc8cd1c47ef58710bd5e591ec8c619243/monotonic-1.4-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.4 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7804519026070567558.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-03 12:00:45,724 b284b6be MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/b284b6be/pkb.log>
2018-05-03 12:00:45,724 b284b6be MainThread INFO PerfKitBenchmarker version: v1.12.0-567-g59dbd5d
2018-05-03 12:00:45,725 b284b6be MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-03 12:00:46,110 b284b6be MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-03 12:00:46,274 b284b6be MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-03 12:00:46,291 b284b6be MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-03 12:00:46,292 b284b6be MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-03 12:00:51,514 b284b6be MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/b284b6be/beam/gradlew> clean install -xcheck --stacktrace -DintegrationTestRunner=dataflow
2018-05-03 12:05:13,777 b284b6be MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-03 12:05:13,785 b284b6be MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/b284b6be/beam/sdks/python/target>
2018-05-03 12:05:13,786 b284b6be MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-03 12:05:13,788 b284b6be MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 780, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/b284b6be/beam/sdks/python/target>
2018-05-03 12:05:13,788 b284b6be MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-03 12:05:13,839 b284b6be MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-03 12:05:13,840 b284b6be MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/b284b6be/pkb.log>
2018-05-03 12:05:13,840 b284b6be MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/b284b6be/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user shoyer@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1222
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1222/display/redirect?page=changes>
Changes:
[tgroh] Add KV and Window Coder Utilities to ModelCoders
[tgroh] Use ModelCoders in ProcessBundleDescriptors
[sidhom] Remove StateRequestHandler from JobBundleFactory
------------------------------------------
[...truncated 4.45 KB...]
Using cached https://files.pythonhosted.org/packages/a2/71/8273a7eeed0aff6a854237ab5453bc9aa67deb49df4832801c21f0ff3782/contextlib2-0.5.5-py2.py3-none-any.whl
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins513585465902491747.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9f/ff/68860dec8b90424028ca54df8b9fc8cd1c47ef58710bd5e591ec8c619243/monotonic-1.4-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.4 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8807632777923104948.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-03 06:01:29,557 4cad8c05 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/4cad8c05/pkb.log>
2018-05-03 06:01:29,558 4cad8c05 MainThread INFO PerfKitBenchmarker version: v1.12.0-567-g59dbd5d
2018-05-03 06:01:29,558 4cad8c05 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-03 06:01:29,886 4cad8c05 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-03 06:01:30,037 4cad8c05 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-03 06:01:30,040 4cad8c05 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-03 06:01:30,041 4cad8c05 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-03 06:01:35,386 4cad8c05 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/4cad8c05/beam/gradlew> clean install -xcheck --stacktrace -DintegrationTestRunner=dataflow
2018-05-03 06:06:07,692 4cad8c05 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-03 06:06:07,702 4cad8c05 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/4cad8c05/beam/sdks/python/target>
2018-05-03 06:06:07,703 4cad8c05 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-03 06:06:07,704 4cad8c05 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 780, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/4cad8c05/beam/sdks/python/target>
2018-05-03 06:06:07,705 4cad8c05 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-03 06:06:07,763 4cad8c05 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-03 06:06:07,764 4cad8c05 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/4cad8c05/pkb.log>
2018-05-03 06:06:07,764 4cad8c05 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/4cad8c05/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user shoyer@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1221
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1221/display/redirect?page=changes>
Changes:
[axelmagn] Create ArtifactSource Interface
[tgroh] Remove unused Test Utilities, Test
[axelmagn] Create RemoteBundle interface
[axelmagn] Create JobInfo for job declaration by operators.
[mairbek] Wait for input collection
[mairbek] Expose DatabaseAdminClient
[mairbek] Consistently use lower case table name and column names
[axelmagn] Create BundleFactory chain.
[mairbek] Addressed comments
------------------------------------------
[...truncated 4.42 KB...]
Using cached https://files.pythonhosted.org/packages/a2/71/8273a7eeed0aff6a854237ab5453bc9aa67deb49df4832801c21f0ff3782/contextlib2-0.5.5-py2.py3-none-any.whl
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4772389610378115721.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9f/ff/68860dec8b90424028ca54df8b9fc8cd1c47ef58710bd5e591ec8c619243/monotonic-1.4-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.4 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7671905243203665238.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-03 00:13:24,711 1cb54242 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/1cb54242/pkb.log>
2018-05-03 00:13:24,711 1cb54242 MainThread INFO PerfKitBenchmarker version: v1.12.0-550-g5a2f0ec
2018-05-03 00:13:24,712 1cb54242 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-03 00:13:25,073 1cb54242 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-03 00:13:25,228 1cb54242 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-03 00:13:25,232 1cb54242 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-03 00:13:25,232 1cb54242 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-03 00:13:30,023 1cb54242 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/1cb54242/beam/gradlew> clean install -xcheck --stacktrace -DintegrationTestRunner=dataflow
2018-05-03 00:17:59,583 1cb54242 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-03 00:17:59,594 1cb54242 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/1cb54242/beam/sdks/python/target>
2018-05-03 00:17:59,595 1cb54242 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-03 00:17:59,597 1cb54242 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 780, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/1cb54242/beam/sdks/python/target>
2018-05-03 00:17:59,597 1cb54242 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-03 00:17:59,611 1cb54242 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-03 00:17:59,612 1cb54242 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/1cb54242/pkb.log>
2018-05-03 00:17:59,612 1cb54242 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/1cb54242/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user shoyer@google.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user mairbek@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1220
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1220/display/redirect?page=changes>
Changes:
[github] Count compressed records with a long to avoid overflow
[alan] [BEAM-4218] Fix failing javadoc build
[github] Fix DynamicDestinations documentation
------------------------------------------
[...truncated 4.42 KB...]
Using cached https://files.pythonhosted.org/packages/a2/71/8273a7eeed0aff6a854237ab5453bc9aa67deb49df4832801c21f0ff3782/contextlib2-0.5.5-py2.py3-none-any.whl
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins998352039491693429.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9f/ff/68860dec8b90424028ca54df8b9fc8cd1c47ef58710bd5e591ec8c619243/monotonic-1.4-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.4 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4098664051267110337.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-02 18:00:40,051 64882461 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/64882461/pkb.log>
2018-05-02 18:00:40,051 64882461 MainThread INFO PerfKitBenchmarker version: v1.12.0-537-gb9fbad5
2018-05-02 18:00:40,052 64882461 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-02 18:00:40,341 64882461 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-02 18:00:40,498 64882461 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-02 18:00:40,501 64882461 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-02 18:00:40,502 64882461 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-02 18:00:45,506 64882461 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/64882461/beam/gradlew> clean install -xcheck --stacktrace -DintegrationTestRunner=dataflow
2018-05-02 18:05:08,594 64882461 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-02 18:05:08,604 64882461 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/64882461/beam/sdks/python/target>
2018-05-02 18:05:08,605 64882461 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-02 18:05:08,607 64882461 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 780, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/64882461/beam/sdks/python/target>
2018-05-02 18:05:08,608 64882461 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-02 18:05:08,665 64882461 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-02 18:05:08,666 64882461 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/64882461/pkb.log>
2018-05-02 18:05:08,666 64882461 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/64882461/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user shoyer@google.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user mairbek@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1219
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1219/display/redirect>
------------------------------------------
[...truncated 4.47 KB...]
Using cached https://files.pythonhosted.org/packages/a2/71/8273a7eeed0aff6a854237ab5453bc9aa67deb49df4832801c21f0ff3782/contextlib2-0.5.5-py2.py3-none-any.whl
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins39168637975386711.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9f/ff/68860dec8b90424028ca54df8b9fc8cd1c47ef58710bd5e591ec8c619243/monotonic-1.4-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.4 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8709922746120296298.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-02 12:00:35,427 1dccc1c9 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/1dccc1c9/pkb.log>
2018-05-02 12:00:35,428 1dccc1c9 MainThread INFO PerfKitBenchmarker version: v1.12.0-536-g43c0135
2018-05-02 12:00:35,428 1dccc1c9 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-02 12:00:35,653 1dccc1c9 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-02 12:00:35,809 1dccc1c9 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-02 12:00:35,812 1dccc1c9 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-02 12:00:35,812 1dccc1c9 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-02 12:00:40,756 1dccc1c9 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/1dccc1c9/beam/gradlew> clean install -xcheck --stacktrace -DintegrationTestRunner=dataflow
2018-05-02 12:04:30,243 1dccc1c9 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-02 12:04:30,252 1dccc1c9 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/1dccc1c9/beam/sdks/python/target>
2018-05-02 12:04:30,253 1dccc1c9 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-02 12:04:30,255 1dccc1c9 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 780, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/1dccc1c9/beam/sdks/python/target>
2018-05-02 12:04:30,255 1dccc1c9 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-02 12:04:30,299 1dccc1c9 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-02 12:04:30,300 1dccc1c9 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/1dccc1c9/pkb.log>
2018-05-02 12:04:30,300 1dccc1c9 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/1dccc1c9/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user shoyer@google.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user mairbek@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1218
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1218/display/redirect?page=changes>
Changes:
[github] Update perf.go: only trigger on processing bundles
------------------------------------------
[...truncated 4.47 KB...]
Using cached https://files.pythonhosted.org/packages/a2/71/8273a7eeed0aff6a854237ab5453bc9aa67deb49df4832801c21f0ff3782/contextlib2-0.5.5-py2.py3-none-any.whl
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5866105267809553540.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9f/ff/68860dec8b90424028ca54df8b9fc8cd1c47ef58710bd5e591ec8c619243/monotonic-1.4-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.4 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6644645470118232010.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-02 06:00:56,861 23dbc788 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/23dbc788/pkb.log>
2018-05-02 06:00:56,862 23dbc788 MainThread INFO PerfKitBenchmarker version: v1.12.0-536-g43c0135
2018-05-02 06:00:56,863 23dbc788 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-02 06:00:57,260 23dbc788 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-02 06:00:57,414 23dbc788 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-02 06:00:57,418 23dbc788 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-02 06:00:57,418 23dbc788 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-02 06:01:02,501 23dbc788 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/23dbc788/beam/gradlew> clean install -xcheck --stacktrace -DintegrationTestRunner=dataflow
2018-05-02 06:05:34,626 23dbc788 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-02 06:05:34,635 23dbc788 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/23dbc788/beam/sdks/python/target>
2018-05-02 06:05:34,636 23dbc788 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-02 06:05:34,637 23dbc788 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 780, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/23dbc788/beam/sdks/python/target>
2018-05-02 06:05:34,638 23dbc788 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-02 06:05:34,696 23dbc788 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-02 06:05:34,697 23dbc788 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/23dbc788/pkb.log>
2018-05-02 06:05:34,697 23dbc788 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/23dbc788/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user shoyer@google.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user mairbek@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1217
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1217/display/redirect?page=changes>
Changes:
[swegner] Findbugs cleanup; no functional changes
[github] [BEAM-3042] Refactor of TransformIOCounters (performance, inheritance).
[kirpichov] [BEAM-4144] Fixes SplittableParDoProcessFnTest
------------------------------------------
[...truncated 4.45 KB...]
Using cached https://files.pythonhosted.org/packages/a2/71/8273a7eeed0aff6a854237ab5453bc9aa67deb49df4832801c21f0ff3782/contextlib2-0.5.5-py2.py3-none-any.whl
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5533562723208021089.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9f/ff/68860dec8b90424028ca54df8b9fc8cd1c47ef58710bd5e591ec8c619243/monotonic-1.4-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.4 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1949013470430979385.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-02 00:06:43,078 a99e8695 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/a99e8695/pkb.log>
2018-05-02 00:06:43,079 a99e8695 MainThread INFO PerfKitBenchmarker version: v1.12.0-536-g43c0135
2018-05-02 00:06:43,080 a99e8695 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-02 00:06:43,444 a99e8695 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-02 00:06:43,604 a99e8695 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-02 00:06:43,643 a99e8695 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-02 00:06:43,644 a99e8695 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-02 00:06:48,571 a99e8695 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/a99e8695/beam/gradlew> clean install -xcheck --stacktrace -DintegrationTestRunner=dataflow
2018-05-02 00:10:26,494 a99e8695 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-02 00:10:26,501 a99e8695 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/a99e8695/beam/sdks/python/target>
2018-05-02 00:10:26,502 a99e8695 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-02 00:10:26,504 a99e8695 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 780, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/a99e8695/beam/sdks/python/target>
2018-05-02 00:10:26,504 a99e8695 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-02 00:10:26,533 a99e8695 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-02 00:10:26,534 a99e8695 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/a99e8695/pkb.log>
2018-05-02 00:10:26,534 a99e8695 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/a99e8695/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user shoyer@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Build failed in Jenkins: beam_PerformanceTests_Python #1216
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1216/display/redirect?page=changes>
Changes:
[tgroh] Mark some ReduceFnRunner arguments Nullable
[Pablo] Add documentaton for quickstart tasks
------------------------------------------
[...truncated 4.42 KB...]
Using cached https://files.pythonhosted.org/packages/a2/71/8273a7eeed0aff6a854237ab5453bc9aa67deb49df4832801c21f0ff3782/contextlib2-0.5.5-py2.py3-none-any.whl
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3393256317851533590.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9f/ff/68860dec8b90424028ca54df8b9fc8cd1c47ef58710bd5e591ec8c619243/monotonic-1.4-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.4 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6332065632938212553.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-01 18:08:40,795 8aa06d8f MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/8aa06d8f/pkb.log>
2018-05-01 18:08:40,795 8aa06d8f MainThread INFO PerfKitBenchmarker version: v1.12.0-535-g5be59b8
2018-05-01 18:08:40,796 8aa06d8f MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-01 18:08:41,024 8aa06d8f MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-01 18:08:41,207 8aa06d8f MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-01 18:08:41,211 8aa06d8f MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-01 18:08:41,211 8aa06d8f MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-01 18:08:46,299 8aa06d8f MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-05-01 18:14:45,446 8aa06d8f MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-01 18:14:45,467 8aa06d8f MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/8aa06d8f/beam/sdks/python/target/apache-beam-2.5.0.dev0.tar.gz>
2018-05-01 18:14:45,701 8aa06d8f MainThread beam_integration_benchmark(1/1) INFO Ran: {python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/8aa06d8f/beam/sdks/python/target/apache-beam-2.5.0.dev0.tar.gz}> ReturnCode:1
STDOUT:
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:397: UserWarning: Normalizing '2.5.0.dev' to '2.5.0.dev0'
normalized_version,
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: invalid command 'nosetests'
2018-05-01 18:14:45,702 8aa06d8f MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-05-01 18:14:45,704 8aa06d8f MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-01 18:14:45,706 8aa06d8f MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 780, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-05-01 18:14:45,707 8aa06d8f MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-01 18:14:45,728 8aa06d8f MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-01 18:14:45,729 8aa06d8f MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/8aa06d8f/pkb.log>
2018-05-01 18:14:45,729 8aa06d8f MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/8aa06d8f/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user shoyer@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Build failed in Jenkins: beam_PerformanceTests_Python #1215
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1215/display/redirect>
------------------------------------------
[...truncated 4.44 KB...]
Using cached https://files.pythonhosted.org/packages/a2/71/8273a7eeed0aff6a854237ab5453bc9aa67deb49df4832801c21f0ff3782/contextlib2-0.5.5-py2.py3-none-any.whl
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4799797187760619386.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9f/ff/68860dec8b90424028ca54df8b9fc8cd1c47ef58710bd5e591ec8c619243/monotonic-1.4-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.4 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1493983922878122074.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-01 12:00:50,244 d6149ea5 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/d6149ea5/pkb.log>
2018-05-01 12:00:50,245 d6149ea5 MainThread INFO PerfKitBenchmarker version: v1.12.0-534-gb60e653
2018-05-01 12:00:50,246 d6149ea5 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-01 12:00:50,501 d6149ea5 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-01 12:00:50,659 d6149ea5 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-01 12:00:50,662 d6149ea5 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-01 12:00:50,663 d6149ea5 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-01 12:00:56,299 d6149ea5 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-05-01 12:06:57,039 d6149ea5 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-01 12:06:57,062 d6149ea5 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/d6149ea5/beam/sdks/python/target/apache-beam-2.5.0.dev0.tar.gz>
2018-05-01 12:06:57,315 d6149ea5 MainThread beam_integration_benchmark(1/1) INFO Ran: {python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/d6149ea5/beam/sdks/python/target/apache-beam-2.5.0.dev0.tar.gz}> ReturnCode:1
STDOUT:
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:397: UserWarning: Normalizing '2.5.0.dev' to '2.5.0.dev0'
normalized_version,
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: invalid command 'nosetests'
2018-05-01 12:06:57,316 d6149ea5 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-05-01 12:06:57,317 d6149ea5 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-01 12:06:57,318 d6149ea5 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 780, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-05-01 12:06:57,319 d6149ea5 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-01 12:06:57,368 d6149ea5 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-01 12:06:57,369 d6149ea5 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/d6149ea5/pkb.log>
2018-05-01 12:06:57,369 d6149ea5 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/d6149ea5/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user shoyer@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Build failed in Jenkins: beam_PerformanceTests_Python #1214
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1214/display/redirect?page=changes>
Changes:
[thw] [BEAM-4131] Include SDK into Python SDK harness container.
[thw] cleanup
------------------------------------------
[...truncated 4.44 KB...]
Using cached https://files.pythonhosted.org/packages/a2/71/8273a7eeed0aff6a854237ab5453bc9aa67deb49df4832801c21f0ff3782/contextlib2-0.5.5-py2.py3-none-any.whl
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7273163152504492280.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9f/ff/68860dec8b90424028ca54df8b9fc8cd1c47ef58710bd5e591ec8c619243/monotonic-1.4-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.4 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4175359635929884124.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-01 06:00:52,985 3b56a02d MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/3b56a02d/pkb.log>
2018-05-01 06:00:52,986 3b56a02d MainThread INFO PerfKitBenchmarker version: v1.12.0-534-gb60e653
2018-05-01 06:00:52,986 3b56a02d MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-01 06:00:53,341 3b56a02d MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-01 06:00:53,606 3b56a02d MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-01 06:00:53,611 3b56a02d MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-01 06:00:53,612 3b56a02d MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-01 06:00:59,644 3b56a02d MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-05-01 06:07:11,308 3b56a02d MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-01 06:07:11,325 3b56a02d MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/3b56a02d/beam/sdks/python/target/apache-beam-2.5.0.dev0.tar.gz>
2018-05-01 06:07:11,584 3b56a02d MainThread beam_integration_benchmark(1/1) INFO Ran: {python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/3b56a02d/beam/sdks/python/target/apache-beam-2.5.0.dev0.tar.gz}> ReturnCode:1
STDOUT:
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:397: UserWarning: Normalizing '2.5.0.dev' to '2.5.0.dev0'
normalized_version,
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: invalid command 'nosetests'
2018-05-01 06:07:11,585 3b56a02d MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-05-01 06:07:11,586 3b56a02d MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-01 06:07:11,587 3b56a02d MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 780, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-05-01 06:07:11,588 3b56a02d MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-01 06:07:11,629 3b56a02d MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-01 06:07:11,630 3b56a02d MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/3b56a02d/pkb.log>
2018-05-01 06:07:11,631 3b56a02d MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/3b56a02d/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user shoyer@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Build failed in Jenkins: beam_PerformanceTests_Python #1213
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1213/display/redirect?page=changes>
Changes:
[herohde] [BEAM-4175] Fix direct ouput pardo issue
[kenn] Add errorprone to basic Gradle config
[kenn] Fix suppressed failure in ResourceIdTester
[kenn] Fix erroneous tests in ViewTest
[kenn] Add missing @Test in CollectionCoderTest
[kenn] Sickbay broken LocalResourceIdTest case
[kenn] Remove inaccurate GuardedBy in both copies of DirectMetrics
[kenn] Fix suppressed failure in RetryHttpRequestInitializerTest
[kenn] Remove unnecessary type arguments in ReduceFnRunnerTest
[kenn] Add missing @Test in SplittableParDoProcessFnTest
[kenn] Remove unnecessary type argument in PipelineTranslationTest
[kenn] Fix compile-type constant int overflow in S3FileSystem
[kenn] Disable findbugs for JdbcIO where it gave false positives, now that we
[kenn] Sickbay broken GcsResourceIdTest case
[kenn] Sickbay broken HadoopResourceIdTest case
[kenn] Sickbay broken SplittableParDoProcessFnTest case
[kenn] Add missing @Test in S3ResourceIdTest
[kenn] Fix typo in CassandraIOIT
[kenn] Fix nonsensical @RunWith in ExampleEchoPipelineTest
[kenn] Fix missing @Test in both copies of WatermarkManagerTest
[kenn] Fix misuse of Arrays.asList in KinesisMockWriteTest
[kenn] Remove extraneous type variables in MongoDBGridFSIOTest
[kenn] Sickbay broken S3ResourceIdTest cases
[kenn] Sickbay broken WatermarkManagerTest case
[apilloud] [BEAM-3983][SQL] Add BigQuery table provider
[iemejia] Fix gradle script to build the docker image of 'sdks/java/container'
[Pablo] Creation of utils.py whit CountingSource class on it.
[Pablo] Changed import from examples.snippets to io.utils
------------------------------------------
[...truncated 4.43 KB...]
Using cached https://files.pythonhosted.org/packages/a2/71/8273a7eeed0aff6a854237ab5453bc9aa67deb49df4832801c21f0ff3782/contextlib2-0.5.5-py2.py3-none-any.whl
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2017567295102981354.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9f/ff/68860dec8b90424028ca54df8b9fc8cd1c47ef58710bd5e591ec8c619243/monotonic-1.4-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.4 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3564933175583089852.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-01 00:00:42,979 5e1e8535 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/5e1e8535/pkb.log>
2018-05-01 00:00:42,979 5e1e8535 MainThread INFO PerfKitBenchmarker version: v1.12.0-534-gb60e653
2018-05-01 00:00:42,980 5e1e8535 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-01 00:00:43,326 5e1e8535 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-01 00:00:43,479 5e1e8535 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-01 00:00:43,483 5e1e8535 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-01 00:00:43,483 5e1e8535 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-01 00:00:48,973 5e1e8535 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-05-01 00:07:01,540 5e1e8535 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-01 00:07:01,558 5e1e8535 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/5e1e8535/beam/sdks/python/target/apache-beam-2.5.0.dev0.tar.gz>
2018-05-01 00:07:01,812 5e1e8535 MainThread beam_integration_benchmark(1/1) INFO Ran: {python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/5e1e8535/beam/sdks/python/target/apache-beam-2.5.0.dev0.tar.gz}> ReturnCode:1
STDOUT:
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:397: UserWarning: Normalizing '2.5.0.dev' to '2.5.0.dev0'
normalized_version,
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: invalid command 'nosetests'
2018-05-01 00:07:01,813 5e1e8535 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-05-01 00:07:01,814 5e1e8535 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-01 00:07:01,816 5e1e8535 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 780, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-05-01 00:07:01,816 5e1e8535 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-01 00:07:01,874 5e1e8535 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-01 00:07:01,875 5e1e8535 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/5e1e8535/pkb.log>
2018-05-01 00:07:01,875 5e1e8535 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/5e1e8535/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user shoyer@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Build failed in Jenkins: beam_PerformanceTests_Python #1212
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1212/display/redirect?page=changes>
Changes:
[ankurgoenka] fixing generic errors
[echauchot] [BEAM-3119] ensure metrics thread is shutdown
[echauchot] [BEAM-4088] Rebase on master, fix conflicts and adapt test to master
[echauchot] [BEAM-4088] Add a test with a real pipeline that has a DoFn with metrics
[echauchot] [BEAM-4088] Blocks until the threads in metricsExecutorService is done,
[pgerver] Enhance awsCredentialsProvider option description
------------------------------------------
[...truncated 4.44 KB...]
Using cached https://files.pythonhosted.org/packages/a2/71/8273a7eeed0aff6a854237ab5453bc9aa67deb49df4832801c21f0ff3782/contextlib2-0.5.5-py2.py3-none-any.whl
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4920621408066048505.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9f/ff/68860dec8b90424028ca54df8b9fc8cd1c47ef58710bd5e591ec8c619243/monotonic-1.4-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.4 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins9167633024371039651.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-04-30 18:01:49,703 8cdf7846 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/8cdf7846/pkb.log>
2018-04-30 18:01:49,703 8cdf7846 MainThread INFO PerfKitBenchmarker version: v1.12.0-533-g60f6470
2018-04-30 18:01:49,704 8cdf7846 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-04-30 18:01:50,029 8cdf7846 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-04-30 18:01:50,196 8cdf7846 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-04-30 18:01:50,199 8cdf7846 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-04-30 18:01:50,200 8cdf7846 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-04-30 18:01:56,399 8cdf7846 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-04-30 18:08:33,304 8cdf7846 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-04-30 18:08:33,320 8cdf7846 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/8cdf7846/beam/sdks/python/target/apache-beam-2.5.0.dev0.tar.gz>
2018-04-30 18:08:33,548 8cdf7846 MainThread beam_integration_benchmark(1/1) INFO Ran: {python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/8cdf7846/beam/sdks/python/target/apache-beam-2.5.0.dev0.tar.gz}> ReturnCode:1
STDOUT:
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:397: UserWarning: Normalizing '2.5.0.dev' to '2.5.0.dev0'
normalized_version,
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: invalid command 'nosetests'
2018-04-30 18:08:33,549 8cdf7846 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-04-30 18:08:33,550 8cdf7846 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-04-30 18:08:33,552 8cdf7846 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 780, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-04-30 18:08:33,552 8cdf7846 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-04-30 18:08:33,591 8cdf7846 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-04-30 18:08:33,592 8cdf7846 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/8cdf7846/pkb.log>
2018-04-30 18:08:33,592 8cdf7846 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/8cdf7846/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user shoyer@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Build failed in Jenkins: beam_PerformanceTests_Python #1211
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1211/display/redirect?page=changes>
Changes:
[aljoscha.krettek] [BEAM-3909] Add tests for Flink DoFnOperator side-input checkpointing
------------------------------------------
[...truncated 4.32 KB...]
Collecting contextlib2>=0.5.1 (from -r PerfKitBenchmarker/requirements.txt (line 24))
Using cached https://files.pythonhosted.org/packages/a2/71/8273a7eeed0aff6a854237ab5453bc9aa67deb49df4832801c21f0ff3782/contextlib2-0.5.5-py2.py3-none-any.whl
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2645658287835764866.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9f/ff/68860dec8b90424028ca54df8b9fc8cd1c47ef58710bd5e591ec8c619243/monotonic-1.4-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.4 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1348428526960774711.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-04-30 12:01:36,892 176e9506 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/176e9506/pkb.log>
2018-04-30 12:01:36,893 176e9506 MainThread INFO PerfKitBenchmarker version: v1.12.0-533-g60f6470
2018-04-30 12:01:36,894 176e9506 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-04-30 12:01:37,253 176e9506 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-04-30 12:01:37,395 176e9506 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-04-30 12:01:37,398 176e9506 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-04-30 12:01:37,399 176e9506 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-04-30 12:01:42,327 176e9506 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-04-30 12:07:47,478 176e9506 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-04-30 12:07:47,557 176e9506 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/176e9506/beam/sdks/python/target/apache-beam-2.5.0.dev0.tar.gz>
2018-04-30 12:07:47,804 176e9506 MainThread beam_integration_benchmark(1/1) INFO Ran: {python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/176e9506/beam/sdks/python/target/apache-beam-2.5.0.dev0.tar.gz}> ReturnCode:1
STDOUT:
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:397: UserWarning: Normalizing '2.5.0.dev' to '2.5.0.dev0'
normalized_version,
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: invalid command 'nosetests'
2018-04-30 12:07:47,804 176e9506 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-04-30 12:07:47,805 176e9506 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-04-30 12:07:47,807 176e9506 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 780, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-04-30 12:07:47,807 176e9506 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-04-30 12:07:47,830 176e9506 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-04-30 12:07:47,830 176e9506 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/176e9506/pkb.log>
2018-04-30 12:07:47,830 176e9506 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/176e9506/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user shoyer@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Build failed in Jenkins: beam_PerformanceTests_Python #1210
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1210/display/redirect?page=changes>
Changes:
[chamikara] [BEAM-3973] Adds a parameter to the Cloud Spanner read connector that
------------------------------------------
[...truncated 4.33 KB...]
Collecting contextlib2>=0.5.1 (from -r PerfKitBenchmarker/requirements.txt (line 24))
Using cached https://files.pythonhosted.org/packages/a2/71/8273a7eeed0aff6a854237ab5453bc9aa67deb49df4832801c21f0ff3782/contextlib2-0.5.5-py2.py3-none-any.whl
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1598592255176044864.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9f/ff/68860dec8b90424028ca54df8b9fc8cd1c47ef58710bd5e591ec8c619243/monotonic-1.4-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.4 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6373523809561692719.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-04-30 06:00:38,958 f947e7a5 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/f947e7a5/pkb.log>
2018-04-30 06:00:38,959 f947e7a5 MainThread INFO PerfKitBenchmarker version: v1.12.0-533-g60f6470
2018-04-30 06:00:38,959 f947e7a5 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-04-30 06:00:39,272 f947e7a5 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-04-30 06:00:39,425 f947e7a5 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-04-30 06:00:39,428 f947e7a5 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-04-30 06:00:39,429 f947e7a5 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-04-30 06:00:44,339 f947e7a5 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-04-30 06:06:48,997 f947e7a5 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-04-30 06:06:49,014 f947e7a5 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/f947e7a5/beam/sdks/python/target/apache-beam-2.5.0.dev0.tar.gz>
2018-04-30 06:06:49,253 f947e7a5 MainThread beam_integration_benchmark(1/1) INFO Ran: {python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/f947e7a5/beam/sdks/python/target/apache-beam-2.5.0.dev0.tar.gz}> ReturnCode:1
STDOUT:
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:397: UserWarning: Normalizing '2.5.0.dev' to '2.5.0.dev0'
normalized_version,
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: invalid command 'nosetests'
2018-04-30 06:06:49,253 f947e7a5 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-04-30 06:06:49,254 f947e7a5 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-04-30 06:06:49,256 f947e7a5 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 780, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-04-30 06:06:49,256 f947e7a5 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-04-30 06:06:49,312 f947e7a5 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-04-30 06:06:49,312 f947e7a5 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/f947e7a5/pkb.log>
2018-04-30 06:06:49,312 f947e7a5 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/f947e7a5/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user shoyer@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Build failed in Jenkins: beam_PerformanceTests_Python #1209
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1209/display/redirect>
------------------------------------------
[...truncated 4.28 KB...]
Collecting contextlib2>=0.5.1 (from -r PerfKitBenchmarker/requirements.txt (line 24))
Using cached https://files.pythonhosted.org/packages/a2/71/8273a7eeed0aff6a854237ab5453bc9aa67deb49df4832801c21f0ff3782/contextlib2-0.5.5-py2.py3-none-any.whl
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8861034256233226150.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9f/ff/68860dec8b90424028ca54df8b9fc8cd1c47ef58710bd5e591ec8c619243/monotonic-1.4-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.4 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7321882618682103576.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-04-30 00:00:49,141 f919251a MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/f919251a/pkb.log>
2018-04-30 00:00:49,142 f919251a MainThread INFO PerfKitBenchmarker version: v1.12.0-533-g60f6470
2018-04-30 00:00:49,142 f919251a MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-04-30 00:00:49,310 f919251a MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-04-30 00:00:49,460 f919251a MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-04-30 00:00:49,464 f919251a MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-04-30 00:00:49,464 f919251a MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-04-30 00:00:55,132 f919251a MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-04-30 00:06:49,467 f919251a MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-04-30 00:06:49,492 f919251a MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/f919251a/beam/sdks/python/target/apache-beam-2.5.0.dev0.tar.gz>
2018-04-30 00:06:49,730 f919251a MainThread beam_integration_benchmark(1/1) INFO Ran: {python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/f919251a/beam/sdks/python/target/apache-beam-2.5.0.dev0.tar.gz}> ReturnCode:1
STDOUT:
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:397: UserWarning: Normalizing '2.5.0.dev' to '2.5.0.dev0'
normalized_version,
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: invalid command 'nosetests'
2018-04-30 00:06:49,731 f919251a MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-04-30 00:06:49,732 f919251a MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-04-30 00:06:49,734 f919251a MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 780, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-04-30 00:06:49,735 f919251a MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-04-30 00:06:49,753 f919251a MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-04-30 00:06:49,754 f919251a MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/f919251a/pkb.log>
2018-04-30 00:06:49,754 f919251a MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/f919251a/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user shoyer@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Build failed in Jenkins: beam_PerformanceTests_Python #1208
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1208/display/redirect>
------------------------------------------
[...truncated 4.28 KB...]
Collecting contextlib2>=0.5.1 (from -r PerfKitBenchmarker/requirements.txt (line 24))
Using cached https://files.pythonhosted.org/packages/a2/71/8273a7eeed0aff6a854237ab5453bc9aa67deb49df4832801c21f0ff3782/contextlib2-0.5.5-py2.py3-none-any.whl
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7138293126766063020.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9f/ff/68860dec8b90424028ca54df8b9fc8cd1c47ef58710bd5e591ec8c619243/monotonic-1.4-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.4 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins624080849028614251.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-04-29 18:00:35,491 aef9c1ec MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/aef9c1ec/pkb.log>
2018-04-29 18:00:35,491 aef9c1ec MainThread INFO PerfKitBenchmarker version: v1.12.0-533-g60f6470
2018-04-29 18:00:35,492 aef9c1ec MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-04-29 18:00:35,967 aef9c1ec MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-04-29 18:00:36,113 aef9c1ec MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-04-29 18:00:36,117 aef9c1ec MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-04-29 18:00:36,117 aef9c1ec MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-04-29 18:00:41,310 aef9c1ec MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-04-29 18:06:45,852 aef9c1ec MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-04-29 18:06:45,870 aef9c1ec MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/aef9c1ec/beam/sdks/python/target/apache-beam-2.5.0.dev0.tar.gz>
2018-04-29 18:06:46,099 aef9c1ec MainThread beam_integration_benchmark(1/1) INFO Ran: {python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/aef9c1ec/beam/sdks/python/target/apache-beam-2.5.0.dev0.tar.gz}> ReturnCode:1
STDOUT:
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:397: UserWarning: Normalizing '2.5.0.dev' to '2.5.0.dev0'
normalized_version,
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: invalid command 'nosetests'
2018-04-29 18:06:46,099 aef9c1ec MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-04-29 18:06:46,100 aef9c1ec MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-04-29 18:06:46,102 aef9c1ec MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 780, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-04-29 18:06:46,102 aef9c1ec MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-04-29 18:06:46,126 aef9c1ec MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-04-29 18:06:46,127 aef9c1ec MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/aef9c1ec/pkb.log>
2018-04-29 18:06:46,127 aef9c1ec MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/aef9c1ec/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user shoyer@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com