You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2018/04/21 12:07:09 UTC
Build failed in Jenkins: beam_PerformanceTests_Python #1175
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1175/display/redirect>
------------------------------------------
[...truncated 617.58 KB...]
[WARNING] - org.apache.http.annotation.NotThreadSafe
[WARNING] - org.apache.http.annotation.Immutable
[WARNING] - org.apache.http.annotation.GuardedBy
[WARNING] - org.apache.http.annotation.ThreadSafe
[WARNING] maven-shade-plugin has detected that some class files are
[WARNING] present in two or more JARs. When this happens, only one
[WARNING] single version of the class is copied to the uber jar.
[WARNING] Usually this is not harmful and you can skip these warnings,
[WARNING] otherwise try to manually exclude artifacts based on
[WARNING] mvn dependency:tree -Ddetail=true and the above output.
[WARNING] See http://maven.apache.org/plugins/maven-shade-plugin/
[INFO] Replacing <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/b6ddb5ad/beam/sdks/java/nexmark/target/beam-sdks-java-nexmark-bundled-2.5.0-SNAPSHOT.jar> with <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/b6ddb5ad/beam/sdks/java/nexmark/target/beam-sdks-java-nexmark-2.5.0-SNAPSHOT-shaded.jar>
[INFO]
[INFO] --- maven-dependency-plugin:3.0.2:analyze-only (default) @ beam-sdks-java-nexmark ---
[INFO] No dependency problems found
[INFO]
[INFO] --- maven-install-plugin:2.5.2:install (default-install) @ beam-sdks-java-nexmark ---
[INFO] Installing <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/b6ddb5ad/beam/sdks/java/nexmark/target/beam-sdks-java-nexmark-2.5.0-SNAPSHOT.jar> to /home/jenkins/.m2/repository/org/apache/beam/beam-sdks-java-nexmark/2.5.0-SNAPSHOT/beam-sdks-java-nexmark-2.5.0-SNAPSHOT.jar
[INFO] Installing <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/b6ddb5ad/beam/sdks/java/nexmark/dependency-reduced-pom.xml> to /home/jenkins/.m2/repository/org/apache/beam/beam-sdks-java-nexmark/2.5.0-SNAPSHOT/beam-sdks-java-nexmark-2.5.0-SNAPSHOT.pom
[INFO] Installing <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/b6ddb5ad/beam/sdks/java/nexmark/target/beam-sdks-java-nexmark-2.5.0-SNAPSHOT-tests.jar> to /home/jenkins/.m2/repository/org/apache/beam/beam-sdks-java-nexmark/2.5.0-SNAPSHOT/beam-sdks-java-nexmark-2.5.0-SNAPSHOT-tests.jar
[INFO] Installing <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/b6ddb5ad/beam/sdks/java/nexmark/target/beam-sdks-java-nexmark-2.5.0-SNAPSHOT-tests.jar> to /home/jenkins/.m2/repository/org/apache/beam/beam-sdks-java-nexmark/2.5.0-SNAPSHOT/beam-sdks-java-nexmark-2.5.0-SNAPSHOT-tests.jar
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building Apache Beam :: SDKs :: Python 2.5.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- groovy-maven-plugin:2.0:execute (find-supported-python-for-clean) @ beam-sdks-python ---
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Beam :: Parent .............................. SUCCESS [ 5.481 s]
[INFO] Apache Beam :: SDKs :: Java :: Build Tools ......... SUCCESS [ 6.327 s]
[INFO] Apache Beam :: Model ............................... SUCCESS [ 0.146 s]
[INFO] Apache Beam :: Model :: Pipeline ................... SUCCESS [ 14.905 s]
[INFO] Apache Beam :: Model :: Job Management ............. SUCCESS [ 6.227 s]
[INFO] Apache Beam :: Model :: Fn Execution ............... SUCCESS [ 7.900 s]
[INFO] Apache Beam :: SDKs ................................ SUCCESS [ 0.266 s]
[INFO] Apache Beam :: SDKs :: Go .......................... SUCCESS [01:01 min]
[INFO] Apache Beam :: SDKs :: Go :: Container ............. SUCCESS [ 20.283 s]
[INFO] Apache Beam :: SDKs :: Java ........................ SUCCESS [ 0.114 s]
[INFO] Apache Beam :: SDKs :: Java :: Core ................ SUCCESS [ 26.363 s]
[INFO] Apache Beam :: SDKs :: Java :: Fn Execution ........ SUCCESS [ 2.723 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions .......... SUCCESS [ 0.090 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Google Cloud Platform Core SUCCESS [ 3.085 s]
[INFO] Apache Beam :: Runners ............................. SUCCESS [ 0.051 s]
[INFO] Apache Beam :: Runners :: Core Construction Java ... SUCCESS [ 3.992 s]
[INFO] Apache Beam :: Runners :: Core Java ................ SUCCESS [ 4.314 s]
[INFO] Apache Beam :: SDKs :: Java :: Harness ............. SUCCESS [ 5.987 s]
[INFO] Apache Beam :: SDKs :: Java :: Container ........... SUCCESS [ 7.575 s]
[INFO] Apache Beam :: SDKs :: Java :: IO .................. SUCCESS [ 0.063 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Amazon Web Services SUCCESS [ 2.706 s]
[INFO] Apache Beam :: Runners :: Local Java Core .......... SUCCESS [ 1.699 s]
[INFO] Apache Beam :: Runners :: Direct Java .............. SUCCESS [ 9.082 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: AMQP .......... SUCCESS [ 2.374 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Common ........ SUCCESS [ 1.628 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Cassandra ..... SUCCESS [ 2.621 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch . SUCCESS [ 1.947 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests SUCCESS [ 0.720 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: Common SUCCESS [ 0.626 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: 2.x SUCCESS [ 1.801 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: 5.x SUCCESS [ 2.410 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: XML ........... SUCCESS [ 1.948 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Protobuf SUCCESS [ 1.751 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Google Cloud Platform SUCCESS [ 5.399 s]
[INFO] Apache Beam :: Runners :: Google Cloud Dataflow .... SUCCESS [ 8.457 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: File-based-io-tests SUCCESS [ 2.620 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop Common . SUCCESS [ 4.563 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop File System SUCCESS [ 3.101 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: JDBC .......... SUCCESS [ 2.951 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop Input Format SUCCESS [ 16.192 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: HBase ......... SUCCESS [ 6.069 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: HCatalog ...... SUCCESS [ 7.881 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: JMS ........... SUCCESS [ 1.943 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Kafka ......... SUCCESS [ 2.316 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Kinesis ....... SUCCESS [ 1.731 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: MongoDB ....... SUCCESS [ 2.620 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: MQTT .......... SUCCESS [ 2.081 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Redis ......... SUCCESS [ 1.722 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Solr .......... SUCCESS [ 4.585 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Tika .......... SUCCESS [ 3.416 s]
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes .... SUCCESS [ 0.061 s]
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Starter SUCCESS [ 6.489 s]
[INFO] Apache Beam :: Examples ............................ SUCCESS [ 0.043 s]
[INFO] Apache Beam :: Examples :: Java .................... SUCCESS [ 2.938 s]
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Examples SUCCESS [ 24.404 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Jackson SUCCESS [ 1.361 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Join library SUCCESS [ 1.492 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Sketching SUCCESS [ 2.542 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Sorter SUCCESS [ 2.123 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: SQL ... SUCCESS [ 15.189 s]
[INFO] Apache Beam :: SDKs :: Java :: Nexmark ............. SUCCESS [ 15.498 s]
[INFO] Apache Beam :: SDKs :: Python ...................... FAILURE [ 5.331 s]
[INFO] Apache Beam :: SDKs :: Python :: Container ......... SKIPPED
[INFO] Apache Beam :: Runners :: Java Fn Execution ........ SKIPPED
[INFO] Apache Beam :: Runners :: Java Local Artifact Service SKIPPED
[INFO] Apache Beam :: Runners :: Reference ................ SKIPPED
[INFO] Apache Beam :: Runners :: Reference :: Java ........ SKIPPED
[INFO] Apache Beam :: Runners :: Reference :: Job Orchestrator SKIPPED
[INFO] Apache Beam :: Runners :: Flink .................... SKIPPED
[INFO] Apache Beam :: Runners :: Gearpump ................. SKIPPED
[INFO] Apache Beam :: Runners :: Spark .................... SKIPPED
[INFO] Apache Beam :: Runners :: Apex ..................... SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform .... SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform :: GCE metadata provisioning SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform :: GCS artifact proxy SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Aggregated Javadoc .. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 06:08 min
[INFO] Finished at: 2018-04-21T12:07:01Z
[INFO] Final Memory: 361M/1996M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.codehaus.gmaven:groovy-maven-plugin:2.0:execute (find-supported-python-for-clean) on project beam-sdks-python: Unable to find pip for Python 2.7 in path -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :beam-sdks-python
STDERR: log4j:WARN No appenders could be found for logger (freemarker.cache).
log4j:WARN Please initialize the log4j system properly.
Warning: Lookahead adequacy checking not being performed since option LOOKAHEAD is more than 1. Set option FORCE_LA_CHECK to true to force checking.
2018-04-21 12:07:02,089 b6ddb5ad MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-04-21 12:07:02,095 b6ddb5ad MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 175, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 265, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 290, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/b6ddb5ad/beam/sdks/python/target>
2018-04-21 12:07:02,096 b6ddb5ad MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-04-21 12:07:02,098 b6ddb5ad MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 780, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 175, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 265, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 290, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/b6ddb5ad/beam/sdks/python/target>
2018-04-21 12:07:02,098 b6ddb5ad MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-04-21 12:07:02,157 b6ddb5ad MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-04-21 12:07:02,157 b6ddb5ad MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/b6ddb5ad/pkb.log>
2018-04-21 12:07:02,157 b6ddb5ad MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/b6ddb5ad/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user shoyer@google.com
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user boyuanz@google.com
Jenkins build is back to normal : beam_PerformanceTests_Python
#1268
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1268/display/redirect?page=changes>
Build failed in Jenkins: beam_PerformanceTests_Python #1267
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1267/display/redirect>
------------------------------------------
[...truncated 100.18 KB...]
2018-08-04 01:09:27,927 0baf5535 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code.
2018-08-04 01:09:46,098 0baf5535 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json>
2018-08-04 01:09:49,974 0baf5535 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json}> ReturnCode:1
STDOUT:
BigQuery error in load operation: Error processing job
'apache-beam-testing:bqjob_r6d1b59a3091e4529_00000165027bb68d_1': Invalid schema
update. Field timestamp has changed type from TIMESTAMP to FLOAT
STDERR: Upload complete.Waiting on bqjob_r6d1b59a3091e4529_00000165027bb68d_1 ... (0s) Current status: RUNNING Waiting on bqjob_r6d1b59a3091e4529_00000165027bb68d_1 ... (0s) Current status: DONE
2018-08-04 01:09:49,975 0baf5535 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code.
2018-08-04 01:10:12,981 0baf5535 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json>
2018-08-04 01:10:16,990 0baf5535 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json}> ReturnCode:1
STDOUT:
BigQuery error in load operation: Error processing job
'apache-beam-testing:bqjob_r4840b2454a8e3494_00000165027c1fc7_1': Invalid schema
update. Field timestamp has changed type from TIMESTAMP to FLOAT
STDERR: Upload complete.Waiting on bqjob_r4840b2454a8e3494_00000165027c1fc7_1 ... (0s) Current status: RUNNING Waiting on bqjob_r4840b2454a8e3494_00000165027c1fc7_1 ... (0s) Current status: DONE
2018-08-04 01:10:16,991 0baf5535 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code.
2018-08-04 01:10:44,182 0baf5535 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json>
2018-08-04 01:10:48,193 0baf5535 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json}> ReturnCode:1
STDOUT:
BigQuery error in load operation: Error processing job
'apache-beam-testing:bqjob_r178283251e23bd57_00000165027c99aa_1': Invalid schema
update. Field timestamp has changed type from TIMESTAMP to FLOAT
STDERR: Upload complete.Waiting on bqjob_r178283251e23bd57_00000165027c99aa_1 ... (0s) Current status: RUNNING Waiting on bqjob_r178283251e23bd57_00000165027c99aa_1 ... (0s) Current status: DONE
2018-08-04 01:10:48,194 0baf5535 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code.
2018-08-04 01:11:09,452 0baf5535 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json>
2018-08-04 01:11:13,329 0baf5535 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json}> ReturnCode:1
STDOUT:
BigQuery error in load operation: Error processing job
'apache-beam-testing:bqjob_r4b3530b4d72d9f35_00000165027cfc3b_1': Invalid schema
update. Field timestamp has changed type from TIMESTAMP to FLOAT
STDERR: Upload complete.Waiting on bqjob_r4b3530b4d72d9f35_00000165027cfc3b_1 ... (0s) Current status: RUNNING Waiting on bqjob_r4b3530b4d72d9f35_00000165027cfc3b_1 ... (0s) Current status: DONE
2018-08-04 01:11:13,330 0baf5535 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code.
2018-08-04 01:11:32,307 0baf5535 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json>
2018-08-04 01:11:36,404 0baf5535 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json}> ReturnCode:1
STDOUT:
BigQuery error in load operation: Error processing job
'apache-beam-testing:bqjob_r2dab7c4da2143862_00000165027d55b3_1': Invalid schema
update. Field timestamp has changed type from TIMESTAMP to FLOAT
STDERR: Upload complete.Waiting on bqjob_r2dab7c4da2143862_00000165027d55b3_1 ... (0s) Current status: RUNNING Waiting on bqjob_r2dab7c4da2143862_00000165027d55b3_1 ... (0s) Current status: DONE
2018-08-04 01:11:36,405 0baf5535 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code.
2018-08-04 01:11:56,492 0baf5535 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json>
2018-08-04 01:12:00,468 0baf5535 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json}> ReturnCode:1
STDOUT:
BigQuery error in load operation: Error processing job
'apache-beam-testing:bqjob_r6afc3f5f01d3da95_00000165027db417_1': Invalid schema
update. Field timestamp has changed type from TIMESTAMP to FLOAT
STDERR: Upload complete.Waiting on bqjob_r6afc3f5f01d3da95_00000165027db417_1 ... (0s) Current status: RUNNING Waiting on bqjob_r6afc3f5f01d3da95_00000165027db417_1 ... (0s) Current status: DONE
2018-08-04 01:12:00,469 0baf5535 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code.
2018-08-04 01:12:28,107 0baf5535 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json>
2018-08-04 01:12:32,126 0baf5535 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json}> ReturnCode:1
STDOUT:
BigQuery error in load operation: Error processing job
'apache-beam-testing:bqjob_r4c35d1eaaa7322ec_00000165027e2fa0_1': Invalid schema
update. Field timestamp has changed type from TIMESTAMP to FLOAT
STDERR: Upload complete.Waiting on bqjob_r4c35d1eaaa7322ec_00000165027e2fa0_1 ... (0s) Current status: RUNNING Waiting on bqjob_r4c35d1eaaa7322ec_00000165027e2fa0_1 ... (0s) Current status: DONE
2018-08-04 01:12:32,127 0baf5535 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code.
2018-08-04 01:12:59,230 0baf5535 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json>
2018-08-04 01:13:03,189 0baf5535 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json}> ReturnCode:1
STDOUT:
BigQuery error in load operation: Error processing job
'apache-beam-testing:bqjob_r56a8555cee86c30e_00000165027ea94b_1': Invalid schema
update. Field timestamp has changed type from TIMESTAMP to FLOAT
STDERR: Upload complete.Waiting on bqjob_r56a8555cee86c30e_00000165027ea94b_1 ... (0s) Current status: RUNNING Waiting on bqjob_r56a8555cee86c30e_00000165027ea94b_1 ... (0s) Current status: DONE
2018-08-04 01:13:03,189 0baf5535 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code.
2018-08-04 01:13:31,562 0baf5535 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json>
2018-08-04 01:13:35,580 0baf5535 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json}> ReturnCode:1
STDOUT:
BigQuery error in load operation: Error processing job
'apache-beam-testing:bqjob_r4a57b6221bd70572_00000165027f2756_1': Invalid schema
update. Field timestamp has changed type from TIMESTAMP to FLOAT
STDERR: Upload complete.Waiting on bqjob_r4a57b6221bd70572_00000165027f2756_1 ... (0s) Current status: RUNNING Waiting on bqjob_r4a57b6221bd70572_00000165027f2756_1 ... (0s) Current status: DONE
2018-08-04 01:13:35,580 0baf5535 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code.
2018-08-04 01:14:00,552 0baf5535 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json>
2018-08-04 01:14:04,695 0baf5535 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json}> ReturnCode:1
STDOUT:
BigQuery error in load operation: Error processing job
'apache-beam-testing:bqjob_r488e26fae3081563_00000165027f98c2_1': Invalid schema
update. Field timestamp has changed type from TIMESTAMP to FLOAT
STDERR: Upload complete.Waiting on bqjob_r488e26fae3081563_00000165027f98c2_1 ... (0s) Current status: RUNNING Waiting on bqjob_r488e26fae3081563_00000165027f98c2_1 ... (0s) Current status: DONE
2018-08-04 01:14:04,695 0baf5535 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code.
2018-08-04 01:14:21,638 0baf5535 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json>
2018-08-04 01:14:25,674 0baf5535 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json}> ReturnCode:1
STDOUT:
BigQuery error in load operation: Error processing job
'apache-beam-testing:bqjob_r4445d9fb83085a53_00000165027feb2c_1': Invalid schema
update. Field timestamp has changed type from TIMESTAMP to FLOAT
STDERR: Upload complete.Waiting on bqjob_r4445d9fb83085a53_00000165027feb2c_1 ... (0s) Current status: RUNNING Waiting on bqjob_r4445d9fb83085a53_00000165027feb2c_1 ... (0s) Current status: DONE
2018-08-04 01:14:25,674 0baf5535 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code.
2018-08-04 01:14:43,773 0baf5535 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json>
2018-08-04 01:14:47,831 0baf5535 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json}> ReturnCode:1
STDOUT:
BigQuery error in load operation: Error processing job
'apache-beam-testing:bqjob_r2fc5cc3b0eccf8_0000016502804192_1': Invalid schema
update. Field timestamp has changed type from TIMESTAMP to FLOAT
STDERR: Upload complete.Waiting on bqjob_r2fc5cc3b0eccf8_0000016502804192_1 ... (0s) Current status: RUNNING Waiting on bqjob_r2fc5cc3b0eccf8_0000016502804192_1 ... (0s) Current status: DONE
2018-08-04 01:14:47,832 0baf5535 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code.
2018-08-04 01:15:09,405 0baf5535 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json>
2018-08-04 01:15:13,393 0baf5535 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json}> ReturnCode:1
STDOUT:
BigQuery error in load operation: Error processing job
'apache-beam-testing:bqjob_r1da0a31736f5c6b5_000001650280a5c2_1': Invalid schema
update. Field timestamp has changed type from TIMESTAMP to FLOAT
STDERR: Upload complete.Waiting on bqjob_r1da0a31736f5c6b5_000001650280a5c2_1 ... (0s) Current status: RUNNING Waiting on bqjob_r1da0a31736f5c6b5_000001650280a5c2_1 ... (0s) Current status: DONE
2018-08-04 01:15:13,394 0baf5535 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code.
2018-08-04 01:15:42,871 0baf5535 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json>
2018-08-04 01:15:46,803 0baf5535 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json}> ReturnCode:1
STDOUT:
BigQuery error in load operation: Error processing job
'apache-beam-testing:bqjob_r296782e907644c9c_000001650281286f_1': Invalid schema
update. Field timestamp has changed type from TIMESTAMP to FLOAT
STDERR: Upload complete.Waiting on bqjob_r296782e907644c9c_000001650281286f_1 ... (0s) Current status: RUNNING Waiting on bqjob_r296782e907644c9c_000001650281286f_1 ... (0s) Current status: DONE
2018-08-04 01:15:46,803 0baf5535 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code.
2018-08-04 01:16:04,053 0baf5535 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json>
2018-08-04 01:16:08,105 0baf5535 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json}> ReturnCode:1
STDOUT:
BigQuery error in load operation: Error processing job
'apache-beam-testing:bqjob_r168cf30e70c68a2_0000016502817b34_1': Invalid schema
update. Field timestamp has changed type from TIMESTAMP to FLOAT
STDERR: Upload complete.Waiting on bqjob_r168cf30e70c68a2_0000016502817b34_1 ... (0s) Current status: RUNNING Waiting on bqjob_r168cf30e70c68a2_0000016502817b34_1 ... (0s) Current status: DONE
2018-08-04 01:16:08,106 0baf5535 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code.
2018-08-04 01:16:31,439 0baf5535 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json>
2018-08-04 01:16:35,400 0baf5535 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json}> ReturnCode:1
STDOUT:
BigQuery error in load operation: Error processing job
'apache-beam-testing:bqjob_r5df6a78dbcd141cd_000001650281e615_1': Invalid schema
update. Field timestamp has changed type from TIMESTAMP to FLOAT
STDERR: Upload complete.Waiting on bqjob_r5df6a78dbcd141cd_000001650281e615_1 ... (0s) Current status: RUNNING Waiting on bqjob_r5df6a78dbcd141cd_000001650281e615_1 ... (0s) Current status: DONE
2018-08-04 01:16:35,401 0baf5535 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code.
2018-08-04 01:16:55,147 0baf5535 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json>
2018-08-04 01:16:59,535 0baf5535 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json}> ReturnCode:1
STDOUT:
BigQuery error in load operation: Error processing job
'apache-beam-testing:bqjob_r6a5b38b8cff7b7cb_00000165028242d8_1': Invalid schema
update. Field timestamp has changed type from TIMESTAMP to FLOAT
STDERR: Upload complete.Waiting on bqjob_r6a5b38b8cff7b7cb_00000165028242d8_1 ... (0s) Current status: RUNNING Waiting on bqjob_r6a5b38b8cff7b7cb_00000165028242d8_1 ... (0s) Current status: DONE
2018-08-04 01:16:59,536 0baf5535 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code.
2018-08-04 01:17:18,503 0baf5535 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json>
2018-08-04 01:17:22,519 0baf5535 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json}> ReturnCode:1
STDOUT:
BigQuery error in load operation: Error processing job
'apache-beam-testing:bqjob_r540d80a10f444f7e_0000016502829e11_1': Invalid schema
update. Field timestamp has changed type from TIMESTAMP to FLOAT
STDERR: Upload complete.Waiting on bqjob_r540d80a10f444f7e_0000016502829e11_1 ... (0s) Current status: RUNNING Waiting on bqjob_r540d80a10f444f7e_0000016502829e11_1 ... (0s) Current status: DONE
2018-08-04 01:17:22,519 0baf5535 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code.
2018-08-04 01:17:46,148 0baf5535 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json>
2018-08-04 01:17:50,073 0baf5535 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json}> ReturnCode:1
STDOUT:
BigQuery error in load operation: Error processing job
'apache-beam-testing:bqjob_r41abbeef2f17231a_00000165028309f8_1': Invalid schema
update. Field timestamp has changed type from TIMESTAMP to FLOAT
STDERR: Upload complete.Waiting on bqjob_r41abbeef2f17231a_00000165028309f8_1 ... (0s) Current status: RUNNING Waiting on bqjob_r41abbeef2f17231a_00000165028309f8_1 ... (0s) Current status: DONE
2018-08-04 01:17:50,073 0baf5535 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code.
2018-08-04 01:18:14,823 0baf5535 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json>
2018-08-04 01:18:18,648 0baf5535 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json}> ReturnCode:1
STDOUT:
BigQuery error in load operation: Error processing job
'apache-beam-testing:bqjob_r1089e7f2c23201e8_00000165028379fc_1': Invalid schema
update. Field timestamp has changed type from TIMESTAMP to FLOAT
STDERR: Upload complete.Waiting on bqjob_r1089e7f2c23201e8_00000165028379fc_1 ... (0s) Current status: RUNNING Waiting on bqjob_r1089e7f2c23201e8_00000165028379fc_1 ... (0s) Current status: DONE
2018-08-04 01:18:18,649 0baf5535 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code.
2018-08-04 01:18:47,328 0baf5535 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json>
2018-08-04 01:18:51,411 0baf5535 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json}> ReturnCode:1
STDOUT:
BigQuery error in load operation: Error processing job
'apache-beam-testing:bqjob_r45ba4d1ae2d72753_000001650283f8e7_1': Invalid schema
update. Field timestamp has changed type from TIMESTAMP to FLOAT
STDERR: Upload complete.Waiting on bqjob_r45ba4d1ae2d72753_000001650283f8e7_1 ... (0s) Current status: RUNNING Waiting on bqjob_r45ba4d1ae2d72753_000001650283f8e7_1 ... (0s) Current status: DONE
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/pkb.py",> line 21, in <module>
sys.exit(Main())
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 1012, in Main
return RunBenchmarks()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 939, in RunBenchmarks
collector.PublishSamples()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/publisher.py",> line 928, in PublishSamples
publisher.PublishSamples(self.samples)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/publisher.py",> line 538, in PublishSamples
vm_util.IssueRetryableCommand(load_cmd)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/vm_util.py",> line 249, in WrappedFunction
return f(*args, **kwargs)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/vm_util.py",> line 387, in IssueRetryableCommand
'Command returned a non-zero exit code.\n')
perfkitbenchmarker.errors.CalledProcessException: Command returned a non-zero exit code.
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PerformanceTests_Python #1266
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1266/display/redirect>
------------------------------------------
GitHub pull request #6142 of commit c48115558d7af37b518f69463dba5bd7612054a5, no merge conflicts.
Setting status of c48115558d7af37b518f69463dba5bd7612054a5 to PENDING with url https://builds.apache.org/job/beam_PerformanceTests_Python/1266/ and message: 'Build started for merge commit.'
Using context: Python SDK Performance Test
[EnvInject] - Loading node environment variables.
Building remotely on beam15 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/6142/*:refs/remotes/origin/pr/6142/*
> git rev-parse refs/remotes/origin/pr/6142/merge^{commit} # timeout=10
> git rev-parse refs/remotes/origin/origin/pr/6142/merge^{commit} # timeout=10
Checking out Revision abc1bce29360cd14fc5fb6f62c1c52193d112fa4 (refs/remotes/origin/pr/6142/merge)
> git config core.sparsecheckout # timeout=10
> git checkout -f abc1bce29360cd14fc5fb6f62c1c52193d112fa4
Commit message: "Merge c48115558d7af37b518f69463dba5bd7612054a5 into 2d3ce5181ad144de9c5cc6f945a6d40f9c0ae795"
First time build. Skipping changelog.
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8333306446748541054.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker>
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3957460423414083293.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/.beam_env>
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2172882893396186781.sh
+ virtualenv <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/.beam_env> --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/.beam_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/.beam_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5967812430427483923.sh
+ virtualenv <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/.perfkit_env> --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/.perfkit_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/.perfkit_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5622511714784926622.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/.beam_env/bin/pip> install --upgrade setuptools pip grpcio-tools==1.3.5
Requirement already up-to-date: setuptools in ./src/.beam_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./src/.beam_env/lib/python2.7/site-packages (18.0)
Collecting grpcio-tools==1.3.5
Using cached https://files.pythonhosted.org/packages/05/f6/0296e29b1bac6f85d2a8556d48adf825307f73109a3c2c17fb734292db0a/grpcio_tools-1.3.5-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied, skipping upgrade: grpcio>=1.3.5 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio-tools==1.3.5) (1.13.0)
Requirement already satisfied, skipping upgrade: protobuf>=3.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio-tools==1.3.5) (3.6.0)
Requirement already satisfied, skipping upgrade: six>=1.5.2 in /usr/local/lib/python2.7/dist-packages (from grpcio>=1.3.5->grpcio-tools==1.3.5) (1.11.0)
Requirement already satisfied, skipping upgrade: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio>=1.3.5->grpcio-tools==1.3.5) (1.1.6)
Requirement already satisfied, skipping upgrade: futures>=2.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio>=1.3.5->grpcio-tools==1.3.5) (3.2.0)
Installing collected packages: grpcio-tools
Successfully installed grpcio-tools-1.3.5
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins514423414470920267.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/.perfkit_env/bin/pip> install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./PerfKitBenchmarker/.perfkit_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./PerfKitBenchmarker/.perfkit_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3743757725069306645.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker>
fatal: destination path '<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker'> already exists and is not an empty directory.
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PerformanceTests_Python #1265
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1265/display/redirect?page=changes>
Changes:
[yifanzou] [BEAM-5018] Upgrade org.tukaani:xz to 1.8
[ryan.blake.williams] Revert "Install fastavro only in linux"
[ankurgoenka] Tests for running Python on Flink.
[ankurgoenka] Fixing lint issues
[gleb] [BEAM-5050] [SQL] Fix aggregation of nulls
[yifanzou] [BEAM-3906] Automate Validation Aganist Python Wheel
[chamikara] Fixes a Javadoc error
[ankurgoenka] Adding gradle task for python streaming validates flink runner
[aaltay] [BEAM-4391] Example of distributed optimization (#5736)
[tweise] [BEAM-4778] add option to flink job server to clean staged artifacts
[pablo] Fixed wrong isRelease() call in publish
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam15 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision f96b25a947a58542189e62fe46236b63ea99ca6a (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f f96b25a947a58542189e62fe46236b63ea99ca6a
Commit message: "Fixed wrong isRelease() call in publish"
> git rev-list --no-walk f2ae73be3d3d1cb7c3d111505f832aed44bc92eb # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4635927866774441150.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6534342280852594361.sh
+ rm -rf .beam_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4523054878649429971.sh
+ rm -rf .perfkit_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2044191547514904883.sh
+ virtualenv .beam_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins869372471515896118.sh
+ virtualenv .perfkit_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3804156420611226352.sh
+ .beam_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.beam_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.beam_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5269395541421208009.sh
+ .perfkit_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.perfkit_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.perfkit_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1926765491110891651.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3145344628393015723.sh
+ .beam_env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.8.2)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.7)
Requirement already satisfied: dill<=0.2.8.2,>=0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.2.8.2)
Requirement already satisfied: fastavro==0.19.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.19.7)
Requirement already satisfied: grpcio<2,>=1.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.13.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.1.0)
Requirement already satisfied: httplib2<=0.11.3,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.11.3)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.0.0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (4.1.2)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.0)
Requirement already satisfied: pydot<1.3,>=1.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.2.4)
Requirement already satisfied: pytz<=2018.4,>=2018.3 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2018.4)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (3.12)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.6.8)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.11.0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.4)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.2.0)
Requirement already satisfied: future<1.0.0,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.16.0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.5.20)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.90.4)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (7.0.1)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.26.0)
Requirement already satisfied: proto-google-cloud-pubsub-v1==0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: nose>=1.3.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.3.7)
Collecting numpy<2,>=1.14.3 (from apache-beam==2.7.0.dev0)
Using cached https://files.pythonhosted.org/packages/85/51/ba4564ded90e093dbb6adfc3e21f99ae953d9ad56477e1b0d4a93bacf7d3/numpy-1.15.0-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.9.0)
Requirement already satisfied: setuptools>=18.0 in ./.beam_env/lib/python2.7/site-packages (from fastavro==0.19.7->apache-beam==2.7.0.dev0) (40.0.0)
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.7.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.8.0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.0.2)
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.4.2)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.2.2)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (3.4.2)
Requirement already satisfied: pyparsing>=2.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from pydot<1.3,>=1.2.0->apache-beam==2.7.0.dev0) (2.2.0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (0.14.1)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.7.0.dev0) (1.5.3)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (1.22)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (1.5)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.16)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.11.4)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.0.3)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (1.5.0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (3.8)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (2.1.0)
Installing collected packages: numpy, apache-beam
Found existing installation: numpy 1.13.3
Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'numpy'. No files were found to uninstall.
Found existing installation: apache-beam 2.7.0.dev0
Not uninstalling apache-beam at /home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_HadoopInputFormat/src/sdks/python, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'apache-beam'. No files were found to uninstall.
Running setup.py develop for apache-beam
Successfully installed apache-beam numpy-1.15.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3715548868458883605.sh
+ .beam_env/bin/python src/sdks/python/setup.py sdist --dist-dir=src/sdks/python/target
Traceback (most recent call last):
File "src/sdks/python/setup.py", line 46, in <module>
PACKAGE_VERSION = get_version()
File "src/sdks/python/setup.py", line 41, in get_version
exec(open(os.path.normpath('./apache_beam/version.py')).read(), global_names) # pylint: disable=exec-used
IOError: [Errno 2] No such file or directory: 'apache_beam/version.py'
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PerformanceTests_Python #1264
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1264/display/redirect>
------------------------------------------
[...truncated 1.73 KB...]
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3345867546547775270.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1399758868827400867.sh
+ rm -rf .beam_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6273576107848702835.sh
+ rm -rf .perfkit_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7994866195948922848.sh
+ virtualenv .beam_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins89803871673548492.sh
+ virtualenv .perfkit_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4391940347346608197.sh
+ .beam_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.beam_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.beam_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins401290153052960708.sh
+ .perfkit_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.perfkit_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.perfkit_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7704870098660130538.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins667339526735438882.sh
+ .beam_env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.8.2)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.7)
Requirement already satisfied: dill<=0.2.8.2,>=0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.2.8.2)
Requirement already satisfied: fastavro==0.19.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.19.7)
Requirement already satisfied: grpcio<2,>=1.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.13.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.1.0)
Requirement already satisfied: httplib2<=0.11.3,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.11.3)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.0.0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (4.1.2)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.0)
Requirement already satisfied: pydot<1.3,>=1.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.2.4)
Requirement already satisfied: pytz<=2018.4,>=2018.3 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2018.4)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (3.12)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.6.8)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.11.0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.4)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.2.0)
Requirement already satisfied: future<1.0.0,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.16.0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.5.20)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.90.4)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (7.0.1)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.26.0)
Requirement already satisfied: proto-google-cloud-pubsub-v1==0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: nose>=1.3.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.3.7)
Collecting numpy<2,>=1.14.3 (from apache-beam==2.7.0.dev0)
Using cached https://files.pythonhosted.org/packages/85/51/ba4564ded90e093dbb6adfc3e21f99ae953d9ad56477e1b0d4a93bacf7d3/numpy-1.15.0-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.9.0)
Requirement already satisfied: setuptools>=18.0 in ./.beam_env/lib/python2.7/site-packages (from fastavro==0.19.7->apache-beam==2.7.0.dev0) (40.0.0)
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.7.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.8.0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.0.2)
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.4.2)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.2.2)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (3.4.2)
Requirement already satisfied: pyparsing>=2.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from pydot<1.3,>=1.2.0->apache-beam==2.7.0.dev0) (2.2.0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (0.14.1)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.7.0.dev0) (1.5.3)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (1.22)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (1.5)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.16)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.11.4)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.0.3)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (1.5.0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (3.8)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (2.1.0)
Installing collected packages: numpy, apache-beam
Found existing installation: numpy 1.13.3
Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'numpy'. No files were found to uninstall.
Found existing installation: apache-beam 2.7.0.dev0
Not uninstalling apache-beam at /home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_HadoopInputFormat/src/sdks/python, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'apache-beam'. No files were found to uninstall.
Running setup.py develop for apache-beam
Successfully installed apache-beam numpy-1.15.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2076722097612132100.sh
+ .beam_env/bin/python src/sdks/python/setup.py sdist --dist-dir=src/sdks/python/target
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/local/lib/python2.7/site-packages/setuptools/dist.py>:398: UserWarning: Normalizing '2.7.0.dev' to '2.7.0.dev0'
normalized_version,
running sdist
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/gen_protos.py>:49: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
warning: cmd: standard file not found: should have one of README, README.rst, README.txt, README.md
running check
creating apache-beam-2.7.0.dev0
creating apache-beam-2.7.0.dev0/apache_beam.egg-info
copying files to apache-beam-2.7.0.dev0...
copying apache_beam.egg-info/PKG-INFO -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/SOURCES.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/dependency_links.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/entry_points.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/not-zip-safe -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/requires.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/top_level.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
Writing apache-beam-2.7.0.dev0/setup.cfg
creating src/sdks/python/target
Creating tar archive
removing 'apache-beam-2.7.0.dev0' (and everything under it)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6171094355052098590.sh
+ .perfkit_env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14)) (0.2.2)
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15)) (2.10)
Requirement already satisfied: setuptools in ./.perfkit_env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16)) (40.0.0)
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17)) (2.6.0)
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18)) (1.4)
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19)) (3.2.0)
Requirement already satisfied: PyYAML==3.12 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 20)) (3.12)
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21)) (0.8.1)
Requirement already satisfied: numpy==1.13.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22)) (1.13.3)
Requirement already satisfied: functools32 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 23)) (3.2.3.post2)
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24)) (0.5.5)
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25)) (0.3.0)
Requirement already satisfied: timeout-decorator in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 26)) (0.4.0)
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Requirement already satisfied: colorama; extra == "windows" in /home/jenkins/.local/lib/python2.7/site-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17)) (0.3.9)
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.0)
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.11.0)
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.2.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2526341026312251996.sh
+ .perfkit_env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --bigquery_table=beam_performance.pkb_results --k8s_get_retry_count=36 --k8s_get_wait_interval=10 --python_binary=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --beam_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_timeout=3600 --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output --beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it
2018-08-02 16:57:46,438 83b8abb4 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/83b8abb4/pkb.log>
2018-08-02 16:57:46,438 83b8abb4 MainThread INFO PerfKitBenchmarker version: v1.12.0-747-g7e4ea3f
2018-08-02 16:57:46,439 83b8abb4 MainThread INFO Flag values:
--beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it
--beam_it_timeout=3600
--beam_sdk=python
--k8s_get_wait_interval=10
--python_binary=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python>
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--beam_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src>
--k8s_get_retry_count=36
--benchmarks=beam_integration_benchmark
2018-08-02 16:57:46,693 83b8abb4 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-08-02 16:57:46,900 83b8abb4 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-08-02 16:57:46,905 83b8abb4 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-08-02 16:57:46,905 83b8abb4 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/gradlew> clean assemble --stacktrace --info -DintegrationTestRunner=dataflow
2018-08-02 17:03:48,932 83b8abb4 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-08-02 17:03:49,025 83b8abb4 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> setup.py nosetests --tests=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/target/apache-beam-2.7.0.dev0.tar.gz>
2018-08-02 18:03:49,033 83b8abb4 Thread-3 ERROR IssueCommand timed out after 3600 seconds. Killing command "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> setup.py nosetests --tests=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/target/apache-beam-2.7.0.dev0.tar.gz".>
2018-08-02 18:03:49,043 83b8abb4 MainThread beam_integration_benchmark(1/1) INFO Ran: {<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> setup.py nosetests --tests=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/target/apache-beam-2.7.0.dev0.tar.gz}> ReturnCode:-9
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/local/lib/python2.7/site-packages/setuptools/dist.py>:398: UserWarning: Normalizing '2.7.0.dev' to '2.7.0.dev0'
normalized_version,
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/gen_protos.py>:49: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py>:54: DeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
print('Found: %s.' % self.build_console_url(pipeline.options))
2018-08-02 18:03:49,044 83b8abb4 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 672, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 552, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-08-02 18:03:49,046 83b8abb4 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-08-02 18:03:49,049 83b8abb4 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 808, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 672, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 552, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-08-02 18:03:49,049 83b8abb4 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-08-02 18:03:49,079 83b8abb4 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-08-02 18:03:49,079 83b8abb4 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/83b8abb4/pkb.log>
2018-08-02 18:03:49,080 83b8abb4 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/83b8abb4/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PerformanceTests_Python #1263
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1263/display/redirect?page=changes>
Changes:
[rober] Avoid redundant computation when invoking ProcessElements in ParDo
[github] Update fn_test.go
[rober] De-dup pre & post invoke code in pardo.go
[timrobertson100] [BEAM-2661] Adds KuduIO
[github] Properly revert log line changes in fn_test.go
[rober] Add comparitive benchmark between cached and single use invoker.
[github] [BEAM-4852] Only read symbol table when required.
[github] Update symbols.go
[timrobertson100] [BEAM-2661] KuduIO: Add missing licenses
[ehudm] Make with_attributes kwarg optional.
[ehudm] Add decode and encode steps to streaming_wordcount
[qinyeli] Interactive Beam -- display update
[ehudm] Fix PubSubMessageMatcher not acking messages.
[ebeach] [BEAM-4862] Fixes bug in Spanner's MutationGroupEncoder by converting
[herohde] Add utilities for manipulating proto pipelines
[herohde] [BEAM-4636] Refactor Go dataflow runner for more flexible uses
[herohde] [BEAM-4813] Switch to translation from model protos to Dataflow v1beta3
[herohde] Add more Go integration tests and fix CoGBK bugs
[herohde] [BEAM-4636] Ensure unique filename for compiled worker binaries
[herohde] Handle explicit flatten
[herohde] Fix Dataflow argument bug
[herohde] CR: tweak code and fix comments
[aaltay] [BEAM-4751] fix missing pylint3 check for io subpackage (#5916)
[pablo] Add GitScm poll trigger for post-commit tests.
[relax] Enable schemas for Dataflow runner.
[tweise] [BEAM-5023] Send HarnessId in DataClient (#6066)
[relax] Fix BigQuery job retry mechanism.
[relax] Add exponential backoff on retries.
[relax] Apply code-review comment.
[tweise] [BEAM-2930] Side inputs are not yet supported in streaming mode.
[github] Don't rely on order of elements in a PCollection after GBK in
[relax] Properly handle running jobs.
[devinduan] Spelling mistakes
[relax] Convert BeamSQL to use Schemas.
[relax] Deprecate getRowCoder.
[relax] Add setSchema to remaining Table objects.
[relax] Delete a bunch of code that is no longer used.
[relax] Move utilities into schemas.utils package.
[relax] Convert Nexmark to use schemas.
[relax] remove deprecated Schema.getRowCoder
[relax] Apply spotless.
[relax] Plumb schema through DoFn.
[relax] Add helper setRowSchema to eliminatre boilerplate.
[rober] Avoid overwritting user changes to Resolver
[rober] Clean up deferedResolver
[thw] [BEAM-4842] Update Flink Runner to Flink 1.5.2
[migryz] Add retries for gpg key query logic.
[pablo] Fix scheduling for jobs
[relax] Add convenience methods for pojo and javabean schema registration.
[relax] Address code-review comments.
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision f2ae73be3d3d1cb7c3d111505f832aed44bc92eb (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f f2ae73be3d3d1cb7c3d111505f832aed44bc92eb
Commit message: "Merge pull request #6109: [BEAM-4842] Update Flink Runner to Flink 1.5.2"
> git rev-list --no-walk ff95a82e461bd8319d9733be60e75992ba90cd7c # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8260919133135875607.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3883146302874079411.sh
+ rm -rf .beam_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8573027766605549178.sh
+ rm -rf .perfkit_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7954528295858307459.sh
+ virtualenv .beam_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins117727936914663087.sh
+ virtualenv .perfkit_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4530960035642880902.sh
+ .beam_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.beam_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.beam_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5189163966448436943.sh
+ .perfkit_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.perfkit_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.perfkit_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6867585134301396263.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1463111188888185076.sh
+ .beam_env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.8.2)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.7)
Requirement already satisfied: dill<=0.2.8.2,>=0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.2.8.2)
Requirement already satisfied: grpcio<2,>=1.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.13.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.1.0)
Requirement already satisfied: httplib2<=0.11.3,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.11.3)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.0.0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (4.1.2)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.0)
Requirement already satisfied: pydot<1.3,>=1.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.2.4)
Requirement already satisfied: pytz<=2018.4,>=2018.3 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2018.4)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (3.12)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.6.8)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.11.0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.4)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.2.0)
Requirement already satisfied: future<1.0.0,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.16.0)
Requirement already satisfied: fastavro==0.19.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.19.7)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.5.20)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.90.4)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (7.0.1)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.26.0)
Requirement already satisfied: proto-google-cloud-pubsub-v1==0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: nose>=1.3.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.3.7)
Collecting numpy<2,>=1.14.3 (from apache-beam==2.7.0.dev0)
Using cached https://files.pythonhosted.org/packages/85/51/ba4564ded90e093dbb6adfc3e21f99ae953d9ad56477e1b0d4a93bacf7d3/numpy-1.15.0-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.9.0)
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.7.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.8.0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.0.2)
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.4.2)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.2.2)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (3.4.2)
Requirement already satisfied: setuptools in ./.beam_env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.7.0.dev0) (40.0.0)
Requirement already satisfied: pyparsing>=2.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from pydot<1.3,>=1.2.0->apache-beam==2.7.0.dev0) (2.2.0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (0.14.1)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.7.0.dev0) (1.5.3)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (1.22)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (1.5)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.16)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.11.4)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.0.3)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (1.5.0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (3.8)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (2.1.0)
Installing collected packages: numpy, apache-beam
Found existing installation: numpy 1.13.3
Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'numpy'. No files were found to uninstall.
Found existing installation: apache-beam 2.7.0.dev0
Not uninstalling apache-beam at /home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_Compressed_TextIOIT_HDFS/src/sdks/python, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'apache-beam'. No files were found to uninstall.
Running setup.py develop for apache-beam
Successfully installed apache-beam numpy-1.15.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6724119042722330580.sh
+ .beam_env/bin/python src/sdks/python/setup.py sdist --dist-dir=src/sdks/python/target
Traceback (most recent call last):
File "src/sdks/python/setup.py", line 46, in <module>
PACKAGE_VERSION = get_version()
File "src/sdks/python/setup.py", line 41, in get_version
exec(open(os.path.normpath('./apache_beam/version.py')).read(), global_names) # pylint: disable=exec-used
IOError: [Errno 2] No such file or directory: 'apache_beam/version.py'
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PerformanceTests_Python #1262
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1262/display/redirect>
------------------------------------------
[...truncated 2.19 KB...]
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins134203476966691273.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins266418180377408786.sh
+ rm -rf .beam_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2763990076871907715.sh
+ rm -rf .perfkit_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3751194031371478445.sh
+ virtualenv .beam_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3369103436413740422.sh
+ virtualenv .perfkit_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2139078975145550914.sh
+ .beam_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.beam_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.beam_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8720383783791910367.sh
+ .perfkit_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.perfkit_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.perfkit_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4964247970169696296.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5972620634519769960.sh
+ .beam_env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.8.2)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.7)
Requirement already satisfied: dill<=0.2.8.2,>=0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.2.8.2)
Requirement already satisfied: grpcio<2,>=1.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.13.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.1.0)
Requirement already satisfied: httplib2<=0.11.3,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.11.3)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.0.0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (4.1.2)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.0)
Requirement already satisfied: pydot<1.3,>=1.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.2.4)
Requirement already satisfied: pytz<=2018.4,>=2018.3 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2018.4)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (3.12)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.6.8)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.11.0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.4)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.2.0)
Requirement already satisfied: future<1.0.0,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.16.0)
Requirement already satisfied: fastavro==0.19.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.19.7)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.5.20)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.90.4)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (7.0.1)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.26.0)
Requirement already satisfied: proto-google-cloud-pubsub-v1==0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: nose>=1.3.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.3.7)
Collecting numpy<2,>=1.14.3 (from apache-beam==2.7.0.dev0)
Using cached https://files.pythonhosted.org/packages/85/51/ba4564ded90e093dbb6adfc3e21f99ae953d9ad56477e1b0d4a93bacf7d3/numpy-1.15.0-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.9.0)
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.7.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.8.0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.0.2)
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.4.2)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.2.2)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (3.4.2)
Requirement already satisfied: setuptools in ./.beam_env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.7.0.dev0) (40.0.0)
Requirement already satisfied: pyparsing>=2.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from pydot<1.3,>=1.2.0->apache-beam==2.7.0.dev0) (2.2.0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (0.14.1)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.7.0.dev0) (1.5.3)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (1.22)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (1.5)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.16)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.11.4)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.0.3)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (1.5.0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (3.8)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (2.1.0)
Installing collected packages: numpy, apache-beam
Found existing installation: numpy 1.13.3
Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'numpy'. No files were found to uninstall.
Found existing installation: apache-beam 2.7.0.dev0
Not uninstalling apache-beam at /home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_HadoopInputFormat/src/sdks/python, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'apache-beam'. No files were found to uninstall.
Running setup.py develop for apache-beam
Successfully installed apache-beam numpy-1.15.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins168109957690731329.sh
+ .beam_env/bin/python src/sdks/python/setup.py sdist --dist-dir=src/sdks/python/target
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/local/lib/python2.7/site-packages/setuptools/dist.py>:398: UserWarning: Normalizing '2.7.0.dev' to '2.7.0.dev0'
normalized_version,
running sdist
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/gen_protos.py>:49: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
running egg_info
creating apache_beam.egg-info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
warning: cmd: standard file not found: should have one of README, README.rst, README.txt, README.md
running check
creating apache-beam-2.7.0.dev0
creating apache-beam-2.7.0.dev0/apache_beam.egg-info
copying files to apache-beam-2.7.0.dev0...
copying apache_beam.egg-info/PKG-INFO -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/SOURCES.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/dependency_links.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/entry_points.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/not-zip-safe -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/requires.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/top_level.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
Writing apache-beam-2.7.0.dev0/setup.cfg
creating src/sdks/python/target
Creating tar archive
removing 'apache-beam-2.7.0.dev0' (and everything under it)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins9219101386029762686.sh
+ .perfkit_env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14)) (0.2.2)
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15)) (2.10)
Requirement already satisfied: setuptools in ./.perfkit_env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16)) (40.0.0)
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17)) (2.6.0)
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18)) (1.4)
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19)) (3.2.0)
Requirement already satisfied: PyYAML==3.12 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 20)) (3.12)
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21)) (0.8.1)
Requirement already satisfied: numpy==1.13.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22)) (1.13.3)
Requirement already satisfied: functools32 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 23)) (3.2.3.post2)
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24)) (0.5.5)
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25)) (0.3.0)
Requirement already satisfied: timeout-decorator in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 26)) (0.4.0)
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Requirement already satisfied: colorama; extra == "windows" in /home/jenkins/.local/lib/python2.7/site-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17)) (0.3.9)
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.0)
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.11.0)
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.2.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1829006631628808140.sh
+ .perfkit_env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --bigquery_table=beam_performance.pkb_results --k8s_get_retry_count=36 --k8s_get_wait_interval=10 --python_binary=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --beam_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_timeout=3600 --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output --beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it
2018-08-01 11:22:25,420 87a2f212 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/87a2f212/pkb.log>
2018-08-01 11:22:25,421 87a2f212 MainThread INFO PerfKitBenchmarker version: v1.12.0-747-g7e4ea3f
2018-08-01 11:22:25,422 87a2f212 MainThread INFO Flag values:
--beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it
--beam_it_timeout=3600
--beam_sdk=python
--k8s_get_wait_interval=10
--python_binary=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python>
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--beam_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src>
--k8s_get_retry_count=36
--benchmarks=beam_integration_benchmark
2018-08-01 11:22:25,728 87a2f212 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-08-01 11:22:25,960 87a2f212 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-08-01 11:22:25,964 87a2f212 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-08-01 11:22:25,964 87a2f212 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/gradlew> clean assemble --stacktrace --info -DintegrationTestRunner=dataflow
2018-08-01 11:28:25,618 87a2f212 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-08-01 11:28:25,719 87a2f212 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> setup.py nosetests --tests=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/target/apache-beam-2.7.0.dev0.tar.gz>
2018-08-01 12:28:25,727 87a2f212 Thread-3 ERROR IssueCommand timed out after 3600 seconds. Killing command "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> setup.py nosetests --tests=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/target/apache-beam-2.7.0.dev0.tar.gz".>
2018-08-01 12:28:25,737 87a2f212 MainThread beam_integration_benchmark(1/1) INFO Ran: {<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> setup.py nosetests --tests=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/target/apache-beam-2.7.0.dev0.tar.gz}> ReturnCode:-9
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/local/lib/python2.7/site-packages/setuptools/dist.py>:398: UserWarning: Normalizing '2.7.0.dev' to '2.7.0.dev0'
normalized_version,
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/gen_protos.py>:49: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py>:54: DeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
print('Found: %s.' % self.build_console_url(pipeline.options))
2018-08-01 12:28:25,738 87a2f212 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 672, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 552, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-08-01 12:28:25,739 87a2f212 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-08-01 12:28:25,741 87a2f212 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 808, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 672, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 552, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-08-01 12:28:25,742 87a2f212 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-08-01 12:28:25,796 87a2f212 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-08-01 12:28:25,797 87a2f212 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/87a2f212/pkb.log>
2018-08-01 12:28:25,797 87a2f212 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/87a2f212/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PerformanceTests_Python #1261
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1261/display/redirect?page=changes>
Changes:
[Jozef.Vilcek] Finalize writes before returning filenames via WriteFileResults
[yifanzou] [BEAM-5012] upgrade spring-expression
[aaltay] [BEAM-4747] mkdirs if they don't exist in localfilesystem (#5903)
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision ff95a82e461bd8319d9733be60e75992ba90cd7c (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f ff95a82e461bd8319d9733be60e75992ba90cd7c
Commit message: "Merge pull request #6091: [BEAM-5012] upgrade spring-expression"
> git rev-list --no-walk 7ea9c82457c96835215bf799836738e256bac1da # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8865120929856213274.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1473121459185172423.sh
+ rm -rf .beam_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3734836492793927241.sh
+ rm -rf .perfkit_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8679179533584043181.sh
+ virtualenv .beam_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2159511384415016943.sh
+ virtualenv .perfkit_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8080162836156584622.sh
+ .beam_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.beam_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.beam_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1309902400704482311.sh
+ .perfkit_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.perfkit_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.perfkit_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4478482058959712825.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1441016414319258742.sh
+ .beam_env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.8.2)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.7)
Requirement already satisfied: dill<=0.2.8.2,>=0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.2.8.2)
Requirement already satisfied: grpcio<2,>=1.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.13.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.1.0)
Requirement already satisfied: httplib2<=0.11.3,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.11.3)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.0.0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (4.1.2)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.0)
Requirement already satisfied: pydot<1.3,>=1.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.2.4)
Requirement already satisfied: pytz<=2018.4,>=2018.3 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2018.4)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (3.12)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.6.8)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.11.0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.4)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.2.0)
Requirement already satisfied: future<1.0.0,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.16.0)
Requirement already satisfied: fastavro==0.19.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.19.7)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.5.20)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.90.4)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (7.0.1)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.26.0)
Requirement already satisfied: proto-google-cloud-pubsub-v1==0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: nose>=1.3.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.3.7)
Collecting numpy<2,>=1.14.3 (from apache-beam==2.7.0.dev0)
Using cached https://files.pythonhosted.org/packages/85/51/ba4564ded90e093dbb6adfc3e21f99ae953d9ad56477e1b0d4a93bacf7d3/numpy-1.15.0-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.9.0)
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.7.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.8.0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.0.2)
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.4.2)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.2.2)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (3.4.2)
Requirement already satisfied: setuptools in ./.beam_env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.7.0.dev0) (40.0.0)
Requirement already satisfied: pyparsing>=2.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from pydot<1.3,>=1.2.0->apache-beam==2.7.0.dev0) (2.2.0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (0.14.1)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.7.0.dev0) (1.5.3)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (1.22)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (1.5)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.16)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.11.4)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.0.3)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (1.5.0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (3.8)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (2.1.0)
Installing collected packages: numpy, apache-beam
Found existing installation: numpy 1.13.3
Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'numpy'. No files were found to uninstall.
Found existing installation: apache-beam 2.7.0.dev0
Not uninstalling apache-beam at /home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_Compressed_TextIOIT_HDFS/src/sdks/python, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'apache-beam'. No files were found to uninstall.
Running setup.py develop for apache-beam
Successfully installed apache-beam numpy-1.15.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3075750881756728596.sh
+ .beam_env/bin/python src/sdks/python/setup.py sdist --dist-dir=src/sdks/python/target
Traceback (most recent call last):
File "src/sdks/python/setup.py", line 46, in <module>
PACKAGE_VERSION = get_version()
File "src/sdks/python/setup.py", line 41, in get_version
exec(open(os.path.normpath('./apache_beam/version.py')).read(), global_names) # pylint: disable=exec-used
IOError: [Errno 2] No such file or directory: 'apache_beam/version.py'
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PerformanceTests_Python #1260
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1260/display/redirect>
------------------------------------------
[...truncated 1.68 KB...]
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2607446597210156195.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2443095237927008836.sh
+ rm -rf .beam_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1189694480124475100.sh
+ rm -rf .perfkit_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2975676980433889651.sh
+ virtualenv .beam_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6197281220405623172.sh
+ virtualenv .perfkit_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3153161168204468762.sh
+ .beam_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.beam_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.beam_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3320282445925404651.sh
+ .perfkit_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.perfkit_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.perfkit_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins9169151424390991382.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins765699272845885303.sh
+ .beam_env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.8.2)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.7)
Requirement already satisfied: dill<=0.2.8.2,>=0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.2.8.2)
Requirement already satisfied: grpcio<2,>=1.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.13.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.1.0)
Requirement already satisfied: httplib2<=0.11.3,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.11.3)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.0.0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (4.1.2)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.0)
Requirement already satisfied: pydot<1.3,>=1.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.2.4)
Requirement already satisfied: pytz<=2018.4,>=2018.3 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2018.4)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (3.12)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.6.8)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.11.0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.4)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.2.0)
Requirement already satisfied: future<1.0.0,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.16.0)
Requirement already satisfied: fastavro==0.19.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.19.7)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.5.20)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.90.4)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (7.0.1)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.26.0)
Requirement already satisfied: proto-google-cloud-pubsub-v1==0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: nose>=1.3.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.3.7)
Collecting numpy<2,>=1.14.3 (from apache-beam==2.7.0.dev0)
Using cached https://files.pythonhosted.org/packages/85/51/ba4564ded90e093dbb6adfc3e21f99ae953d9ad56477e1b0d4a93bacf7d3/numpy-1.15.0-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.9.0)
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.7.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.8.0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.0.2)
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.4.2)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.2.2)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (3.4.2)
Requirement already satisfied: setuptools in ./.beam_env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.7.0.dev0) (40.0.0)
Requirement already satisfied: pyparsing>=2.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from pydot<1.3,>=1.2.0->apache-beam==2.7.0.dev0) (2.2.0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (0.14.1)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.7.0.dev0) (1.5.3)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (1.22)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (1.5)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.16)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.11.4)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.0.3)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (1.5.0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (3.8)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (2.1.0)
Installing collected packages: numpy, apache-beam
Found existing installation: numpy 1.13.3
Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'numpy'. No files were found to uninstall.
Found existing installation: apache-beam 2.7.0.dev0
Not uninstalling apache-beam at /home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_Compressed_TextIOIT_HDFS/src/sdks/python, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'apache-beam'. No files were found to uninstall.
Running setup.py develop for apache-beam
Successfully installed apache-beam numpy-1.15.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4561233640386505352.sh
+ .beam_env/bin/python src/sdks/python/setup.py sdist --dist-dir=src/sdks/python/target
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/local/lib/python2.7/site-packages/setuptools/dist.py>:398: UserWarning: Normalizing '2.7.0.dev' to '2.7.0.dev0'
normalized_version,
running sdist
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/gen_protos.py>:49: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
warning: cmd: standard file not found: should have one of README, README.rst, README.txt, README.md
running check
creating apache-beam-2.7.0.dev0
creating apache-beam-2.7.0.dev0/apache_beam.egg-info
copying files to apache-beam-2.7.0.dev0...
copying apache_beam.egg-info/PKG-INFO -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/SOURCES.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/dependency_links.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/entry_points.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/not-zip-safe -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/requires.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/top_level.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
Writing apache-beam-2.7.0.dev0/setup.cfg
creating src/sdks/python/target
Creating tar archive
removing 'apache-beam-2.7.0.dev0' (and everything under it)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4538288153756298284.sh
+ .perfkit_env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14)) (0.2.2)
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15)) (2.10)
Requirement already satisfied: setuptools in ./.perfkit_env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16)) (40.0.0)
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17)) (2.6.0)
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18)) (1.4)
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19)) (3.2.0)
Requirement already satisfied: PyYAML==3.12 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 20)) (3.12)
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21)) (0.8.1)
Requirement already satisfied: numpy==1.13.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22)) (1.13.3)
Requirement already satisfied: functools32 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 23)) (3.2.3.post2)
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24)) (0.5.5)
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25)) (0.3.0)
Requirement already satisfied: timeout-decorator in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 26)) (0.4.0)
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Requirement already satisfied: colorama; extra == "windows" in /home/jenkins/.local/lib/python2.7/site-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17)) (0.3.9)
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.0)
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.11.0)
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.2.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2064771233173343101.sh
+ .perfkit_env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --bigquery_table=beam_performance.pkb_results --k8s_get_retry_count=36 --k8s_get_wait_interval=10 --python_binary=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --beam_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_timeout=3600 --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-07-30 10:18:46,224 4159b0b0 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/4159b0b0/pkb.log>
2018-07-30 10:18:46,225 4159b0b0 MainThread INFO PerfKitBenchmarker version: v1.12.0-742-g2a71a6d
2018-07-30 10:18:46,226 4159b0b0 MainThread INFO Flag values:
--beam_it_timeout=3600
--beam_sdk=python
--k8s_get_wait_interval=10
--python_binary=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python>
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--beam_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src>
--k8s_get_retry_count=36
--benchmarks=beam_integration_benchmark
2018-07-30 10:18:46,492 4159b0b0 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-07-30 10:18:46,668 4159b0b0 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-07-30 10:18:46,672 4159b0b0 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-07-30 10:18:46,672 4159b0b0 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/gradlew> clean assemble --stacktrace --info -DintegrationTestRunner=dataflow
2018-07-30 10:24:25,112 4159b0b0 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-07-30 10:24:25,205 4159b0b0 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/target/apache-beam-2.7.0.dev0.tar.gz>
2018-07-30 11:24:25,213 4159b0b0 Thread-3 ERROR IssueCommand timed out after 3600 seconds. Killing command "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/target/apache-beam-2.7.0.dev0.tar.gz".>
2018-07-30 11:24:25,222 4159b0b0 MainThread beam_integration_benchmark(1/1) INFO Ran: {<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/target/apache-beam-2.7.0.dev0.tar.gz}> ReturnCode:-9
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/local/lib/python2.7/site-packages/setuptools/dist.py>:398: UserWarning: Normalizing '2.7.0.dev' to '2.7.0.dev0'
normalized_version,
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/gen_protos.py>:49: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py>:54: DeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
print('Found: %s.' % self.build_console_url(pipeline.options))
2018-07-30 11:24:25,223 4159b0b0 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 670, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 550, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-07-30 11:24:25,224 4159b0b0 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-07-30 11:24:25,226 4159b0b0 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 806, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 670, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 550, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-07-30 11:24:25,227 4159b0b0 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-07-30 11:24:25,243 4159b0b0 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-07-30 11:24:25,244 4159b0b0 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/4159b0b0/pkb.log>
2018-07-30 11:24:25,244 4159b0b0 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/4159b0b0/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PerformanceTests_Python #1259
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1259/display/redirect>
------------------------------------------
[...truncated 1.68 KB...]
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1960843742017956862.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5401992170880531760.sh
+ rm -rf .beam_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3905576078412088570.sh
+ rm -rf .perfkit_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2493266811912257130.sh
+ virtualenv .beam_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6936278536347873486.sh
+ virtualenv .perfkit_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins458815562325241686.sh
+ .beam_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.beam_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.beam_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2381364775078793896.sh
+ .perfkit_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.perfkit_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.perfkit_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4726826176375157776.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8334416447047675338.sh
+ .beam_env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.8.2)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.7)
Requirement already satisfied: dill<=0.2.8.2,>=0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.2.8.2)
Requirement already satisfied: grpcio<2,>=1.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.13.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.1.0)
Requirement already satisfied: httplib2<=0.11.3,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.11.3)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.0.0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (4.1.2)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.0)
Requirement already satisfied: pydot<1.3,>=1.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.2.4)
Requirement already satisfied: pytz<=2018.4,>=2018.3 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2018.4)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (3.12)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.6.8)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.11.0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.4)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.2.0)
Requirement already satisfied: future<1.0.0,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.16.0)
Requirement already satisfied: fastavro==0.19.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.19.7)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.5.20)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.90.4)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (7.0.1)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.26.0)
Requirement already satisfied: proto-google-cloud-pubsub-v1==0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: nose>=1.3.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.3.7)
Collecting numpy<2,>=1.14.3 (from apache-beam==2.7.0.dev0)
Using cached https://files.pythonhosted.org/packages/85/51/ba4564ded90e093dbb6adfc3e21f99ae953d9ad56477e1b0d4a93bacf7d3/numpy-1.15.0-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.9.0)
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.7.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.8.0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.0.2)
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.4.2)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.2.2)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (3.4.2)
Requirement already satisfied: setuptools in ./.beam_env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.7.0.dev0) (40.0.0)
Requirement already satisfied: pyparsing>=2.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from pydot<1.3,>=1.2.0->apache-beam==2.7.0.dev0) (2.2.0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (0.14.1)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.7.0.dev0) (1.5.3)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (1.22)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (1.5)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.16)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.11.4)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.0.3)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (1.5.0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (3.8)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (2.1.0)
Installing collected packages: numpy, apache-beam
Found existing installation: numpy 1.13.3
Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'numpy'. No files were found to uninstall.
Found existing installation: apache-beam 2.7.0.dev0
Not uninstalling apache-beam at /home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_Compressed_TextIOIT_HDFS/src/sdks/python, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'apache-beam'. No files were found to uninstall.
Running setup.py develop for apache-beam
Successfully installed apache-beam numpy-1.15.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4358843471068673199.sh
+ .beam_env/bin/python src/sdks/python/setup.py sdist --dist-dir=src/sdks/python/target
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/local/lib/python2.7/site-packages/setuptools/dist.py>:398: UserWarning: Normalizing '2.7.0.dev' to '2.7.0.dev0'
normalized_version,
running sdist
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/gen_protos.py>:49: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
warning: cmd: standard file not found: should have one of README, README.rst, README.txt, README.md
running check
creating apache-beam-2.7.0.dev0
creating apache-beam-2.7.0.dev0/apache_beam.egg-info
copying files to apache-beam-2.7.0.dev0...
copying apache_beam.egg-info/PKG-INFO -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/SOURCES.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/dependency_links.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/entry_points.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/not-zip-safe -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/requires.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/top_level.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
Writing apache-beam-2.7.0.dev0/setup.cfg
creating src/sdks/python/target
Creating tar archive
removing 'apache-beam-2.7.0.dev0' (and everything under it)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2274858807667533021.sh
+ .perfkit_env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14)) (0.2.2)
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15)) (2.10)
Requirement already satisfied: setuptools in ./.perfkit_env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16)) (40.0.0)
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17)) (2.6.0)
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18)) (1.4)
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19)) (3.2.0)
Requirement already satisfied: PyYAML==3.12 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 20)) (3.12)
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21)) (0.8.1)
Requirement already satisfied: numpy==1.13.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22)) (1.13.3)
Requirement already satisfied: functools32 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 23)) (3.2.3.post2)
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24)) (0.5.5)
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25)) (0.3.0)
Requirement already satisfied: timeout-decorator in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 26)) (0.4.0)
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Requirement already satisfied: colorama; extra == "windows" in /home/jenkins/.local/lib/python2.7/site-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17)) (0.3.9)
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.0)
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.11.0)
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.2.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7191424344200540075.sh
+ .perfkit_env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --bigquery_table=beam_performance.pkb_results --k8s_get_retry_count=36 --k8s_get_wait_interval=10 --python_binary=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --beam_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_timeout=2400 --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-07-30 08:41:41,842 2c549279 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/2c549279/pkb.log>
2018-07-30 08:41:41,844 2c549279 MainThread INFO PerfKitBenchmarker version: v1.12.0-742-g2a71a6d
2018-07-30 08:41:41,846 2c549279 MainThread INFO Flag values:
--beam_it_timeout=2400
--beam_sdk=python
--k8s_get_wait_interval=10
--python_binary=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python>
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--beam_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src>
--k8s_get_retry_count=36
--benchmarks=beam_integration_benchmark
2018-07-30 08:41:42,216 2c549279 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-07-30 08:41:42,393 2c549279 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-07-30 08:41:42,396 2c549279 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-07-30 08:41:42,397 2c549279 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/gradlew> clean assemble --stacktrace --info -DintegrationTestRunner=dataflow
2018-07-30 08:47:24,989 2c549279 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-07-30 08:47:25,083 2c549279 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/target/apache-beam-2.7.0.dev0.tar.gz>
2018-07-30 09:27:25,091 2c549279 Thread-3 ERROR IssueCommand timed out after 2400 seconds. Killing command "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/target/apache-beam-2.7.0.dev0.tar.gz".>
2018-07-30 09:27:25,099 2c549279 MainThread beam_integration_benchmark(1/1) INFO Ran: {<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/target/apache-beam-2.7.0.dev0.tar.gz}> ReturnCode:-9
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/local/lib/python2.7/site-packages/setuptools/dist.py>:398: UserWarning: Normalizing '2.7.0.dev' to '2.7.0.dev0'
normalized_version,
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/gen_protos.py>:49: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py>:54: DeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
print('Found: %s.' % self.build_console_url(pipeline.options))
2018-07-30 09:27:25,100 2c549279 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 670, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 550, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-07-30 09:27:25,101 2c549279 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-07-30 09:27:25,103 2c549279 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 806, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 670, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 550, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-07-30 09:27:25,104 2c549279 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-07-30 09:27:25,140 2c549279 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-07-30 09:27:25,141 2c549279 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/2c549279/pkb.log>
2018-07-30 09:27:25,141 2c549279 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/2c549279/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PerformanceTests_Python #1258
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1258/display/redirect?page=changes>
Changes:
[ehudm] Python PubSub updates:
[ehudm] Fix lint errors and clarify PubsubMessage docs.
[ehudm] Skip unit test if pubsub is not installed.
[ehudm] Fix docstring indentation.
[ehudm] Minor fixes.
[lcwik] [BEAM-4860] Ensure that shaded artifacts that are produced don't leak
[pablo] [BEAM-4818] Move the visteg plugin to the root project.
[lcwik] [BEAM-4262] Bumping bigtable versions (#6078)
[github] [BEAM-4658] Update pipeline representation in runner support libraries
[pablo] Changed hardcoded version number into env variable. Added more comments.
[aaltay] [BEAM-4076] Fix schemas on Dataflow and FnApi (#6072)
[qinyeli] Introducing version into CacheManager
[lukasz.gajowy] [BEAM-5037] Fix deserialization problem due to transient field
[lukasz.gajowy] Remove redundant static field
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 7ea9c82457c96835215bf799836738e256bac1da (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 7ea9c82457c96835215bf799836738e256bac1da
Commit message: "Merge pull request #6083: [BEAM-5037] fix synthetic options"
> git rev-list --no-walk aa2b585a5356ce3b607dd3d17a3c6ab2ce65353b # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8055711672218079793.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3631238739723495981.sh
+ rm -rf .beam_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1463782467337498506.sh
+ rm -rf .perfkit_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7762123783080025661.sh
+ virtualenv .beam_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4373957814379593676.sh
+ virtualenv .perfkit_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2690147038434867229.sh
+ .beam_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.beam_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.beam_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7250500320601567036.sh
+ .perfkit_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.perfkit_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.perfkit_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7762139428107416882.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5165480218991547042.sh
+ .beam_env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.8.2)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.7)
Requirement already satisfied: dill<=0.2.8.2,>=0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.2.8.2)
Requirement already satisfied: grpcio<2,>=1.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.13.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.1.0)
Requirement already satisfied: httplib2<=0.11.3,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.11.3)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.0.0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (4.1.2)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.0)
Requirement already satisfied: pydot<1.3,>=1.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.2.4)
Requirement already satisfied: pytz<=2018.4,>=2018.3 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2018.4)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (3.12)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.6.8)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.11.0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.4)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.2.0)
Requirement already satisfied: future<1.0.0,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.16.0)
Requirement already satisfied: fastavro==0.19.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.19.7)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.5.20)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.90.4)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (7.0.1)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.26.0)
Requirement already satisfied: proto-google-cloud-pubsub-v1==0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: nose>=1.3.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.3.7)
Collecting numpy<2,>=1.14.3 (from apache-beam==2.7.0.dev0)
Using cached https://files.pythonhosted.org/packages/85/51/ba4564ded90e093dbb6adfc3e21f99ae953d9ad56477e1b0d4a93bacf7d3/numpy-1.15.0-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.9.0)
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.7.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.8.0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.0.2)
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.4.2)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.2.2)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (3.4.2)
Requirement already satisfied: setuptools in ./.beam_env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.7.0.dev0) (40.0.0)
Requirement already satisfied: pyparsing>=2.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from pydot<1.3,>=1.2.0->apache-beam==2.7.0.dev0) (2.2.0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (0.14.1)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.7.0.dev0) (1.5.3)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (1.22)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (1.5)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.16)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.11.4)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.0.3)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (1.5.0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (3.8)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (2.1.0)
Installing collected packages: numpy, apache-beam
Found existing installation: numpy 1.13.3
Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'numpy'. No files were found to uninstall.
Found existing installation: apache-beam 2.7.0.dev0
Not uninstalling apache-beam at /home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_Compressed_TextIOIT_HDFS/src/sdks/python, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'apache-beam'. No files were found to uninstall.
Running setup.py develop for apache-beam
Successfully installed apache-beam numpy-1.15.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2566799033419822571.sh
+ .beam_env/bin/python src/sdks/python/setup.py sdist --dist-dir=src/sdks/python/target
Traceback (most recent call last):
File "src/sdks/python/setup.py", line 46, in <module>
PACKAGE_VERSION = get_version()
File "src/sdks/python/setup.py", line 41, in get_version
exec(open(os.path.normpath('./apache_beam/version.py')).read(), global_names) # pylint: disable=exec-used
IOError: [Errno 2] No such file or directory: 'apache_beam/version.py'
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PerformanceTests_Python #1257
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1257/display/redirect>
------------------------------------------
[...truncated 1.68 KB...]
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3313402882749527139.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4447330106744310889.sh
+ rm -rf .beam_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3105004998856343976.sh
+ rm -rf .perfkit_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1869377136885516741.sh
+ virtualenv .beam_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins416301535208885651.sh
+ virtualenv .perfkit_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2952868600336155078.sh
+ .beam_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.beam_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.beam_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6523644204269782559.sh
+ .perfkit_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.perfkit_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.perfkit_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins300810585250411513.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4829542257814454879.sh
+ .beam_env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.8.2)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.7)
Requirement already satisfied: dill<=0.2.8.2,>=0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.2.8.2)
Requirement already satisfied: grpcio<2,>=1.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.13.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.1.0)
Requirement already satisfied: httplib2<=0.11.3,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.11.3)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.0.0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (4.1.2)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.0)
Requirement already satisfied: pydot<1.3,>=1.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.2.4)
Requirement already satisfied: pytz<=2018.4,>=2018.3 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2018.4)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (3.12)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.6.8)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.11.0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.4)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.2.0)
Requirement already satisfied: future<1.0.0,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.16.0)
Requirement already satisfied: fastavro==0.19.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.19.7)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.5.20)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.90.4)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (7.0.1)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.26.0)
Requirement already satisfied: proto-google-cloud-pubsub-v1==0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: nose>=1.3.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.3.7)
Collecting numpy<2,>=1.14.3 (from apache-beam==2.7.0.dev0)
Using cached https://files.pythonhosted.org/packages/85/51/ba4564ded90e093dbb6adfc3e21f99ae953d9ad56477e1b0d4a93bacf7d3/numpy-1.15.0-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.9.0)
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.7.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.8.0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.0.2)
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.4.2)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.2.2)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (3.4.2)
Requirement already satisfied: setuptools in ./.beam_env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.7.0.dev0) (40.0.0)
Requirement already satisfied: pyparsing>=2.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from pydot<1.3,>=1.2.0->apache-beam==2.7.0.dev0) (2.2.0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (0.14.1)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.7.0.dev0) (1.5.3)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (1.22)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (1.5)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.16)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.11.4)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.0.3)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (1.5.0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (3.8)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (2.1.0)
Installing collected packages: numpy, apache-beam
Found existing installation: numpy 1.13.3
Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'numpy'. No files were found to uninstall.
Found existing installation: apache-beam 2.7.0.dev0
Not uninstalling apache-beam at /home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_Compressed_TextIOIT_HDFS/src/sdks/python, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'apache-beam'. No files were found to uninstall.
Running setup.py develop for apache-beam
Successfully installed apache-beam numpy-1.15.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins824765903847337789.sh
+ .beam_env/bin/python src/sdks/python/setup.py sdist --dist-dir=src/sdks/python/target
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/local/lib/python2.7/site-packages/setuptools/dist.py>:398: UserWarning: Normalizing '2.7.0.dev' to '2.7.0.dev0'
normalized_version,
running sdist
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/gen_protos.py>:49: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
warning: cmd: standard file not found: should have one of README, README.rst, README.txt, README.md
running check
creating apache-beam-2.7.0.dev0
creating apache-beam-2.7.0.dev0/apache_beam.egg-info
copying files to apache-beam-2.7.0.dev0...
copying apache_beam.egg-info/PKG-INFO -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/SOURCES.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/dependency_links.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/entry_points.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/not-zip-safe -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/requires.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/top_level.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
Writing apache-beam-2.7.0.dev0/setup.cfg
creating src/sdks/python/target
Creating tar archive
removing 'apache-beam-2.7.0.dev0' (and everything under it)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6662012102409384417.sh
+ .perfkit_env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14)) (0.2.2)
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15)) (2.10)
Requirement already satisfied: setuptools in ./.perfkit_env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16)) (40.0.0)
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17)) (2.6.0)
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18)) (1.4)
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19)) (3.2.0)
Requirement already satisfied: PyYAML==3.12 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 20)) (3.12)
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21)) (0.8.1)
Requirement already satisfied: numpy==1.13.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22)) (1.13.3)
Requirement already satisfied: functools32 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 23)) (3.2.3.post2)
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24)) (0.5.5)
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25)) (0.3.0)
Requirement already satisfied: timeout-decorator in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 26)) (0.4.0)
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Requirement already satisfied: colorama; extra == "windows" in /home/jenkins/.local/lib/python2.7/site-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17)) (0.3.9)
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.0)
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.11.0)
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.2.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6039706310566305906.sh
+ .perfkit_env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --bigquery_table=beam_performance.pkb_results --k8s_get_retry_count=36 --k8s_get_wait_interval=10 --python_binary=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --beam_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_timeout=1500 --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-07-27 16:27:40,882 0be68e21 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0be68e21/pkb.log>
2018-07-27 16:27:40,882 0be68e21 MainThread INFO PerfKitBenchmarker version: v1.12.0-735-ga3f334a
2018-07-27 16:27:40,883 0be68e21 MainThread INFO Flag values:
--beam_it_timeout=1500
--beam_sdk=python
--k8s_get_wait_interval=10
--python_binary=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python>
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--beam_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src>
--k8s_get_retry_count=36
--benchmarks=beam_integration_benchmark
2018-07-27 16:27:41,228 0be68e21 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-07-27 16:27:41,417 0be68e21 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-07-27 16:27:41,421 0be68e21 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-07-27 16:27:41,422 0be68e21 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/gradlew> clean assemble --stacktrace --info -DintegrationTestRunner=dataflow
2018-07-27 16:33:23,819 0be68e21 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-07-27 16:33:23,915 0be68e21 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/target/apache-beam-2.7.0.dev0.tar.gz>
2018-07-27 16:58:23,923 0be68e21 Thread-3 ERROR IssueCommand timed out after 1500 seconds. Killing command "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/target/apache-beam-2.7.0.dev0.tar.gz".>
2018-07-27 16:58:23,932 0be68e21 MainThread beam_integration_benchmark(1/1) INFO Ran: {<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/target/apache-beam-2.7.0.dev0.tar.gz}> ReturnCode:-9
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/local/lib/python2.7/site-packages/setuptools/dist.py>:398: UserWarning: Normalizing '2.7.0.dev' to '2.7.0.dev0'
normalized_version,
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/gen_protos.py>:49: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py>:54: DeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
print('Found: %s.' % self.build_console_url(pipeline.options))
2018-07-27 16:58:23,933 0be68e21 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 670, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 550, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-07-27 16:58:23,934 0be68e21 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-07-27 16:58:23,937 0be68e21 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 806, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 670, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 550, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-07-27 16:58:23,937 0be68e21 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-07-27 16:58:23,992 0be68e21 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-07-27 16:58:23,993 0be68e21 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0be68e21/pkb.log>
2018-07-27 16:58:23,994 0be68e21 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0be68e21/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PerformanceTests_Python #1256
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1256/display/redirect>
------------------------------------------
[...truncated 1.66 KB...]
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3539873326631408448.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8647486184674439503.sh
+ rm -rf .beam_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1424138257925272438.sh
+ rm -rf .perfkit_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5754435381585259556.sh
+ virtualenv .beam_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2154991725107398410.sh
+ virtualenv .perfkit_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2814333558721416384.sh
+ .beam_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.beam_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.beam_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3696887024533641845.sh
+ .perfkit_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.perfkit_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.perfkit_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4378314190794416191.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8949469932260524169.sh
+ .beam_env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.8.2)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.7)
Requirement already satisfied: dill<=0.2.8.2,>=0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.2.8.2)
Requirement already satisfied: grpcio<2,>=1.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.13.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.1.0)
Requirement already satisfied: httplib2<=0.11.3,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.11.3)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.0.0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (4.1.2)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.0)
Requirement already satisfied: pydot<1.3,>=1.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.2.4)
Requirement already satisfied: pytz<=2018.4,>=2018.3 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2018.4)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (3.12)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.6.8)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.11.0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.4)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.2.0)
Requirement already satisfied: future<1.0.0,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.16.0)
Requirement already satisfied: fastavro==0.19.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.19.7)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.5.20)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.90.4)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (7.0.1)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.26.0)
Requirement already satisfied: proto-google-cloud-pubsub-v1==0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: nose>=1.3.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.3.7)
Collecting numpy<2,>=1.14.3 (from apache-beam==2.7.0.dev0)
Using cached https://files.pythonhosted.org/packages/85/51/ba4564ded90e093dbb6adfc3e21f99ae953d9ad56477e1b0d4a93bacf7d3/numpy-1.15.0-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.9.0)
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.7.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.8.0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.0.2)
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.4.2)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.2.2)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (3.4.2)
Requirement already satisfied: setuptools in ./.beam_env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.7.0.dev0) (40.0.0)
Requirement already satisfied: pyparsing>=2.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from pydot<1.3,>=1.2.0->apache-beam==2.7.0.dev0) (2.2.0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (0.14.1)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.7.0.dev0) (1.5.3)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (1.22)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (1.5)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.16)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.11.4)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.0.3)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (1.5.0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (3.8)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (2.1.0)
Installing collected packages: numpy, apache-beam
Found existing installation: numpy 1.13.3
Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'numpy'. No files were found to uninstall.
Found existing installation: apache-beam 2.7.0.dev0
Not uninstalling apache-beam at /home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_Compressed_TextIOIT_HDFS/src/sdks/python, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'apache-beam'. No files were found to uninstall.
Running setup.py develop for apache-beam
Successfully installed apache-beam numpy-1.15.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8410315169384367550.sh
+ .beam_env/bin/python src/sdks/python/setup.py sdist --dist-dir=src/sdks/python/target
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/local/lib/python2.7/site-packages/setuptools/dist.py>:398: UserWarning: Normalizing '2.7.0.dev' to '2.7.0.dev0'
normalized_version,
running sdist
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/gen_protos.py>:49: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
warning: cmd: standard file not found: should have one of README, README.rst, README.txt, README.md
running check
creating apache-beam-2.7.0.dev0
creating apache-beam-2.7.0.dev0/apache_beam.egg-info
copying files to apache-beam-2.7.0.dev0...
copying apache_beam.egg-info/PKG-INFO -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/SOURCES.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/dependency_links.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/entry_points.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/not-zip-safe -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/requires.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/top_level.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
Writing apache-beam-2.7.0.dev0/setup.cfg
creating src/sdks/python/target
Creating tar archive
removing 'apache-beam-2.7.0.dev0' (and everything under it)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8857752751154454891.sh
+ .perfkit_env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14)) (0.2.2)
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15)) (2.10)
Requirement already satisfied: setuptools in ./.perfkit_env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16)) (40.0.0)
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17)) (2.6.0)
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18)) (1.4)
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19)) (3.2.0)
Requirement already satisfied: PyYAML==3.12 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 20)) (3.12)
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21)) (0.8.1)
Requirement already satisfied: numpy==1.13.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22)) (1.13.3)
Requirement already satisfied: functools32 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 23)) (3.2.3.post2)
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24)) (0.5.5)
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25)) (0.3.0)
Requirement already satisfied: timeout-decorator in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 26)) (0.4.0)
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Requirement already satisfied: colorama; extra == "windows" in /home/jenkins/.local/lib/python2.7/site-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17)) (0.3.9)
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.0)
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.11.0)
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.2.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1099949695612912981.sh
+ .perfkit_env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --bigquery_table=beam_performance.pkb_results --k8s_get_retry_count=36 --k8s_get_wait_interval=10 --python_binary=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --beam_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-07-27 16:04:45,694 d1536932 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/d1536932/pkb.log>
2018-07-27 16:04:45,695 d1536932 MainThread INFO PerfKitBenchmarker version: v1.12.0-735-ga3f334a
2018-07-27 16:04:45,696 d1536932 MainThread INFO Flag values:
--beam_sdk=python
--k8s_get_wait_interval=10
--python_binary=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python>
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--beam_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src>
--k8s_get_retry_count=36
--benchmarks=beam_integration_benchmark
2018-07-27 16:04:46,023 d1536932 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-07-27 16:04:46,220 d1536932 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-07-27 16:04:46,224 d1536932 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-07-27 16:04:46,225 d1536932 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/gradlew> clean assemble --stacktrace --info -DintegrationTestRunner=dataflow
2018-07-27 16:10:33,485 d1536932 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-07-27 16:10:33,584 d1536932 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/target/apache-beam-2.7.0.dev0.tar.gz>
2018-07-27 16:20:33,592 d1536932 Thread-3 ERROR IssueCommand timed out after 600 seconds. Killing command "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/target/apache-beam-2.7.0.dev0.tar.gz".>
2018-07-27 16:20:33,601 d1536932 MainThread beam_integration_benchmark(1/1) INFO Ran: {<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/target/apache-beam-2.7.0.dev0.tar.gz}> ReturnCode:-9
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/local/lib/python2.7/site-packages/setuptools/dist.py>:398: UserWarning: Normalizing '2.7.0.dev' to '2.7.0.dev0'
normalized_version,
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/gen_protos.py>:49: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py>:54: DeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
print('Found: %s.' % self.build_console_url(pipeline.options))
2018-07-27 16:20:33,602 d1536932 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 670, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 550, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-07-27 16:20:33,603 d1536932 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-07-27 16:20:33,605 d1536932 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 806, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 670, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 550, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-07-27 16:20:33,606 d1536932 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-07-27 16:20:33,620 d1536932 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-07-27 16:20:33,621 d1536932 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/d1536932/pkb.log>
2018-07-27 16:20:33,621 d1536932 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/d1536932/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PerformanceTests_Python #1255
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1255/display/redirect?page=changes>
Changes:
[kedin] [SQL] Remove explicit timestamp parsing from PubsubIOJsonTable
[kedin] [SQL] Wrap Pubsub payload into a message schema
[pablo] Change reading sideinput experimental flag into new pattern.
[mariagh] Call assert_that per window via reify_windows flag
[github] [BEAM-4268] Improving the separation between Metrics API and Execution
[aromanenko.dev] [BEAM-4048] Refactor COMBINE mode for reading/writing from/to Pub/Sub
[matthias] Futurize metrics subpackage
[pablo] Adding context switch to operations missing it.
[sidhom] [BEAM-2597] Flink batch ExecutableStage operator
[thw] Fix compile error in RemoteExecutionTest.
[klk] Move Release Guide to committers section
[klk] Move committer section to the bottom (we can find it :-)
[tgroh] Suppress more flaky tests.
[klk] Streamline the contribution section
[tgroh] Add a FlattenRunnerTest for a multi-consumed input
[tgroh] Use ListMultimap when creating Transform Runners
[tgroh] Migrate ImpulseEvaluatorFactory
[tgroh] Migrate FlattenEvaluatorFactoryTest
[tgroh] Migrate GroupByKeyOnly to the RR
[tgroh] Use ModelCoders in LengthPrefixUnknownCoders
[ekirpichov] [BEAM-3515] Portable translation of SplittableProcessKeyed
[ekirpichov] Fixes compile error in FlinkExecutableStageFunctionTest
[ekirpichov] Sickbays flaky test
[robbe.sneyders] Add py27-lint3 test to gradle.build
[klk] Google has scrapped research, now only AI
[iemejia] Fix error-prone warnings for HCatalogIO
[iemejia] Fix error-prone warnings and multiple cleanups based on static analysis
[lcwik] [BEAM-4280] Prevent DirectStreamObserver from blocking indefinitely if
[tgroh] Enable MapFnRunner to provide full WindowedValues
[tgroh] Implement AssignWindows in the Java SDK Harness
[tgroh] Add a static constructor to GrpcStateService
[tgroh] Log when Handler Delegation Throws
[kirpichov] fromQuery().withTemplateCompatibility() should do the same thing as
[klk] Temporarily suppress validation of ai.google links
[iemejia] Fix error-prone warnings for HBaseIO
[timrobertson100] Fix error-prone warnings for SolrIO
[timrobertson100] Fix error-prone warnings for ElasticsearchIO
[timrobertson100] Fix error-prone and some static analysis warnings in
[mergebot] Add link to portability support sheet
[timrobertson100] Fix error-prone and some static analysis warnings in XmlIO
[pablo] Fix error-prone warnings for file-based-io-tests
[kedin] [SQL] Add integration test for PubsubIO JSON -> SQL
[kedin] [SQL] Fix typos, add rewindowing and retriggering to TestPubsubSignal
[apilloud] [BEAM-3983] Add utils for converting to BigQuery types
[apilloud] [BEAM-3983][SQL] Add BigQuery table provider
[apilloud] [BEAM-4248] Copy enums from com.google.cloud
[kedin] Add SQL postCommit to main javaPostCommit
[wcn] Fix log statement to mark sink as generator of info.
[timrobertson100] Fix error-prone warnings for io/hadoop-common
[mairbek] Make Cloud Spanner transaction id public. Allows to build non-library
[daniel.o.programmer] [BEAM-4277] Adding unshaded jars used by Dataflow to poms.
[tgroh] Revert "Log when Handler Delegation Throws"
[tgroh] Log when Handler Delegation Throws
[tgroh] squash! Revert "Log when Handler Delegation Throws"
[pablo] [BEAM-4249] Fix failing validates container test
[valentyn] Add a Python 3 entry to the list of ongoing projects.
[lcwik] [BEAM-4127] Flink streaming translation for portable pipeline. (#5358)
[herohde] [BEAM-3826] Add missing package comments
[kedin] [SQL] Add support for DQL to PubsubJsonTable
[mergebot] Add Go SDK quickstart and minimal documentation page
[mergebot] Add Go to the Wordcount example
[mergebot] Fix mailing list links and docker image
[pablo] Add a dependency on future.
[timrobertson100] Fix error-prone and some static analysis warnings in
[lukasz.gajowy] [BEAM-4140] Utilize beam_location perfkit flag
[github] Update index.md
[kpeterson] Adding a ValueProvider interface.
[klk] Reorganize SQL documentation into sections
[swegner] Upgrade Byte Buddy and FindBugs annotations dependencies.
[swegner] Fix ErrorProne warnings
[swegner] Remove isTriggerSpecified from WindowingStategy equality checks.
[kpeterson] Fixing checkstyle error.
[lcwik] Add window mapping transform.
[lcwik] Use the coder from the side input PCollection definition by removing the
[lcwik] Remove garbage collection from PValueCache
[pablo] Fixing incorrect assert in dependency_test.py
[kedin] [SQL] Add integration test for Pubsub Json Table DLQ
[github] [BEAM-4300] Fix ValidatesRunner tests in Python: run with same mechanism
[tgroh] Add an IdGenerator class to java/fn-execution
[ringwalt] Improve the TaggedOutput error when a tuple is passed as a tag.
[lukasz.gajowy] [BEAM-214] Add ParquetIO
[lukasz.gajowy] [BEAM-214] Add gradle build in ParquetIO
[lukasz.gajowy] [BEAM-214] Refactore the reads to leverage FileIO
[lukasz.gajowy] [BEAM-214] Use Parquet 1.10.0 version in ParquetIO.Sink
[lukasz.gajowy] [BEAM-214] Add ParquetIOIT
[lukasz.gajowy] [BEAM-214] Update ReadFn to an non deprecated, newest Parquet API
[lukasz.gajowy] [BEAM-214] Simplify the IO by removing redundant PTransforms
[lukasz.gajowy] [BEAM-214] Implement writing slices of bytes in BeamOutputStream
[apilloud] [BEAM-3733] [SQL] Use saffron.properties for charset
[pablo] [BEAM-4331] Disable integrationTest task caching
[tgroh] Add a new DockerEnvironmentFactory Constructor
[pablo] Fixing Seed Job
[samuelw] [BEAM-3776] Fix issue with merging late windows where a watermark hold
[pablo] Add future dependency to the Dockerfile
[apilloud] [SQL] Add jdbc:beam:
[apilloud] [SQL] Use CalciteConnection as context
[axelmagn] Create jobsubmission package for job management.
[axelmagn] Add Job Invocation Interfaces
[axelmagn] Add in-memory implementation of JobService
[pablo] Remove stale TODO comments for adding archetype to java precommit
[pablo] Upgrade Gradle shadowJar plugin to latest.
[github] Mention adding a reviewer in PR template. (#5397)
[kedin] Fix Maven build
[rober] Update generated protos in Go SDK
[herohde] Invoke Go Dataflow integration tests from post-commit
[herohde] Add the Go SDK to the README
[timrobertson100] [BEAM-4342] Enforce ErrorProne analysis in hadoop IO, and static
[timrobertson100] [BEAM-4340] Enforce ErrorProne analysis in file-based-io-tests
[timrobertson100] [BEAM-4341] Enforce ErrorProne analysis in google-cloud-platform IO
[timrobertson100] [BEAM-4335] Enforce ErrorProne analysis in amazon-web-services IO
[timrobertson100] [BEAM-4339] Enforce ErrorProne analysis in elasticsearch IO
[timrobertson100] [BEAM-4355] Enforce ErrorProne analysis in XML IO
[timrobertson100] [BEAM-4338] Enforce ErrorProne analysis in common IO
[timrobertson100] [BEAM-4337] Enforce ErrorProne analysis in cassandra IO
[timrobertson100] [BEAM-4355] Reduces scope of findbugs annotations to build time only
[timrobertson100] [BEAM-4353] Enforce ErrorProne analysis in solr IO
[timrobertson100] [BEAM-4345] Enforce ErrorProne analysis in JDBC IO
[timrobertson100] [BEAM-4336] Enforce ErrorProne analysis in AMQP IO
[timrobertson100] [BEAM-4347] Enforce ErrorProne analysis in kafka IO
[dariusz.aniszewski] [BEAM-4041] Increase timeout for getting K8s LoadBalancer external IP
[timrobertson100] [BEAM-4346] Enforce ErrorProne analysis in JMS IO
[timrobertson100] [BEAM-4352] Enforce ErrorProne analysis in Redis IO
[lcwik] [BEAM-4271] Support side inputs for ExecutableStage and provide runner
[daniel.o.programmer] [BEAM-2937] Add new Combine URNs.
[tgroh] Update worker_id Documentation
[robbe.sneyders] Futurize internal subpackage
[dariusz.aniszewski] fixed typo
[mariagh] Add validation for streaming wordcount (streaming_wordcount_debugging)
[pablo] Increasing the concurrent test execution count (#5408)
[tgroh] Add a Default RemoteEnvironment Implementation
[sidhom] Require explicit IdGenerator for all DockerEnvironmentFactory
[sidhom] [BEAM-4148] Add pluggable UrlFactory to ServerFactory
[sidhom] [BEAM-4267] Docker-based JobBundleFactory
[tgroh] Add a Window-only State Context
[lcwik] Address Ben's comments.
[github] Make hash function in Coder base class more conservative.
[github] updated right documentation for HCatalogIO.write
[sidhom] Fix DataStreamsTest checkstyle error
[ankurgoenka] Updating python container version to beam-master-20180518
[wcn] Fix the test to properly run.
[ankurgoenka] Reverting fnapi container version
[ankurgoenka] Copyting dependency.py and dependency_test.py
[ankurgoenka] reformatting code
[ankurgoenka] Introducing classes
[ankurgoenka] gruping public and non public methods
[ankurgoenka] Renaming method and updating reference for file_copy and file_download
[ankurgoenka] Removing Google specific code and unused code in stager.
[ankurgoenka] Removing reference to Google/GCS/Dataflow and restructuring a bit of
[ankurgoenka] Fixing Stager Tests
[ankurgoenka] Making dependency.py use stager.py and fixing dependency_test.py test
[ankurgoenka] Moving stager to portability
[ankurgoenka] Removing stage_job_resources from dependencies.py and other minor
[ankurgoenka] Merging filehandler in stager
[ankurgoenka] renaming GCSStager to _ParameterizedStager
[ankurgoenka] Making a few methods static and moving code around
[ankurgoenka] Applying changes from rebase
[szewinho] HDFS large cluster configuration. Jenkins job updated to use large
[ankurgoenka] Fixing lint
[iemejia] [BEAM-4343] Enforce ErrorProne analysis in HBaseIO
[iemejia] [BEAM-4344] Enforce ErrorProne analysis in the HCatalogIO
[tgroh] Add a BundleFactoryOutputReceiverFactory
[tgroh] Add a DirectJobBundleFactory
[tgroh] Fix Copyright Headers in sdks/java/fn-execution
[yifanzou] rollback BEAM-3256 add archetype testing to gradle build PreCommits
[ankurgoenka] Adding ArtifactStagingFileHandler to push files to staging server
[ankurgoenka] Adding filesystem based artifact service to ULR job service
[ankurgoenka] Adding context manager to ArtifactStagingFileHandler
[ankurgoenka] Converting ArtifactServiceFileHandler to PortableStager based on
[ankurgoenka] Simplifying numeric constants
[ekirpichov] [BEAM-4375] Replaces usage of JUnit runners that do not work well with
[matthias] Futurize options subpackage
[szewinho] [BEAM-4065] Basic performance tests analysis added.
[szewinho] Fixed typo in env name, added unit tests for create report logic, slack
[szewinho] Jenkins job will run tests before executing script. Extended testing of
[timrobertson100] [BEAM-4347] Improve ErrorProne analysis in kafka IO
[timrobertson100] [BEAM-4348] Enforce ErrorProne analysis in kinesis IO
[timrobertson100] [BEAM-4349] Enforce ErrorProne analysis in mongodb IO
[timrobertson100] [BEAM-4351] Enforce ErrorProne analysis in mqtt IO
[lcwik] [BEAM-4060] Add the pom.xml and pom.properties file to the artifact when
[github] [BEAM-1755] Add a directory with build-specific scripts to clear up the
[ankurgoenka] Logging and naming changes
[ankurgoenka] Enhancing test cases
[tgroh] Add an InProcessEnvironmentFactory
[timrobertson100] [BEAM-4348] Removes double-checked locking to aid readability
[ankurgoenka] Updating the dostring
[tgroh] Update InProcessManagedChannelFactory
[tgroh] Add a RemoteStageEvaluatorFactory
[timrobertson100] [BEAM-4354] Enforce ErrorProne analysis in tika IO
[iemejia] Fix build with Maven (Again!)
[lukasz.gajowy] [BEAM-214] Apply reviewer's suggestions
[timrobertson100] [BEAM-4347] Minor changes following PR review
[timrobertson100] [BEAM-4347] Correction of typo following PR review
[tgroh] Use WindowOnlyContext in ReduceFnContextFactory
[tgroh] Update GroupAlsoByWindowEvaluatorFactory
[tgroh] Move InProcessEnvironmentFactory to java-fn-execution
[klk] Autoformat sdks/java/extensions/sql
[klk] Add spotless config to global gradle file
[klk] Activate spotless for SQL module
[boyuanz] Solve BEAM-4393: Missing copyright notices for shaded packages
[iemejia] Remove repeated GCPIO dependency in SQL module (Again!)
[iemejia] Enforce error-prone in ParquetIO, fix missing javadoc and refine deps
[katarzyna.kucharczyk] Replaced created datetime to jenkins build id in kubernetes namespace.
[schroederc] Use ElementEncoders for beam.Create
[boyuanz] Change bulid rules of sdks/java/harness to make harness as a uber-jar
[swegner] Remove DoFnTester from testing guide.
[robertwb] Slowly ramp up wait time in state sampler.
[tgroh] Move the Local Artifact Services to the DirectRunner
[tgroh] Migrate the ReferenceRunnerJobServer to runners-direct
[tgroh] Fix License Headers
[schroederc] Add test case for proto oneof fields in create_test.go
[tgroh] Add a test to show Flatten Execution
[apilloud] [SQL] Fix transitive dependencies
[apilloud] [SQL] Run tests against actual jar
[boyuanz] [BEAM-4393]: Copy root repo's LICENSE & NOTICE into shadowJar
[herohde] Tweak Go integration test driver
[tgroh] Update Docker Command
[iemejia] [BEAM-4306] Enforce ErrorProne analysis in apex runner
[daniel.o.programmer] [BEAM-4406] Updating Java portable environment major version.
[daniel.o.programmer] [BEAM-4406] Updating Python portable environment major version.
[ehudm] Add a custom _url_dirname for local filesystems.
[tgroh] Add a Portable TransformEvaluatorRegistry
[tgroh] Consistent Parameter Order in Evaluator Factories
[tgroh] Make ImmutableExecutableStage Public
[mergebot] Fix typo in becoming-a-committer docs
[iemejia] Fix broken build (again third time this week yay!)
[timrobertson100] [BEAM-4361] Document usage of HBase TableSnapshotInputFormat
[jbonofre] Fix Maven parent version in ParquetIO
[iemejia] [BEAM-4312] Enforce ErrorProne analysis in gearpump runner
[mergebot] Add Apache Kudu IO to the planned IOs table
[mergebot] Add next Java LTS to the WIP section of the contribution guide
[mergebot] Add PubSubIO to Python IO table row
[iemejia] [BEAM-4307] Enforce ErrorProne analysis in
[mergebot] Change ParquetIO status in built-in I/O section
[iemejia] [BEAM-4308] Enforce ErrorProne analysis in runners-core-java
[ammons] Say that merging windows are not supported in Distinct.java.
[lcwik] [BEAM-4411] Update Jackson version to latest to pick up security fixes.
[kenn] Implement UNNEST operator for SQL
[iemejia] [BEAM-4317] Enforce ErrorProne analysis in runners/reference/java
[iemejia] Refine dependencies, make new ones explicit and minor maven plugins
[iemejia] [BEAM-4323] Enforce ErrorProne analysis in sketching extensions
[iemejia] [BEAM-4324] Enforce ErrorProne analysis in sorter extensions
[iemejia] [BEAM-4321] Enforce ErrorProne analysis in join-library extensions
[kenn] Add equals, hashCode, and structuralValue to Nexmark model objects
[thw] [BEAM-4297] Streaming executable stage translation and operator for
[thw] [BEAM-4297] Add serialization test and make output mapping stable.
[lcwik] Add link to design docs and template to the contribution guide.
[timrobertson100] [BEAM-4389] Enable partial updates in ElasticsearchIO
[kenn] Force load of Calcite JDBC driver (avoids internal error)
[kenn] Explicitly load Beam JDBC driver before test
[iemejia] [BEAM-3813] Support (de)serialization of S3 encryption options via JSON
[pablo] [BEAM-4426]: Addressed DataflowDistributionAccumulatorTest failure on
[aaltay] Support installing Beam SDK from a wheel distribution in SDK containers.
[cademarkegard] fix errorprone annotations and enable failOnWarning
[lukasz.gajowy] Add Performance Testing to "Work in Progress" section
[iemejia] Update postgres version to 42.2.2 (Java 8)
[coheigea] Update README to include virtualenv in the build instructions
[iemejia] [BEAM-4356] Enforce ErrorProne analysis in nexmark
[kenn] Finish RowType -> Schema rename
[kenn] Add convenience aliases for nullary type constructors
[kenn] Add MAP field support to Schema.Builder
[kenn] Add static factories for array, row, map types
[kenn] Schema fields are non-null by default
[kenn] TestUtils.RowsBuilder to use FieldType instead of TypeName
[kenn] Clean up CalciteUtils conversions a bit
[pablo] enable failOnWarning for build-tools
[apilloud] Run everything with shadowJar
[apilloud] guava is on kinesis public API
[apilloud] Fix direct runner shadow config
[kenn] Add more field conveniences to Schema.Builder
[kenn] Remove extraneous RowSqlTypes
[pablo] [BEAM-4322] Enforce ErrorProne analysis in protobuf extensions project
[apilloud] Run test on shadowJar on a flag
[kenn] Remove Schema.TypeName.type()
[kenn] Remove unsafe builder methods from FieldType
[apilloud] [SQL] Build mega jar for JDBC
[apilloud] [SQL] JDBC Class Loader IT
[ankurgoenka] proto changes to support artifact_staging_id
[ankurgoenka] Doc changes
[ankurgoenka] Renaming artifact_staging_id to staging_session_token
[ankurgoenka] Fixing python code for staging_session_token proto changes
[marek.simunek] [BEAM-4368] add euphoria java 8 dsl documentation section
[coheigea] Avoid fully qualified class names, when there is an existing (static)
[coheigea] Fixing ErrorProne MissingOverrides warnings
[kedin] [SQL] Fix PubsubsJsonIT
[apilloud] [SQL] Init compiler factory with JDBC class loader
[pablo] clean generated files prior to lint
[ankurgoenka] Fixing java code for staging_session_token proto changes
[ankurgoenka] Go proto update
[amaliujia] Add LIKE operator to Beam SQL.
[kedin] [SQL] Replace planner.compilePipeline() with sqlEnv.parseQuery()
[kedin] [SQL] Add sqlEnv.executeDdl()
[kedin] [SQL] Make planner package-private
[kedin] [SQL] Add factory methods to BeamSqlEnv
[kedin] [SQL] Rename ReadOnlyTableProvider
[coheigea] Fixing some ErrorProne warnings
[github] print() is a function in Python 3
[cademarkegard] [BEAM-4303] Enforce ErrorProne analysis in examples project
[boyuanz] Address BEAM-4328: :beam-sdks-java-io-google-cloud-platform:test failure
[kedin] [SQL] Wrap SQL parsing exceptions in ParseException
[tgroh] Add an abstraction for State and Timers
[tgroh] DirectRunner Cleanups
[tgroh] Link up the Portable DirectRunner
[tgroh] Reuse ID Generators across Environments
[tgroh] Use the Current Users Container as the Environment
[aromanenko.dev] [BEAM-4421] Fix for issue with reading s3 files using ParquetIO
[github] Fix flaky comparison in log_handler_test.py
[tgroh] Use the Current Users Container as the Environment
[tgroh] Enable Docker in the PortableDirectRunner
[tgroh] Move DirectJobBundleFactory to fn-execution
[tgroh] Cleanups to the PortableDirectRunner
[tgroh] Cleanups and Wiring for the ReferenceRunnerJobService
[amyrvold] [BEAM-4253] Update dataflow worker to fix ParDoTest fails in Dataflow
[kedin] [SQL] Add ParseException to sqlEnv.explain()
[kedin] [SQL] Add integration tests for BigQuery writes
[kedin] Add TestBigQuery rule
[tgroh] Use an UnsupportedArtifactRetrievalService
[altay] Move assert_that's new keyword argument to the end
[jhsueh] [BEAM-4320] Enforce ErrorProne analysis in jackson extensions project
[jiangkai] introduce optimized logic plan
[jiangkai] update
[jiangkai] fix unit tests failed
[iemejia] [BEAM-4310] Enforce ErrorProne analysis in
[jiangkai] address comments
[relax] Now that Dataflow is updated, we can finish updating tests and usages to
[szewinho] [BEAM-3214] Add integration test for HBaseIO.
[iemejia] [BEAM-4137] Remove MongoDB specific options from
[ankurgoenka] Code review comments fixes
[ajamato] Add new metrics protos based on s.apache.org/beam-fn-api-metrics
[jiangkai] separate rules into sets
[amaliujia] Address comments: 1. Use SqlFunction.like to implement like operator. 2.
[apilloud] [SQL] Use AutoService for jdbc
[apilloud] [SQL] AutoService for TableProvider
[apilloud] [SQL] Don't cache IT results
[apilloud] [SQL] Test that TextIO and Direct Runner work
[apilloud] [SQL] Add context class loader hack
[rober] [BEAM-4276] Add combiner lifting support to Go SDK
[rober] fixup! Address comments.
[herohde] [BEAM-4474] Ensure unbounded Dataflow jobs are submitted as streaming
[herohde] Fix build break due to artifact staging change
[relax] Add a test with ReferenceRunner
[aromanenko.dev] Added design docs page
[aromanenko.dev] Added first intro sentence.
[aromanenko.dev] Moved one doc to other section and added several new docs
[aromanenko.dev] Added more docs
[github] [BEAM-4480] Fixed deprecated method invoking
[lcwik] [BEAM-4481] Remove duplicate definitions of dependencies.
[herohde] [BEAM-4475] Add container boot code to Go precommit
[herohde] [BEAM-4475] Add Go submodule build to Go precommit
[axelmagn] Fix ProvisionInfo in DockerJobBundleFactory
[github] Improve default value for experiments set in RuntimeValueProvider
[cademarkegard] [BEAM-4326] Enforce ErrorProne analysis in the fn-execution project
[alan] [BEAM-4423] Mark pull requests stale after 60 days; close 7 days after
[jbonofre] Moving to 2.6.0-SNAPSHOT on master branch
[lukasz.gajowy] [BEAM-4415] Add jenkins job for HDFS ParquetIOIT
[lukasz.gajowy] [BEAM-4137] Extract file-related options from IOTestPipelineOptions
[lukasz.gajowy] [BEAM-4137] Extract postgres related options from IOTestPipelineOptions
[timrobertson100] [BEAM-4311] Enforce ErrorProne analysis in Flink runner
[github] [BEAM-4517] Add maven status in README
[jiangkai] add expression boolean casting
[coheigea] Removing extraneous whitespace around the equals operator
[katarzyna.kucharczyk] Changed a namespace to parsed job name with current job number.
[jiangkai] add boolean type for agg function
[jiangkai] support EXISTS operator
[github] Update Environments.java
[github] Remove unneeded collection import.
[github] Combine immutability type fixes. (#3)
[github] fixup!
[lcwik] BEAM-3876 avoid NPE if checkpoint is null in an unbounded source
[lcwik] testing npe fix and exception rethrow in unit tests
[lcwik] Update https://github.com/apache/beam/pull/4894 to correspond with
[github] [BEAM-4276] Add missing scope line
[jiangkai] explain plan unit tests
[lcwik] [BEAM-4517] Add PyPI status badge.
[aljoscha.krettek] [BEAM-3905] Update Flink Runner to Flink 1.5.0
[kedin] [SQL] Manually read rows in TestBigQuery
[robertwb] [BEAM-3981] Cleanup of coders futurization.
[robertwb] Use ByteCountingOutputStream for estimate_size.
[kedin] Add polling assertion support to TestBigQuery
[kedin] [SQL] Add Pubsub to BigQuery E2E integration test
[sidhom] Add reference to ArtifactSourcePool implementation
[sidhom] Clean up FlinkExecutableStageFunction and tests
[sidhom] [BEAM-4523] Implement batch flink executable stage context
[kedin] [SQL] Delete BeamSql
[kedin] [SQL] Rename QueryTransform to SqlTransform
[mergebot] Add SQL area to ongoing projects
[robertwb] Reject unknown arguments in WindowInto.
[mergebot] [BEAM-4488] How to install Python SDK from source [BEAM-4489] Document
[mergebot] More changes
[mergebot] updates after review
[mergebot] Updating release guide to use Gradle commands.
[mergebot] Addressing comments. Improving commands.
[mergebot] Addressing comments. Improving commands.
[mergebot] Update release guide for publishing source zip from GitHub
[mergebot] Addressing TODOs
[mergebot] Address comments.
[mergebot] Improving.
[mergebot] Add more updates
[mergebot] More updates
[mergebot] Addressed Alan's comments
[mergebot] Add instructions about how to stage python binaries
[mergebot] copy renamed zi[ file to sdks/python/staging subdir
[mergebot] Update gradle publish with --no-parallel --no-daemon
[kenn] Suppress immutability warning caused by LoadingCache in enum instance
[kenn] Make BeamSqlExpression for operators, use it for string operators
[ekirpichov] Add retrieval_token to GetManifestRequest
[ekirpichov] Refactor code that accesses staging_token
[ekirpichov] Regenerate go protobufs
[kenn] Replace project and filter with calc
[kenn] Disable CalcRemoveRule
[szewinho] Improve consistency of hadoop and hbase scripts, fix shellcheck issues
[iemejia] Fix build (missed auto service for SQL) and model-fn-execution on Flink
[iemejia] [BEAM-4318] Enforce ErrorProne analysis in Spark runner
[wcn] Encode position ids.
[robertwb] Explain why module namespace is modified in Cython codepath.
[apilloud] [SQL] Make getString more generic
[ankurgoenka] Beam File System based Artifact Staging Service
[ankurgoenka] BeamFileSystemArtifactStagingService Tests
[ankurgoenka] Enhancing test case and error handling.
[ankurgoenka] Enhancing testcases and incorporating review comments.
[ankurgoenka] Review comments Fix
[ankurgoenka] Rebasing
[apilloud] [SQL] Support unquoted table types
[herohde] Remove new Go precommit dependencies due to failures
[jiangkai] address conflicts and comments
[jiangkai] remove volcano plan function
[klk] [SQL] Separate assertions for each expression in ExpressionChecker
[klk] Remove StringOperatorsTest in favor of SQL syntax tests
[mergebot] Add Python Kafka to list of in-progress I/O transforms.
[pablo] Upgrade Gradle to version 4.8
[ehudm] Remove with_attributes keyword from ReadFromPubSub.
[xiliu] [BEAM-3079] Update samza-runner with more features and improvements
[xiliu] Add missing apache headers
[swegner] Use new DSL methods for generated POM metadata
[lcwik] [BEAM-4540] Migrate junit/hamcrest to provided scope.
[swegner] [BEAM-925] Fix FindBugs violations in Dataflow Runner and re-enable
[swegner] [BEAM-4313] Fix ErrorProne warnings and enable validation in Dataflow
[ankurgoenka] Followup PR for #5591
[swegner] Add support for pre-commit trigger paths
[swegner] Set path triggers for existing pre-commit test jobs
[rober] Add graph.Fn encoding functions.
[robinyq] Defer calling formatTimestamp() to achieve better performance
[ccy] [BEAM-4549] Use per-pipeline unique ids for side inputs in
[altay] Explain experimental/deprecated from users perspective.
[jiangkai] call SqlFunctions in numeric type conversion
[github] Update Dockerfile for new setup.py changes.
[robbe.sneyders] Futurize utils subpackage
[iemejia] [BEAM-4551] Update spark runner to Spark version 2.3.1
[iemejia] Fix maven build error on sdks/java/io/google-cloud-platform module
[robinyq] Throws exception directly instead of calling checkState()
[github] Remove GPL findbugs dependency (#5609)
[lcwik] Clean-up BeamFnDataBufferingOutboundObserver since Dataflow has migrated
[jiangkai] unit test for BuildExpression literal casting
[aaltay] Futurize portability subpackage (#5385)
[altay] Futurize unpackaged files
[altay] resolved six.string_types equivalency
[altay] Futurize testing subpackage
[altay] Futurize tools subpackage
[altay] Remove old_div
[swegner] Revert "Merge pull request #5611: [BEAM-4445] Filter pre-commit
[robertwb] [BEAM-4546] Implement hot key fanout for combiners.
[robertwb] Use discaring mode for first level of combine.
[echauchot] [BEAM-4283] Export nexmark query execution times to BigQuery
[echauchot] [BEAM-4283] Make BigQueryIO Services classes public and experimental and
[echauchot] [BEAM-4283] Fix PerfsToBigQueryTest and serialization issue in
[echauchot] Add Etienne Chauchot's ongoing work to the contribute page.
[fvoichick] Fixed link to nonexistent IsBounded.apply method
[fvoichick] Fixed documentation of default window size in example code
[fvoichick] Fixed inappropriate Javadoc comments
[aaltay] [BEAM-4333] Add integration tests for python mobile games (#5630)
[jiangkai] ignore explain unit test
[kedin] [SQL] Add INSERT tests to JdbcDriverTest
[kedin] [SQL] Add complex type test to JdbcDriverTest
[kedin] [SQL] Make field names case-sensitive
[robertwb] Suppress warning.
[robertwb] Implement BeamFileSystemArtifactSource
[robertwb] Add job submission capabilities to Flink runner.
[robertwb] Update to new JobInvoker/Invocation interface.
[robertwb] Remove references to Flink distributed file cache.
[robertwb] A bit more documentation.
[robertwb] Address comments on Flink code.
[kirpichov] Removed some dead code, fixed some warnings
[kirpichov] Adds tracing of watermarks in direct runner
[kirpichov] Fixes typo in class name
[kirpichov] Simplifies GBK expansion code a bit
[kirpichov] Fixes bug: Processes values arriving for a GBK key sequentially
[kirpichov] Introduces BundleProgressHandler.
[swegner] Fix ErrorProne violations in direct runner and enable failOnWarnings
[robertwb] [BEAM-4565] Fix hot key fanout in the face of combiner lifting.
[robertwb] Remove trailing whitespace.
[kirpichov] [BEAM-4473,BEAM-4281] Makes ReferenceRunner use a Direct stream observer
[ccy] [BEAM-4549] Revert #5623
[kirpichov] [BEAM-4291] Implements distributed artifact retrieval
[kirpichov] Deletes ArtifactSource
[thw] Fix Flink runner Maven build.
[ccy] [BEAM-4549] Fix side inputs for streaming pipelines on Dataflow
[mergebot] [BEAM-2852] Update Nexmark documentation with Kafka support
[mergebot] [BEAM-4479] Fixed document for Coder
[swegner] Fix ErrorProne violations in fn harness and enable failOnWarnings
[swegner] fixup! Fix ErrorProne violations in fn harness and enable failOnWarnings
[pablo] Improving state sampler Cython annotations.
[pablo] Adding msecs tracking to side inputs iterator
[pablo] Adding the sampling thread to side input microbench.
[pablo] Adding flag to mask change
[pablo] Fixing all lint raising-format-tuple warnings.
[github] Make build_rules.gradle version of shadow match build.gradle
[swegner] Temporarily exclude website sources from RAT
[swegner] fixup! Fix ErrorProne violations in fn harness and enable failOnWarnings
[ehudm] Increase Jenkins build retention from 14 to 30 days.
[ankurgoenka] Fixing concurrency bug in BeamFileSystemArtifactServicesTest
[iemejia] Fix Maven build 7
[iemejia] [BEAM-4570] Use pagination in
[thw] Fix ValidatesRunner_Apex #5604
[iemejia] [BEAM-4399] Get Cassandra options out of IOTestPipelineOptions
[iemejia] [BEAM-4399] Refactor CassandraIOIT to be writeThenRead
[iemejia] [BEAM-4399] Update Cassandra driver to version 3.5.0
[kenn] Fix name of heuristic JAVA_CLASS_TO_TYPENAME
[kenn] [SQL] Allow explicit return type in ExpressionChecker
[kenn] DSL-level tests for standard operators
[kenn] Add test that all supported operators are tested
[kenn] [SQL] Make parse failure slightly more actionable
[pablo] Upgrading python dependency on httplib2
[kenn] Fix case sensitivity in JdbcDriverTest
[ehudm] Jupyter notebook for monitoring precommit job times
[lcwik] [BEAM-4576] Add hamcrest dependencies to this scope that was moved to
[mergebot] Add link to @RequiresStableInput design doc
[matthias] Futurize examples subpackage
[apilloud] [SQL] Add parse code for SET
[apilloud] [SQL] Plumb PipelineOptions through BeamRelNode
[apilloud] [SQL] Plumb PipelineOptions through BeamSqlCli
[apilloud] [SQL] Remove unused BeamIOType
[ccy] [BEAM-4582] Fix streaming Create transform on Dataflow
[kirpichov] Propagates artifact retrieval token in Flink runner and to the Java
[ankurgoenka] load beam fileSystems in FnHarness
[axelmagn] Implement new staging token contract in java SDK
[robertwb] Add Impulse primitive to Python SDK
[robertwb] Expand Read to Impulse | ParDo(Split) | ParDo(Read) in Python SDK
[robertwb] Fix default docker image name.
[kirpichov] Add an AddHarnessIdInterceptor
[kirpichov] Remove the runners/reference InProcessManagedChannelFactory
[kirpichov] Allow ManagedChannelFactories to attach Interceptors
[kirpichov] Populate and Use Harness IDs in the Java Harness
[apilloud] [SQL] Set application name to BeamSql
[apilloud] [SQL] Correctly represent pipeline
[apilloud] [SQL] Cleanup table IO functions
[apilloud] [SQL] Simplify overly verbose names
[apilloud] [SQL] Add each Calcite node to Beam graph once
[apilloud] [SQL] Seperate parse from pipeline construction
[swegner] fixup! Fix ErrorProne violations in fn harness and enable failOnWarnings
[kirpichov] Fixes checkstyle
[chamikara] Fixes broken links for downloading source releases.
[lcwik] [BEAM-4592] Make Dataflow understand kind:varint as a well known since
[chamikara] Fixes null pointer exception on empty pubsub message data (#5016)
[kirpichov] A couple more trivial changes extracted from #5662
[lcwik] Update Dataflow container images
[katarzyna.kucharczyk] Created static retry method which accepts retriable method, amount of
[robertwb] Update impulse-based read.
[kenn] Make BeamRelNode take a PCollectionList<Row> always
[rober] Fix build break from PR 5676, which modified method signatures.
[migryz] Implemented working prototype for PostCommit jobs
[migryz] Update post commit jobs to utilize new helper methods
[ankurgoenka] Fix cache references in portable flink runner
[robertwb] [BEAM-4150] Extract IO coders from the RemoteGrpcPort.
[pablo] Adding ignore parameters for gitattributes
[swegner] Remove submodule
[robertwb] Fix and test global combine with fanout.
[migryz] Fix typo in _ghprb suffix
[migryz] Update suffix to be PullRequest
[ccy] Revert #5689 to fix build
[ankurgoenka] Remove unused import
[github] Make suffix shorter
[kirpichov] Make ImmutableExecutableStage constructors public
[kirpichov] [BEAM-4285] Implement Flink batch side input handler
[ankurgoenka] Fixing key lookup in FlinkBatchSideInputHandlerFactory.SideInputHandler
[ekirpichov] Introduces PipelineValidator that checks the well-formedness of a
[cademarkegard] [BEAM-4325] Enforce ErrorProne analysis in the SQL project
[echauchot] [BEAM-4283] Fix naming of the BigQuery fields
[coheigea] Removing some null checks, where we already know that the variable in
[iemejia] [BEAM-3314] Set correctly host and port on RedisIO
[iemejia] [BEAM-3314] Fix error-prone warnings and add extra test for Read
[matthias] Futurize direct runner
[matthias] Futurize dataflow,experimental,job,portability and test subpackages of
[matthias] Futurize complete runners subpackage
[matthias] Cleanup futurize runners
[lcwik] Revert "Revert #5689 to fix build"
[github] Add migration notice to apache/beam website README.me
[robertwb] Always consider Read to be a primitive for dataflow pipelines.
[migryz] Point gearpump to master
[robertwb] [BEAM-4605] Remove subtransform of GBK.
[robbe.sneyders] Futurize typehints subpackage
[kenn] Migrate build_rules.gradle to buildSrc plugin
[amaliujia] support unbounded limit.
[aaltay] Fix backlog bytes reported by CountingSource. (#5719)
[mairbek] Fixed utf8 encoding in Cloud Spanner mutation encoder
[robertwb] Don't require a staged SDK for the Python portable worker.
[pablo] Enabling gradle build to receive extra buildscript repositories
[kenn] Remove build_rules.gradle
[daniel.o.programmer] [BEAM-2937] Comment Combine Grouped Values URN.
[daniel.o.programmer] [BEAM-2973] Moving CombineValues URN to different enum.
[robertwb] Add back old code path for dataflow.
[klk] Fix errors in BeamModulePlugin perf test section
[mergebot] Adds a dependencies guide to Beam Website
[iemejia] Fix maven compile for the 8th time (yay!)
[iemejia] Fix errorprone warnings on extensions/google-cloud-platform-core module
[iemejia] Enable errorprone globally
[klk] Fix Gearpump validatesRunner gradle config
[robertwb] Stage empty manifest if a staging session is given.
[migryz] [BEAM-4559] Add latest post-commit tests status to GH PR
[sidhom] Add registrars for the portable runner
[sidhom] Add endpoint logging for PortableRunner
[github] dataflow_runner.py: use the pbegin's pipeline
[github] materialize_test.go: delint: non-idiomatic block
[klk] Fix docker gated code paths
[altay] Add instructions about how to perform release validations
[ehudm] Parallelize Java precommits
[scott] Spread jenkins load by randomizing daily cron schedule
[amaliujia] logical operator tests
[kirpichov] Adds support for SDF in ULR and the Java SDK.
[klk] Allow specifying delimiter in TextIO
[robbe.sneyders] Address PR comments
[klk] Migrate some date expressions to operators
[klk] Use label "stale" when closing stale PRs
[kenn] Add support for "lines" format text tables
[kenn] Beam SQL shell prompt is now 'BeamSQL>'
[kenn] Override SqlLine.properties to customize welcome and help messages
[pablo] Improving documentation for Metrics results
[xiliu] Finalize Samza Runner with gradle and findbugs issues
[amaliujia] add comparison operator test
[ekirpichov] Address easy review comments
[xiliu] Fix a unit test timeout and minor issues in pom.xml
[kirpichov] Handles memoization of encoded key/window in StateAccessor
[kirpichov] Simplifies API of WireCoders and LengthPrefixUnknownCoders.
[sidhom] [BEAM-4615] Flink job server wrapper and shadow jar
[apilloud] [SQL] Don't wait on Unbounded PCollections
[kirpichov] Address comments
[xiliu] [BEAM-4640] Samza runner postcommit ValidatesRunner job
[aaltay] [BEAM-4333] Add integration tests for python mobile games (#5747)
[jiangkai] [BEAM-4602][BEAM-4598] Date type comparison
[github] Cythonizing a few hot calls (#5733)
[daniel.o.programmer] [BEAM-3708] Adding Combine component implementations to Java SDK
[iemejia] Fix maven build part 9
[xiliu] Remove unused previous names
[daniel.o.programmer] Quick fix, removing commented out code.
[altay] Publish 2.5.0 release
[altay] Change download link for 2.4.0 version to use the archive link
[altay] improvements to fanout
[yifanzou] [BEAM-4302] Add Jenkins job to check Beam dependencies and generate
[ekirpichov] Fixes Flink ValidatesRunner and a compile error
[scott] Split pre-commit tests into separate jobs based on trigger condition.
[scott] Filter pre-commit triggering based on touched files
[scott] Rename setPostCommit to setAutoJob
[amyrvold] [BEAM-4645] Cleanup docker images on failure
[amaliujia] replace byte[] with String as Map key.
[scott] Fix ErrorProne warnings in sdks-java-core project
[scott] Enable ErrorProne analysis for all Java projects.
[scott] Update Findbugs annotations dependency to version with
[scott] Enable failOnWarnings for all projects.
[pablo] Removing debug string from UserScore.java
[scott] Add checkstyle validation to ban usage of @SuppressFBWarnings.
[amyrvold] [BEAM-3261] Fix pipeline failures by waiting for logging server to be
[github] Revert "improvements to fanout"
[klk] Apply rename of JobBuilder to Samza VR job
[xiliu] [BEAM-4641] Samza runner postcommit status in PR template
[kenn] Add bundle-ready SQL shell using application plugin
[relax] Use ByteBuddy to generate a coder class for a specific Schema.
[relax] Subclass proper generic type, removing need for extra typecast.
[relax] Address code-review comments.
[scott] Rename refactored Precommit source files to remove GradleBuild suffix.
[herohde] Run gofmt
[kenn] Reduce stdout spam from BeamModulePlugin
[scott] Update Pre-commit job names tracked in precommit_job_times.ipynb
[lcwik] [BEAM-4661] Define well known timer URN.
[herohde] [BEAM-4635] Avoid cross-compiling worker binary if linux-amd64
[klk] Increase website job timeout to 100 by default
[scott] Add Apache license headers
[ankurgoenka] Adding flinkMarterUrl as an option
[robertwb] Give the root transform in Python a non-empty name.
[herohde] [BEAM-4635] Do not remove worker binary on Go Dataflow staging
[amaliujia] Improve BigQuery write integration tests and support TIMESTAMP SQL type
[ryan.blake.williams] [BEAM-2810] use fastavro-blocks in Avro IO
[klk] Fix accidental license header javadoc comments
[klk] Fix HTML error in ApproximateQuantiles
[klk] Turn on spotless java everywhere
[aaltay] [BEAM-4606] Pin pytz version (#5751)
[scott] Update pre-commit filter to include migrated build rules
[klk] Do not email individuals about failures in Jenkins
[klk] Pipeline options in JDBC URI; default userAgent=BeamSQL
[klk] Spotless Java everywhere!
[klk] Simplify checkstyle now that spotless is in play
[klk] Temporarily disable JavadocParagraph and LineLength checkstyle
[scott] Remove blanket website from rat and add appropriate exclusions.
[github] improvements to fanout (#5800)
[herohde] Do not launch containers with transient semi_persistent_dir. Fails on
[herohde] Go-on-Flink: treat artifact staging EOF error as ok
[herohde] Go-on-Flink: make transform UniqueNames _globally_ unique
[herohde] Go-on-Flink: remove internal java runner class hack
[herohde] Go-on-Flink: fix swapped arguments in artifact staging
[herohde] Go-on-Flink: use clearer error if the artifact retrieval token or
[herohde] Go-on-Flink: fix Go to actually send worker_id gRPC header
[lukasz.gajowy] Remove mvn leftovers from jenkins IOIT job definitions
[klk] Rearrange checkArgument to help findbugs
[chamikara] Reduces the size of pretty string of BigQuery jobs
[mariagh] Add Python Nexmark suite
[herohde] CR: Don't enforce non-empty control worker_id yet
[xiliu] Make Samza runner in lexicographical order
[robertwb] [BEAM-4285] Extend side input handlers to handle multiple access
[relax] Rename RowType -> Schema and move to new schemas package.
[relax] Add classes for SchemaRegistry and SchemaProvider.
[relax] Add SchemaCoder, some more detail in SchemaProvider/Registery, and
[relax] Update to comment.
[relax] Integrate Schemas into PCollection Coder inference.
[relax] Add setSchema convenience method.
[relax] Fix compilation errors.
[relax] Refactor Schema: Schema now is a fully-specified based on primitive
[relax] Implement SchemaCoder and fix up schema classes.
[relax] Flesh out FieldAccessDescriptor.
[relax] Start plumbing through DoFn.
[robertwb] Rename several MultimapSideInput* classes to more generic SideInput*.
[relax] Start editing signatures
[relax] Finish plumbing schemas through.
[relax] Add test for schema PCollection.
[relax] Add withSchema to Create.java
[relax] Plumb schemas through OutputReceiver.
[relax] Add an implementation of SchemaProvider.
[relax] Add SchemaRegistry and a test for schema inference. Change schema
[relax] Add unit test for SchemaRegistry.
[relax] Add FieldAccessDescriptorTest
[relax] Add test for Create changes.
[relax] Plumb FieldAccess through.
[relax] Make sure everything is marked experimental with Kind.SCHEMAS
[relax] Add graph-construction time verification of schemas.
[relax] Fix build and style failures.
[relax] Address code-review comments.
[relax] Fixup after merge.
[relax] Apply spotless autoformat.
[lcwik] [BEAM-4659] Add well known timer coder to Java SDK.
[kirpichov] Makes fuser materialize PCollections used as side inputs
[relax] Fix compilation error introduced by merge.
[ankurgoenka] Uploading job server jar when running on remote flink environment
[ankurgoenka] Passing files to stage on flink explicitly while creating the
[ankurgoenka] Spotless
[robertwb] Remove obsolete comment about multimap fitting into memory.
[relax] Fix CheckStyle errors.
[ankurgoenka] Job service additionally need google cloud dependency to use GCS
[relax] Fix Apex runner.
[relax] Fix more breakages.
[relax] run spotless.
[kirpichov] Makes Python PortableRunner usable via PipelineOptions
[amaliujia] Remove unused system.out.println
[ankurgoenka] TODO to add more file systems
[kirpichov] Brings back hack for embedded FnApiRunner
[robertwb] [BEAM-3883] Stage files in the portability runner.
[robertwb] Fix tests due to api change.
[kirpichov] Changes portableWordCount gradle task to be runnable at master
[ankurgoenka] Adding non standard side input urn to flink
[relax] Fix issues.
[relax] Fix Spark issue.
[kirpichov] Fixes spotless errors at master
[mariagh] Allow streaming mobile examples to read from either a topic or a
[relax] Address code-review comments and fix a failing test.
[relax] Fix Apex runner.
[echauchot] [BEAM-2850] Add Nexmark PostCommit runs for spark, flink and direct
[relax] Change ParDoSchemaTest to ValidatesRunner, and exclude unsupported
[relax] Make worker changes backwards compatible with DataflowRunner. Once
[mariagh] Address review comments
[amaliujia] Support BigQuery read from SQL.
[relax] Add another backwards-compatible method.
[ekirpichov] Makes FileIO.match watermark advance even without new files
[pablo] Test case for providing experiments to pipeline
[kirpichov] [BEAM-4689] Reverts change of SDF key type
[daniel.o.programmer] [BEAM-3708] Adding grouping table to Precombine step.
[daniel.o.programmer] [BEAM-3708] Simplifying precombine grouping tables.
[klk] Add integrationTest task for GCP IO module
[klk] PubsubIO: Always use GCP project from PipelineOptions for subscriptions
[klk] Use BagState in PubsubSignal since it doesn't use SetState methods
[klk] Replace println with logging
[klk] More logging when PubsubJsonIT fails
[klk] Add start signal to TestPubsubSignal
[klk] Use futures in PubsubJsonIT LIMIT test
[klk] Integration test of Pubsub public dataset
[klk] Use test PipelineOptions in PubsubJsonIT LIMIT test
[klk] Sickbay PubsubReadIT on Dataflow
[klk] Fix import order
[mariagh] Add query 1 & 2
[mariagh] Clean up
[mariagh] Cleaup
[mariagh] Fix linter errors
[mariagh] Address review commnets; Reuse topic for queries
[github] Return timeout value to 180
[relax] Fix build.
[relax] Add SchemaProviderRegistrar class.
[klk] Revert accidental clobber of MinimalWordCount.java
[relax] Exclude schema tests from Samza runner.
[klk] Update BigQueryAvroUtils to support BigQuery's NUMERIC type.
[relax] Add DefaultSchema and fix failures turned up by plugins.
[klk] Convert Beam Row to Avatica Row in BeamEnumerableCollector
[kenn] sqlline dep to 1.4.0
[kenn] Fix broken javadoc
[kenn] Remove more extraneous printlns from build
[kenn] Remove aliased tables in Nexmark SQL query 5
[lcwik] Revert "Enabling gradle build to receive extra buildscript repositories"
[lcwik] Don't enable plugins which require you to be online when run in
[kenn] fixup! Remove aliased tables in Nexmark SQL query 5
[lcwik] [BEAM-4126] Delete Maven build files.
[kenn] Simplify/fix windowing check in BeamSortRel
[robertwb] Fix global hot key fanout setting.
[scott] Increase concurrency in ValidateRunner execution
[lcwik] [BEAM-3971, BEAM-4284] Remove fromProto for Pipeline and PTransform
[robinyqiu] Support @RequiresStableInput on Dataflow runner in Java SDK
[kenn] Run SQL operator tests through JDBC as well
[kenn] Test retrieval of TIMESTAMP via JDBC
[kenn] Instantiate $SUM0 as a SUM operator
[ankurgoenka] Read worker harness image name from pipeline option for portable runners
[ankurgoenka] Spotless
[kenn] Instantiate "+" as DATETIME_PLUS
[ankurgoenka] Support only a single environment in SdkComponents
[ankurgoenka] Renaming createEnvironment to createOrGetDefaultEnvironment.
[lcwik] [BEAM-4716] Remove findbugs declarations in build.gradle files since it
[relax] Address review commemnts.
[ccy] [BEAM-4007] Fix TODO style in typehints.py
[relax] Add schema inference from POJO.
[relax] Add BYTES type to schema, and POJO inference.
[relax] Add automatic getter/setter inference from POJOS. * Recursive POJOs
[relax] Handle setter for ByteBuffer fields. Setter will be called with a byte[]
[relax] Split Row into separate subclass for RowWithStorage and RowWithGetters.
[relax] Introduce FieldValueGetterFactory and FieldValueSetterFactory.
[relax] Add support for nested, array, and map types.
[relax] Refactor getter-based SchemaProvider into a common base class.
[relax] Refactor getter/setter conversion utilities into a library. This cleans
[relax] Refactor Schema inference from POJOs so much of the code can be reused
[relax] Add encoding byte BYTES type.
[relax] Make FieldValueGetter and FieldValueSetter generic types again.
[relax] Start adding JavaBean getters and setters.
[relax] Make sure that field order is stable, and make tests robust to
[relax] Simplify convertArray.
[amaliujia] support decimal type in reading from PubSub JSON
[ankurgoenka] Fix artifact retrival file system
[jiangkai] [BEAM-4723] Support date type plus interval
[ankurgoenka] Create SDKComponents with pipeline options
[echauchot] [BEAM-4707] Add timestamp field to Nexmark tables and use explicit mode
[lukasz.gajowy] [BEAM-4137] Divide separate PipelineOptions for elasticsearch tests
[lukasz.gajowy] [BEAM-4137] Remove unused options from IOTestPipelineOptions
[katarzyna.kucharczyk] Added integration test configuration to beam-examples gradle. Fixed
[markliu] Move gradle-spotless plugin to buildscript
[rober] Add a simple benchmark to the ParDo execution node.
[robertwb] Use main output for cold keys.
[pablo] Unpin GCP test requirement
[lcwik] [BEAM-4654] Translate timers as PCollections within the Java SDK.
[robertwb] Use local GCP credentials, if any, in local docker environment.
[cclauss] [BEAM-1251] Modernize Python 2 code to get ready for Python 3
[robertwb] Split bounded sources in larger than 1k chunks.
[cclauss] [BEAM-1251] Upgrade from buffer to memoryview (again)
[thw] Use the beam:option:value:v1 as the portable pipeline options
[thw] [BEAM-4733] Pass pipeline options from Python portable runner to job
[matthias] removed unnecessary install_aliases, import io and unicode imports
[thw] Fix Flink portable streaming translation executable stage output
[echauchot] [BEAM-4707] insert timestamp to bigQuery as seconds as expected by
[echauchot] [BEAM-4671] Skip MetricsPusherTest validates runner test for runners
[matthias] Futurize io subpackage
[matthias] incorporated all feedback for futurize io subpackage
[robertwb] object() takes no parameters
[amaliujia] remove decimal support in BigQuery SQL read.
[iemejia] [BEAM-4016] Invoke Setup and TearDown on SplitRestrictionFn and
[ankurgoenka] Initialization BeamFileSystem in FlinkExecutableStageFunction
[ankurgoenka] [BEAM-4644] Use SLF4J in ExecutableStageDoFnOperator
[ccy] Revert #5878 to unbreak Python postcommit
[ankurgoenka] Pull the docker image before starting docker container.
[iemejia] Rename COMBINE_TRANSFORM_URN to the more proper
[ccy] [BEAM-4003] Fix missing iteritems import
[relax] Address code-review comments.
[relax] Address code-review comments.
[github] Avoid silent error when calling unimplemented function.
[github] Avoid silent error when calling unimplemented function.
[relax] Add better validation to isSetter isGetter
[ekirpichov] Infer boundedness for SDF application at application time
[ekirpichov] [BEAM-4745] Revert "[BEAM-4016] Invoke Setup and TearDown on
[markliu] Remove :buildSrc from settings.gradle since it's not a subproject
[pablo] Logging relies on StateSampler for context
[ccy] [BEAM-4594] Beam Python state and timers user-facing API
[ccy] [BEAM-4593] Remove refcounts from the Python SDK
[ankurgoenka] FileSystems should throw exception if filesystem for schema is not found
[ankurgoenka] Make schema pattern stricter
[ccy] [BEAM-1251] Revert #5887 to unbreak Python PostCommit
[pablo] Improving signing of all published artifacts and adding test publication
[ccy] [BEAM-1251] Upgrade snappy and use a memoryview
[pablo] Removing extra publication
[cclauss] [BEAM-3761] Define cmp() in Python 3
[cclauss] [BEAM-3959] Add Python 3 undefined names to flake8
[amyrvold] [BEAM-3457] Upgrade version of gogradle and add examples and containers
[apilloud] [SQL] Inject JDBC rules through Hook.
[lcwik] [BEAM-4481, BEAM-4484] Start vendoring portability dependencies to not
[apilloud] [SQL] Plumb pipelineOptions through IOSinkRel
[lcwik] [BEAM-4653] Add support to the Java SDK harness to execute timers.
[apilloud] [SQL] Default timezone is UTC
[lcwik] [BEAM-2915] Add support for handling bag user state to the
[amaliujia] add testArithmeticOperator
[altay] Wait on connection channel to be ready
[ekirpichov] Bumps Dataflow worker container version
[ekirpichov] [BEAM-4745, BEAM-4016] Reintroduces Setup and TearDown on
[amaliujia] simplify unbounded limit
[iemejia] [BEAM-4020] Add an HBaseIO read implementation based on SDF
[iemejia] [BEAM-4020] Add HBaseIO.readAll() based on SDF
[lukasz.gajowy] [BEAM-4691] Rename common_job_properties
[aromanenko.dev] [BEAM-4622] Makes required to call Beam SQL expressions validation
[aromanenko.dev] Check number of arguments at first
[lcwik] [BEAM-4764] Fix missing reference to project to be able to refer to the
[lcwik] [BEAM-4764] Fix java precommit to ensure that any checks in model/* are
[apilloud] [SQL] Apply JDBC rules globally
[rober] Add a small benchmark for metrics
[ankurgoenka] Warn on docker image pull fail
[robinyqiu] Fix bug in java mobile gaming example
[lcwik] [BEAM-4744] Enable parallel publishing/release
[qinyeli] Interactive runner and corresponding tests
[github] [BEAM-4648] Remove experimental Python RPC DirectRunner (#5777)
[s.lebedev] [BEAM-1251] Replace NameError-driven dispatch with ``past``
[altay] Install fastavro only in linux
[pablo] [BEAM-4718]Run gradle build before publish
[pablo] Add CODEOWNERS file (#5909)
[pablo] Full tracking of Batch side inputs. Performance tests show no regression
[kirpichov] Adds license header to CODEOWNERS
[kirpichov] Adds a naive implementation of bounded SDFs
[echauchot] Move echauchot from runner/core to runner/core/metrics in CODEOWNERS
[github] [BEAM-4752] Add support for newer dill dependency (#5931)
[elliottb] Fix the expected encoding of BigQuery's NUMERIC type when reading from
[ankurgoenka] Adding tet cases for matchResouces
[kirpichov] Makes SplittableDoFnTest exercise both bounded and unbounded SDFs.
[github] [BEAM-4752] Add dill compatibility for older versions of dill (#5949)
[ankurgoenka] Ignore generated vendor files in python container build
[kirpichov] Supports bounded SDF in all runners.
[relax] Add new Convert transform for schemas.
[relax] Fix two bugs: * proper handling of nullable fields. * Row stores
[relax] Add tests.
[relax] Fix broken test.
[relax] Add @Nullable annotation.
[github] [BEAM-4432] Adding Sources to produce Synthetic output for Batch
[ryan.blake.williams] send PutArtifactResponse in BeamFileSystemArtifactStagingService
[apilloud] [BEAM-4774] Add Nexmark SQL to postcommits
[aaltay] [BEAM-2810] fastavro integration test (#5862)
[kirpichov] Improve error messages on Spark and Flink SDF translation
[qinyeli] Interactive Beam -- yields Read Trans after apply()
[apilloud] [SQL] Actually run JDBC Jar Test
[apilloud] [SQL] Wrap PipelineOptions with correct class loader
[kirpichov] Converts BoundedReadFromUnboundedSource to a DoFn
[boyuanz] Automate release branch cut process
[yifanzou] [BEAM-4801] add dependency owners file
[lcwik] [BEAM-4769] Fix issue when performing coder inference for wildcard
[lcwik] [BEAM-4744] Fix runners/google-cloud-dataflow-java/examples-streaming
[ankurgoenka] Fixing Test case for beam Artifact Staging Service
[boyuanz] Fix javadoc broken
[daniel.o.programmer] [BEAM-3711] Enabling combiner lifting in Dataflow Runner.
[ankurgoenka] Fix for Flaky test BeamFileSystemArtifactServicesTest
[lcwik] [BEAM-4418] Fix duplicate source set declaration conflict.
[pablo] Add coders microbenchmark. (#5565)
[melissapa] [BEAM-3769] Fix broken JIRA links in README
[pablo] Moving to 2.7.0-SNAPSHOT on master branch.
[thw] [BEAM-4791] Portable Flink runner integration test.
[thw] Use shadowTest configuration since we don't plan to shade away the SDK
[thw] [BEAM-4791] Move executor to Before/After and use
[github] Increase the complete time out
[github] Minor javadoc fix up.
[amaliujia] BEAM-4820: Add timeout to Future.get when waiting for Pubsub messages
[github] Adding @angoenka to CODEOWNERS
[boyuanz] Fix javadoc build broken in synthetic for release branch
[garrettjonesgoogle] Bumping google cloud versions
[thw] Add JIRA reference.
[lcwik] Address PR comments.
[pablo] Fix bugs in 'cut release branch step'
[lcwik] Address PR comments.
[amaliujia] [BEAM-4806] improve unit tests for SqlLineMain
[qinyeli] Added a round trip through proto in Interactive runner.
[github] Add version bound to numpy.
[echauchot] CodeOwners: Narrow down extensions-java to metrics and assign echauchot
[jxlewis] Added ClientConfiguration to AwsOptions and S3FileSystem
[jxlewis] Updated documentation, added BEAM-4828 to In-Progress
[github] Fix indentation.
[github] SQS -> Amazon SQS
[boyuanz] Added '--continue' switches into nightly build
[supercclank] Set BatchPath on google.api.services.storage.Storage Storage Builder
[kirpichov] Converts SolrIO away from BoundedSource
[supercclank] Change to using java.nio.file.Paths over string concatenation for batch
[apilloud] [BEAM-4761] Run Nexmark on Dataflow
[github] [BEAM-4832] Fix concurrent race in datamgr.go
[github] Update datamgr.go
[carlos.alonso] Adds the BigQueryInsertError model class
[carlos.alonso] Adds BigQueryInsertErrorCoder and tests
[carlos.alonso] Adds extended errors info to WriteResult
[carlos.alonso] Makes streaming methods generic but limited to the implemented error
[carlos.alonso] Removes ErrorContainers auxiliary class. Adds BigQueryServicesImpl test
[carlos.alonso] Makes BigQueryIO.Write to be configurable to decide which kind of errors
[carlos.alonso] Removes unneeded coder registration
[carlos.alonso] Simplifies BigQueryInsertError codification
[carlos.alonso] A few style improvements
[katarzyna.kucharczyk] Changed order of installing perfkit requirements and beam requirements
[thw] Remove dead link from README.
[amyrvold] [BEAM-4831] Ignore failures during :beam-sdks-go:vet to allow ./gradlew
[aaltay] Add automation script for verifying release build (#5973)
[ekirpichov] Avoids use of discouraged Duration(long) ctor
[qinyeli] Interactive Runner -- keep_alive
[thw] [BEAM-4842] Update Flink Runner to Flink 1.5.1
[daniel.o.programmer] [BEAM-3709] Implementing new combine URNs in python.
[aaltay] Add bash script to automate "Preparation for GPG" (#6015)
[github] Fixing log message
[lukasz.gajowy] [BEAM-4845] Make BigQueryServices and FakeDatasetService public to fix
[amyrvold] Fix [BEAM-4847] by manually setting max workers and jvm memory and add
[relax] Enable schemas for Apex runner.
[relax] Enable Schemas on samza runner.
[relax] Enable schemas for Flink runner.
[relax] Enable schemas for Spark.
[relax] Fix bugs.
[relax] Fix Apex breakage.
[ehudm] Remove CODEOWNERs.
[kedin] [SQL] Enable running BeamSqlLine from gradle
[thw] [BEAM-4847] Reduce Gradle JVM Xmx to 4g to fix Jenkins build failures.
[aaltay] Automate 'Start a snapshot build' step in beam release guide (#6042)
[lukasz.gajowy] [BEAM-4838] Add dockerfile for standalone Jenkins. Plugins included.
[dariusz.aniszewski] separate virtualenvs for Beam and PerfKit
[lcwik] [BEAM-4866] Fix missing licenses.
[pablo] Removing scoped metrics container
[pablo] Remove old style metrics context management
[garrettjonesgoogle] Bumping versions that were missed in #5988
[altay] Remove reference to dataflow-distribution.properties
[lcwik] [BEAM-4176] Initial implementation for running portable runner tests
[pablo] Fix Java Nightly Snapshot Failures
[lcwik] [BEAM-4629] Output the names of the failing licenses as part of the
[aaltay] [BEAM-4859] Enable Python VR tests in streaming in postcommit task
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision aa2b585a5356ce3b607dd3d17a3c6ab2ce65353b (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f aa2b585a5356ce3b607dd3d17a3c6ab2ce65353b
Commit message: "Merge pull request #6056: [BEAM-4836] separate virtualenvs for Beam and PerfKit"
> git rev-list --no-walk 0892e23940a584a170e2668d86b7bdc50c03e024 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins266031542697616565.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2118418907451248094.sh
+ rm -rf .beam_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1013385759992617214.sh
+ rm -rf .perfkit_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1440131868588494871.sh
+ virtualenv .beam_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3203404953492079456.sh
+ virtualenv .perfkit_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3388230821306939726.sh
+ .beam_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.beam_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.beam_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8681682376553589136.sh
+ .perfkit_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.perfkit_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.perfkit_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1014785923945432727.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7488451273964902903.sh
+ .beam_env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.8.2)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.7)
Requirement already satisfied: dill<=0.2.8.2,>=0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.2.8.2)
Requirement already satisfied: grpcio<2,>=1.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.13.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.1.0)
Requirement already satisfied: httplib2<=0.11.3,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.11.3)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.0.0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (4.1.2)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.0)
Requirement already satisfied: pydot<1.3,>=1.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.2.4)
Requirement already satisfied: pytz<=2018.4,>=2018.3 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2018.4)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (3.12)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.6.8)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.11.0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.4)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.2.0)
Requirement already satisfied: future<1.0.0,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.16.0)
Requirement already satisfied: fastavro==0.19.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.19.7)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.5.20)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.90.4)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (7.0.1)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.26.0)
Requirement already satisfied: proto-google-cloud-pubsub-v1==0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: nose>=1.3.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.3.7)
Collecting numpy<2,>=1.14.3 (from apache-beam==2.7.0.dev0)
Using cached https://files.pythonhosted.org/packages/85/51/ba4564ded90e093dbb6adfc3e21f99ae953d9ad56477e1b0d4a93bacf7d3/numpy-1.15.0-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.9.0)
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.7.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.8.0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.0.2)
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.4.2)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.2.2)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (3.4.2)
Requirement already satisfied: setuptools in ./.beam_env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.7.0.dev0) (40.0.0)
Requirement already satisfied: pyparsing>=2.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from pydot<1.3,>=1.2.0->apache-beam==2.7.0.dev0) (2.2.0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (0.14.1)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.7.0.dev0) (1.5.3)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (1.22)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (1.5)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.16)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.11.4)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.0.3)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (1.5.0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (3.8)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (2.1.0)
Installing collected packages: numpy, apache-beam
Found existing installation: numpy 1.13.3
Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'numpy'. No files were found to uninstall.
Found existing installation: apache-beam 2.7.0.dev0
Not uninstalling apache-beam at /home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_Compressed_TextIOIT_HDFS/src/sdks/python, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'apache-beam'. No files were found to uninstall.
Running setup.py develop for apache-beam
Successfully installed apache-beam numpy-1.15.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7216896748234028238.sh
+ .beam_env/bin/python src/sdks/python/setup.py sdist --dist-dir=src/sdks/python/target
Traceback (most recent call last):
File "src/sdks/python/setup.py", line 46, in <module>
PACKAGE_VERSION = get_version()
File "src/sdks/python/setup.py", line 41, in get_version
exec(open(os.path.normpath('./apache_beam/version.py')).read(), global_names) # pylint: disable=exec-used
IOError: [Errno 2] No such file or directory: 'apache_beam/version.py'
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PerformanceTests_Python #1254
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1254/display/redirect>
------------------------------------------
[...truncated 1.72 KB...]
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2255626873956628632.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins9165999473588564426.sh
+ rm -rf .beam_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins9063788179791268742.sh
+ rm -rf .perfkit_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5884993017406052429.sh
+ virtualenv .beam_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1342281336343959987.sh
+ virtualenv .perfkit_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3118154934927439155.sh
+ .beam_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.beam_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.beam_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6504331171475556563.sh
+ .perfkit_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.perfkit_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.perfkit_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4209305034972557352.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5351899519785124003.sh
+ .beam_env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.8.2)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.7)
Requirement already satisfied: dill<=0.2.8.2,>=0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.2.8.2)
Requirement already satisfied: grpcio<2,>=1.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.13.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.1.0)
Requirement already satisfied: httplib2<=0.11.3,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.11.3)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.0.0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (4.1.2)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.0)
Requirement already satisfied: pydot<1.3,>=1.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.2.4)
Requirement already satisfied: pytz<=2018.4,>=2018.3 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2018.4)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (3.12)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.6.8)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.11.0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.4)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.2.0)
Requirement already satisfied: future<1.0.0,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.16.0)
Requirement already satisfied: fastavro==0.19.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.19.7)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.5.20)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.90.4)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (7.0.1)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.26.0)
Requirement already satisfied: proto-google-cloud-pubsub-v1==0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: nose>=1.3.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.3.7)
Collecting numpy<2,>=1.14.3 (from apache-beam==2.7.0.dev0)
Using cached https://files.pythonhosted.org/packages/85/51/ba4564ded90e093dbb6adfc3e21f99ae953d9ad56477e1b0d4a93bacf7d3/numpy-1.15.0-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.9.0)
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.7.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.8.0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.0.2)
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.4.2)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.2.2)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (3.4.2)
Requirement already satisfied: setuptools in ./.beam_env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.7.0.dev0) (40.0.0)
Requirement already satisfied: pyparsing>=2.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from pydot<1.3,>=1.2.0->apache-beam==2.7.0.dev0) (2.2.0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (0.14.1)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.7.0.dev0) (1.5.3)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (1.22)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (1.5)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.16)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.11.4)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.0.3)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (1.5.0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (3.8)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (2.1.0)
Installing collected packages: numpy, apache-beam
Found existing installation: numpy 1.13.3
Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'numpy'. No files were found to uninstall.
Found existing installation: apache-beam 2.7.0.dev0
Not uninstalling apache-beam at /home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_Compressed_TextIOIT_HDFS/src/sdks/python, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'apache-beam'. No files were found to uninstall.
Running setup.py develop for apache-beam
Successfully installed apache-beam numpy-1.15.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1563680810598338892.sh
+ .beam_env/bin/python src/sdks/python/setup.py sdist --dist-dir=src/sdks/python/target
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/local/lib/python2.7/site-packages/setuptools/dist.py>:398: UserWarning: Normalizing '2.7.0.dev' to '2.7.0.dev0'
normalized_version,
running sdist
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/gen_protos.py>:49: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
running egg_info
creating apache_beam.egg-info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
warning: cmd: standard file not found: should have one of README, README.rst, README.txt, README.md
running check
creating apache-beam-2.7.0.dev0
creating apache-beam-2.7.0.dev0/apache_beam.egg-info
copying files to apache-beam-2.7.0.dev0...
copying apache_beam.egg-info/PKG-INFO -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/SOURCES.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/dependency_links.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/entry_points.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/not-zip-safe -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/requires.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/top_level.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
Writing apache-beam-2.7.0.dev0/setup.cfg
creating src/sdks/python/target
Creating tar archive
removing 'apache-beam-2.7.0.dev0' (and everything under it)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3779598970039815852.sh
+ .perfkit_env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14)) (0.2.2)
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15)) (2.10)
Requirement already satisfied: setuptools in ./.perfkit_env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16)) (40.0.0)
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17)) (2.6.0)
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18)) (1.4)
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19)) (3.2.0)
Requirement already satisfied: PyYAML==3.12 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 20)) (3.12)
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21)) (0.8.1)
Requirement already satisfied: numpy==1.13.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22)) (1.13.3)
Requirement already satisfied: functools32 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 23)) (3.2.3.post2)
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24)) (0.5.5)
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25)) (0.3.0)
Requirement already satisfied: timeout-decorator in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 26)) (0.4.0)
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Requirement already satisfied: colorama; extra == "windows" in /home/jenkins/.local/lib/python2.7/site-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17)) (0.3.9)
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.0)
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.11.0)
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.2.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7173993871705149011.sh
+ .perfkit_env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --bigquery_table=beam_performance.pkb_results --k8s_get_retry_count=36 --k8s_get_wait_interval=10 --python_binary=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --beam_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-07-26 10:46:45,580 b302cf0c MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/b302cf0c/pkb.log>
2018-07-26 10:46:45,580 b302cf0c MainThread INFO PerfKitBenchmarker version: v1.12.0-729-ga00bd87
2018-07-26 10:46:45,581 b302cf0c MainThread INFO Flag values:
--beam_sdk=python
--k8s_get_wait_interval=10
--python_binary=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python>
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--beam_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src>
--k8s_get_retry_count=36
--benchmarks=beam_integration_benchmark
2018-07-26 10:46:45,875 b302cf0c MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-07-26 10:46:46,063 b302cf0c MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-07-26 10:46:46,067 b302cf0c MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-07-26 10:46:46,068 b302cf0c MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/gradlew> clean assemble --stacktrace --info -DintegrationTestRunner=dataflow
2018-07-26 10:52:45,185 b302cf0c MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-07-26 10:52:45,278 b302cf0c MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/target/apache-beam-2.7.0.dev0.tar.gz>
2018-07-26 11:02:45,286 b302cf0c Thread-3 ERROR IssueCommand timed out after 600 seconds. Killing command "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/target/apache-beam-2.7.0.dev0.tar.gz".>
2018-07-26 11:02:45,296 b302cf0c MainThread beam_integration_benchmark(1/1) INFO Ran: {<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/target/apache-beam-2.7.0.dev0.tar.gz}> ReturnCode:-9
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/local/lib/python2.7/site-packages/setuptools/dist.py>:398: UserWarning: Normalizing '2.7.0.dev' to '2.7.0.dev0'
normalized_version,
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/gen_protos.py>:49: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py>:54: DeprecationWarning: options is deprecated since First stable release.. References to <pipeline>.options will not be supported
print('Found: %s.' % self.build_console_url(pipeline.options))
2018-07-26 11:02:45,297 b302cf0c MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 668, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 548, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-07-26 11:02:45,298 b302cf0c MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-07-26 11:02:45,301 b302cf0c MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 804, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 668, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 548, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-07-26 11:02:45,301 b302cf0c MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-07-26 11:02:45,317 b302cf0c MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-07-26 11:02:45,318 b302cf0c MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/b302cf0c/pkb.log>
2018-07-26 11:02:45,318 b302cf0c MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/b302cf0c/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PerformanceTests_Python #1253
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1253/display/redirect>
------------------------------------------
GitHub pull request #6056 of commit d951271cb4412facc746bf3aab4b9dcf86259472, no merge conflicts.
Setting status of d951271cb4412facc746bf3aab4b9dcf86259472 to PENDING with url https://builds.apache.org/job/beam_PerformanceTests_Python/1253/ and message: 'Build started for merge commit.'
Using context: Python SDK Performance Test
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
Cloning the remote Git repository
Cloning repository https://github.com/apache/beam.git
> git init <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src> # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/*
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
> git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/6056/*:refs/remotes/origin/pr/6056/*
> git rev-parse refs/remotes/origin/pr/6056/merge^{commit} # timeout=10
> git rev-parse refs/remotes/origin/origin/pr/6056/merge^{commit} # timeout=10
Checking out Revision f5b091a50c6bfa3e7f10e361eab9d4d107763ddc (refs/remotes/origin/pr/6056/merge)
> git config core.sparsecheckout # timeout=10
> git checkout -f f5b091a50c6bfa3e7f10e361eab9d4d107763ddc
Commit message: "Merge d951271cb4412facc746bf3aab4b9dcf86259472 into f2a1fd4307929aa938067058205a5ff79dc0207f"
First time build. Skipping changelog.
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4613836979666378994.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7257108433714162759.sh
+ rm -rf .beam_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6164015999892067355.sh
+ rm -rf .perfkit_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8665091684199027555.sh
+ virtualenv .beam_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5604433107216637169.sh
+ virtualenv .perfkit_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6086223940262032725.sh
+ .beam_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.beam_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.beam_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8326416354300863133.sh
+ .perfkit_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.perfkit_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.perfkit_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6909867195755979876.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1403573749653444305.sh
+ .beam_env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.8.2)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.7)
Requirement already satisfied: dill<=0.2.8.2,>=0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.2.8.2)
Requirement already satisfied: grpcio<2,>=1.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.13.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.1.0)
Requirement already satisfied: httplib2<=0.11.3,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.11.3)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.0.0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (4.1.2)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.0)
Requirement already satisfied: pydot<1.3,>=1.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.2.4)
Requirement already satisfied: pytz<=2018.4,>=2018.3 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2018.4)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (3.12)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.6.8)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.11.0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.4)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.2.0)
Requirement already satisfied: future<1.0.0,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.16.0)
Requirement already satisfied: fastavro==0.19.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.19.7)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.5.20)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.90.4)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (7.0.1)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.26.0)
Requirement already satisfied: proto-google-cloud-pubsub-v1==0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: nose>=1.3.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.3.7)
Collecting numpy<2,>=1.14.3 (from apache-beam==2.7.0.dev0)
Using cached https://files.pythonhosted.org/packages/85/51/ba4564ded90e093dbb6adfc3e21f99ae953d9ad56477e1b0d4a93bacf7d3/numpy-1.15.0-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.9.0)
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.7.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.8.0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.0.2)
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.4.2)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.2.2)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (3.4.2)
Requirement already satisfied: setuptools in ./.beam_env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.7.0.dev0) (40.0.0)
Requirement already satisfied: pyparsing>=2.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from pydot<1.3,>=1.2.0->apache-beam==2.7.0.dev0) (2.2.0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (0.14.1)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.7.0.dev0) (1.5.3)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (1.22)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (1.5)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.16)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.11.4)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.0.3)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (1.5.0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (3.8)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (2.1.0)
Installing collected packages: numpy, apache-beam
Found existing installation: numpy 1.13.3
Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'numpy'. No files were found to uninstall.
Found existing installation: apache-beam 2.7.0.dev0
Not uninstalling apache-beam at /home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_Compressed_TextIOIT_HDFS/src/sdks/python, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'apache-beam'. No files were found to uninstall.
Running setup.py develop for apache-beam
Successfully installed apache-beam numpy-1.15.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8809190521784436377.sh
+ .perfkit_env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14)) (0.2.2)
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15)) (2.10)
Requirement already satisfied: setuptools in ./.perfkit_env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16)) (40.0.0)
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17)) (2.6.0)
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18)) (1.4)
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19)) (3.2.0)
Requirement already satisfied: PyYAML==3.12 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 20)) (3.12)
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21)) (0.8.1)
Requirement already satisfied: numpy==1.13.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22)) (1.13.3)
Requirement already satisfied: functools32 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 23)) (3.2.3.post2)
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24)) (0.5.5)
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25)) (0.3.0)
Requirement already satisfied: timeout-decorator in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 26)) (0.4.0)
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Requirement already satisfied: colorama; extra == "windows" in /home/jenkins/.local/lib/python2.7/site-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17)) (0.3.9)
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.0)
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.11.0)
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.2.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1458877980335576090.sh
+ .perfkit_env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --bigquery_table=beam_performance.pkb_results --k8s_get_retry_count=36 --k8s_get_wait_interval=10 --python_binary=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --beam_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-07-25 15:38:58,755 2f8cc01e MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/2f8cc01e/pkb.log>
2018-07-25 15:38:58,756 2f8cc01e MainThread INFO PerfKitBenchmarker version: v1.12.0-729-ga00bd87
2018-07-25 15:38:58,757 2f8cc01e MainThread INFO Flag values:
--beam_sdk=python
--k8s_get_wait_interval=10
--python_binary=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python>
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--beam_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src>
--k8s_get_retry_count=36
--benchmarks=beam_integration_benchmark
2018-07-25 15:38:59,028 2f8cc01e MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-07-25 15:38:59,215 2f8cc01e MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-07-25 15:38:59,223 2f8cc01e MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-07-25 15:38:59,224 2f8cc01e MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/gradlew> clean assemble --stacktrace --info -DintegrationTestRunner=dataflow
2018-07-25 15:45:10,639 2f8cc01e MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-07-25 15:45:10,735 2f8cc01e MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 668, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 548, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 198, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 274, in _BuildPythonCommand
raise RuntimeError('No python binary is found')
RuntimeError: No python binary is found
2018-07-25 15:45:10,736 2f8cc01e MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-07-25 15:45:10,739 2f8cc01e MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 804, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 668, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 548, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 198, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 274, in _BuildPythonCommand
raise RuntimeError('No python binary is found')
RuntimeError: No python binary is found
2018-07-25 15:45:10,739 2f8cc01e MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-07-25 15:45:10,794 2f8cc01e MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-07-25 15:45:10,795 2f8cc01e MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/2f8cc01e/pkb.log>
2018-07-25 15:45:10,795 2f8cc01e MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/2f8cc01e/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PerformanceTests_Python #1252
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1252/display/redirect>
------------------------------------------
[...truncated 4.68 KB...]
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins147823451126947701.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/aa/063eca6a416f397bd99552c534c6d11d57f58f2e94c14780f3bbf818c4cf/monotonic-1.5-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.5 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3533660477905184729.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-10 12:00:52,567 422a7ff4 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/422a7ff4/pkb.log>
2018-05-10 12:00:52,568 422a7ff4 MainThread INFO PerfKitBenchmarker version: v1.12.0-584-g9260fab
2018-05-10 12:00:52,569 422a7ff4 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-10 12:00:52,941 422a7ff4 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-10 12:00:53,095 422a7ff4 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-10 12:00:53,123 422a7ff4 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-10 12:00:53,124 422a7ff4 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-10 12:00:58,281 422a7ff4 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/422a7ff4/beam/gradlew> clean install -xcheck --stacktrace -DintegrationTestRunner=dataflow
2018-05-10 12:05:21,135 422a7ff4 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-10 12:05:21,146 422a7ff4 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/422a7ff4/beam/sdks/python/target>
2018-05-10 12:05:21,148 422a7ff4 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-10 12:05:21,149 422a7ff4 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 781, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/422a7ff4/beam/sdks/python/target>
2018-05-10 12:05:21,150 422a7ff4 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-10 12:05:21,195 422a7ff4 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-10 12:05:21,196 422a7ff4 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/422a7ff4/pkb.log>
2018-05-10 12:05:21,197 422a7ff4 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/422a7ff4/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user jasonkuster@google.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user schroederc@google.com
Not sending mail to unregistered user shoyer@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1251
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1251/display/redirect?page=changes>
Changes:
[apilloud] [SQL] Use Lex.JAVA and drop toLowerCase in DDL
[apilloud] [SQL] Planner is private within BeamQueryPlanner
[github] Porting Python PostCommit to Gradle. (#5146)
------------------------------------------
[...truncated 4.68 KB...]
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6452696876120264024.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/aa/063eca6a416f397bd99552c534c6d11d57f58f2e94c14780f3bbf818c4cf/monotonic-1.5-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.5 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4374926398290278148.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-10 06:00:42,785 bb421231 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/bb421231/pkb.log>
2018-05-10 06:00:42,786 bb421231 MainThread INFO PerfKitBenchmarker version: v1.12.0-584-g9260fab
2018-05-10 06:00:42,787 bb421231 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-10 06:00:43,099 bb421231 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-10 06:00:43,264 bb421231 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-10 06:00:43,268 bb421231 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-10 06:00:43,269 bb421231 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-10 06:00:48,431 bb421231 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/bb421231/beam/gradlew> clean install -xcheck --stacktrace -DintegrationTestRunner=dataflow
2018-05-10 06:04:55,789 bb421231 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-10 06:04:55,799 bb421231 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/bb421231/beam/sdks/python/target>
2018-05-10 06:04:55,801 bb421231 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-10 06:04:55,803 bb421231 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 781, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/bb421231/beam/sdks/python/target>
2018-05-10 06:04:55,803 bb421231 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-10 06:04:55,841 bb421231 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-10 06:04:55,842 bb421231 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/bb421231/pkb.log>
2018-05-10 06:04:55,842 bb421231 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/bb421231/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user jasonkuster@google.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user schroederc@google.com
Not sending mail to unregistered user shoyer@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1250
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1250/display/redirect>
------------------------------------------
[...truncated 5.08 KB...]
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1792401711501177360.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/aa/063eca6a416f397bd99552c534c6d11d57f58f2e94c14780f3bbf818c4cf/monotonic-1.5-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.5 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2757522060705429708.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-10 01:04:58,668 155f91e5 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/155f91e5/pkb.log>
2018-05-10 01:04:58,668 155f91e5 MainThread INFO PerfKitBenchmarker version: v1.12.0-584-g9260fab
2018-05-10 01:04:58,669 155f91e5 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-10 01:04:58,965 155f91e5 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-10 01:04:59,114 155f91e5 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-10 01:04:59,118 155f91e5 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-10 01:04:59,118 155f91e5 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-10 01:05:03,970 155f91e5 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/155f91e5/beam/gradlew> clean install -xcheck --stacktrace -DintegrationTestRunner=dataflow
2018-05-10 01:08:57,843 155f91e5 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-10 01:08:57,858 155f91e5 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/155f91e5/beam/sdks/python/target>
2018-05-10 01:08:57,859 155f91e5 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-10 01:08:57,861 155f91e5 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 781, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/155f91e5/beam/sdks/python/target>
2018-05-10 01:08:57,862 155f91e5 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-10 01:08:57,897 155f91e5 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-10 01:08:57,898 155f91e5 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/155f91e5/pkb.log>
2018-05-10 01:08:57,898 155f91e5 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/155f91e5/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user jasonkuster@google.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user schroederc@google.com
Not sending mail to unregistered user shoyer@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1249
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1249/display/redirect?page=changes>
Changes:
[apilloud] [BEAM-4251] Fix broken javadoc ref
[apilloud] [SQL] Simplify planner usage, fix assertions
[apilloud] [SQL] Cleanup BeamAggregation rule and rel to fix
[apilloud] [SQL] Fix assertion failures in test
[kirpichov] [BEAM-4242] Fixes O(n) complexity of Wait.on() due to combiner lifting
[iemejia] Fix warning on repeated core-construction-java dependency on
[aaltay] Nexmark pubsub benchmarks. (#5247)
------------------------------------------
[...truncated 4.66 KB...]
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2730193946477290219.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/aa/063eca6a416f397bd99552c534c6d11d57f58f2e94c14780f3bbf818c4cf/monotonic-1.5-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.5 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5647444987526960164.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-10 00:00:38,955 c2fa1eca MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/c2fa1eca/pkb.log>
2018-05-10 00:00:38,955 c2fa1eca MainThread INFO PerfKitBenchmarker version: v1.12.0-584-g9260fab
2018-05-10 00:00:38,956 c2fa1eca MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-10 00:00:39,389 c2fa1eca MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-10 00:00:39,557 c2fa1eca MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-10 00:00:39,561 c2fa1eca MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-10 00:00:39,562 c2fa1eca MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-10 00:00:44,692 c2fa1eca MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/c2fa1eca/beam/gradlew> clean install -xcheck --stacktrace -DintegrationTestRunner=dataflow
2018-05-10 00:05:18,331 c2fa1eca MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-10 00:05:18,342 c2fa1eca MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/c2fa1eca/beam/sdks/python/target>
2018-05-10 00:05:18,344 c2fa1eca MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-10 00:05:18,346 c2fa1eca MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 781, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/c2fa1eca/beam/sdks/python/target>
2018-05-10 00:05:18,346 c2fa1eca MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-10 00:05:18,395 c2fa1eca MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-10 00:05:18,396 c2fa1eca MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/c2fa1eca/pkb.log>
2018-05-10 00:05:18,397 c2fa1eca MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/c2fa1eca/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user jasonkuster@google.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user schroederc@google.com
Not sending mail to unregistered user shoyer@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1248
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1248/display/redirect?page=changes>
Changes:
[kedin] [SQL] Change DDL parser to create Beam Schema types instead of Calcite
[kedin] [SQL] Support arrays of primitive types in DDL
[kedin] [SQL] Parse Row fields
[kedin] [SQL] Remove Column. Replace with Schema
[kedin] [SQL] Validate Table schemas in BeamSqlCliTest
[kedin] [SQL] Add support for angular brackets array definition
[kedin] [SQL] Add support for Maps in DDL
[kedin] [SQL] Add support for angular brackets Row syntax
[kedin] [SQL] Fix rebase conflicts
[yifanzou] update machine numbers to 1..16 for inventory tests
[kedin] [SQL] Add QuickCheck and tests for DDL schema verification
[kedin] [SQL] Remove support for postfix array declaration in DDL
[Pablo] StateSampler knows the execution thread it tracks.
[github] Revert "Enable githubCommitNotifier for post commits"
------------------------------------------
[...truncated 4.69 KB...]
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins122492978219254013.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/aa/063eca6a416f397bd99552c534c6d11d57f58f2e94c14780f3bbf818c4cf/monotonic-1.5-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.5 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8769515154695154695.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-09 19:09:58,835 6da85548 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/6da85548/pkb.log>
2018-05-09 19:09:58,836 6da85548 MainThread INFO PerfKitBenchmarker version: v1.12.0-580-g36cfc82
2018-05-09 19:09:58,837 6da85548 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-09 19:09:59,177 6da85548 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-09 19:09:59,352 6da85548 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-09 19:09:59,356 6da85548 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-09 19:09:59,357 6da85548 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-09 19:10:04,310 6da85548 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/6da85548/beam/gradlew> clean install -xcheck --stacktrace -DintegrationTestRunner=dataflow
2018-05-09 19:14:02,936 6da85548 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-09 19:14:02,946 6da85548 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/6da85548/beam/sdks/python/target>
2018-05-09 19:14:02,948 6da85548 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-09 19:14:02,950 6da85548 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 781, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/6da85548/beam/sdks/python/target>
2018-05-09 19:14:02,950 6da85548 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-09 19:14:02,960 6da85548 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-09 19:14:02,961 6da85548 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/6da85548/pkb.log>
2018-05-09 19:14:02,961 6da85548 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/6da85548/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user jasonkuster@google.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user schroederc@google.com
Not sending mail to unregistered user shoyer@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1247
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1247/display/redirect>
------------------------------------------
[...truncated 9.21 KB...]
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/aa/063eca6a416f397bd99552c534c6d11d57f58f2e94c14780f3bbf818c4cf/monotonic-1.5-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.5 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5460359802169586681.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-09 12:00:37,807 99050ccc MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/99050ccc/pkb.log>
2018-05-09 12:00:37,808 99050ccc MainThread INFO PerfKitBenchmarker version: v1.12.0-580-g36cfc82
2018-05-09 12:00:37,809 99050ccc MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-09 12:00:38,178 99050ccc MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-09 12:00:38,345 99050ccc MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-09 12:00:38,349 99050ccc MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-09 12:00:38,349 99050ccc MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-09 12:00:43,641 99050ccc MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/99050ccc/beam/gradlew> clean install -xcheck --stacktrace -DintegrationTestRunner=dataflow
2018-05-09 12:04:59,730 99050ccc MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-09 12:04:59,744 99050ccc MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/99050ccc/beam/sdks/python/target>
2018-05-09 12:04:59,746 99050ccc MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-09 12:04:59,749 99050ccc MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 781, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/99050ccc/beam/sdks/python/target>
2018-05-09 12:04:59,749 99050ccc MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-09 12:04:59,765 99050ccc MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-09 12:04:59,766 99050ccc MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/99050ccc/pkb.log>
2018-05-09 12:04:59,766 99050ccc MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/99050ccc/completion_statuses.json>
Build step 'Execute shell' marked build as failure
[Set GitHub commit status (universal)] ERROR on repos [GHRepository@86299f5[description=Apache Beam,homepage=,name=beam,fork=false,size=58754,milestones={},language=Java,commits={},source=<null>,parent=<null>,responseHeaderFields={null=[HTTP/1.1 200 OK], Access-Control-Allow-Origin=[*], Access-Control-Expose-Headers=[ETag, Link, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval], Cache-Control=[private, max-age=60, s-maxage=60], Content-Encoding=[gzip], Content-Security-Policy=[default-src 'none'], Content-Type=[application/json; charset=utf-8], Date=[Wed, 09 May 2018 12:05:00 GMT], ETag=[W/"c9c57ab721dc1f578ccd914bb574f9f9"], Last-Modified=[Wed, 09 May 2018 10:15:34 GMT], OkHttp-Received-Millis=[1525867500544], OkHttp-Response-Source=[NETWORK 200], OkHttp-Selected-Protocol=[http/1.1], OkHttp-Sent-Millis=[1525867500393], Referrer-Policy=[origin-when-cross-origin, strict-origin-when-cross-origin], Server=[GitHub.com], Status=[200 OK], Strict-Transport-Security=[max-age=31536000; includeSubdomains; preload], Transfer-Encoding=[chunked], Vary=[Accept, Authorization, Cookie, X-GitHub-OTP], X-Accepted-OAuth-Scopes=[repo], X-Content-Type-Options=[nosniff], X-Frame-Options=[deny], X-GitHub-Media-Type=[github.v3; format=json], X-GitHub-Request-Id=[948A:4F42:12D28AE:2634ECF:5AF2E3B2], X-OAuth-Scopes=[admin:repo_hook, repo, repo:status], X-RateLimit-Limit=[5000], X-RateLimit-Remaining=[4947], X-RateLimit-Reset=[1525867519], X-Runtime-rack=[0.068357], X-XSS-Protection=[1; mode=block]},url=https://api.github.com/repos/apache/beam,id=50904245]] (sha:60f90c8) with context:beam_PerformanceTests_Python
Setting commit status on GitHub for https://github.com/apache/beam/commit/60f90c8dcb229c35a82c7be15e64a89678bae058
ERROR: Build step failed with exception
java.io.FileNotFoundException: https://api.github.com/repos/apache/beam/statuses/60f90c8dcb229c35a82c7be15e64a89678bae058
at com.squareup.okhttp.internal.huc.HttpURLConnectionImpl.getInputStream(HttpURLConnectionImpl.java:243)
at com.squareup.okhttp.internal.huc.DelegatingHttpsURLConnection.getInputStream(DelegatingHttpsURLConnection.java:210)
at com.squareup.okhttp.internal.huc.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:25)
at org.kohsuke.github.Requester.parse(Requester.java:612)
at org.kohsuke.github.Requester.parse(Requester.java:594)
at org.kohsuke.github.Requester._to(Requester.java:272)
Caused: org.kohsuke.github.GHFileNotFoundException: {"message":"Not Found","documentation_url":"https://developer.github.com/v3/repos/statuses/#create-a-status"}
at org.kohsuke.github.Requester.handleApiError(Requester.java:686)
at org.kohsuke.github.Requester._to(Requester.java:293)
at org.kohsuke.github.Requester.to(Requester.java:234)
at org.kohsuke.github.GHRepository.createCommitStatus(GHRepository.java:1075)
at org.jenkinsci.plugins.github.status.GitHubCommitStatusSetter.perform(GitHubCommitStatusSetter.java:160)
Caused: org.jenkinsci.plugins.github.common.CombineErrorHandler$ErrorHandlingException
at org.jenkinsci.plugins.github.common.CombineErrorHandler.handle(CombineErrorHandler.java:74)
at org.jenkinsci.plugins.github.status.GitHubCommitStatusSetter.perform(GitHubCommitStatusSetter.java:164)
at com.cloudbees.jenkins.GitHubCommitNotifier.perform(GitHubCommitNotifier.java:151)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
at hudson.model.Run.execute(Run.java:1749)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Build step 'Set build status on GitHub commit [deprecated]' marked build as failure
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user jasonkuster@google.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user schroederc@google.com
Not sending mail to unregistered user shoyer@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1246
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1246/display/redirect?page=changes>
Changes:
[Pablo] Make experiments as set attr of RuntimeValueProvider
------------------------------------------
[...truncated 9.21 KB...]
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/aa/063eca6a416f397bd99552c534c6d11d57f58f2e94c14780f3bbf818c4cf/monotonic-1.5-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.5 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8165186307560750003.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-09 06:00:37,363 fc32181a MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/fc32181a/pkb.log>
2018-05-09 06:00:37,364 fc32181a MainThread INFO PerfKitBenchmarker version: v1.12.0-580-g36cfc82
2018-05-09 06:00:37,365 fc32181a MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-09 06:00:37,762 fc32181a MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-09 06:00:37,929 fc32181a MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-09 06:00:37,933 fc32181a MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-09 06:00:37,934 fc32181a MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-09 06:00:43,390 fc32181a MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/fc32181a/beam/gradlew> clean install -xcheck --stacktrace -DintegrationTestRunner=dataflow
2018-05-09 06:05:35,136 fc32181a MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-09 06:05:35,150 fc32181a MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/fc32181a/beam/sdks/python/target>
2018-05-09 06:05:35,151 fc32181a MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-09 06:05:35,154 fc32181a MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 781, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/fc32181a/beam/sdks/python/target>
2018-05-09 06:05:35,155 fc32181a MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-09 06:05:35,165 fc32181a MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-09 06:05:35,166 fc32181a MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/fc32181a/pkb.log>
2018-05-09 06:05:35,166 fc32181a MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/fc32181a/completion_statuses.json>
Build step 'Execute shell' marked build as failure
[Set GitHub commit status (universal)] ERROR on repos [GHRepository@3e9a4f53[description=Apache Beam,homepage=,name=beam,fork=false,size=58754,milestones={},language=Java,commits={},source=<null>,parent=<null>,responseHeaderFields={null=[HTTP/1.1 200 OK], Access-Control-Allow-Origin=[*], Access-Control-Expose-Headers=[ETag, Link, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval], Cache-Control=[private, max-age=60, s-maxage=60], Content-Encoding=[gzip], Content-Security-Policy=[default-src 'none'], Content-Type=[application/json; charset=utf-8], Date=[Wed, 09 May 2018 06:05:35 GMT], ETag=[W/"b325fcee1bcdcb6673ffb3a642ac3060"], Last-Modified=[Wed, 09 May 2018 00:26:13 GMT], OkHttp-Received-Millis=[1525845935908], OkHttp-Response-Source=[NETWORK 200], OkHttp-Selected-Protocol=[http/1.1], OkHttp-Sent-Millis=[1525845935761], Referrer-Policy=[origin-when-cross-origin, strict-origin-when-cross-origin], Server=[GitHub.com], Status=[200 OK], Strict-Transport-Security=[max-age=31536000; includeSubdomains; preload], Transfer-Encoding=[chunked], Vary=[Accept, Authorization, Cookie, X-GitHub-OTP], X-Accepted-OAuth-Scopes=[repo], X-Content-Type-Options=[nosniff], X-Frame-Options=[deny], X-GitHub-Media-Type=[github.v3; format=json], X-GitHub-Request-Id=[DE20:4F40:989E8C:14F5A45:5AF28F9A], X-OAuth-Scopes=[admin:repo_hook, repo, repo:status], X-RateLimit-Limit=[5000], X-RateLimit-Remaining=[4967], X-RateLimit-Reset=[1525848688], X-Runtime-rack=[0.051818], X-XSS-Protection=[1; mode=block]},url=https://api.github.com/repos/apache/beam,id=50904245]] (sha:60f90c8) with context:beam_PerformanceTests_Python
Setting commit status on GitHub for https://github.com/apache/beam/commit/60f90c8dcb229c35a82c7be15e64a89678bae058
ERROR: Build step failed with exception
java.io.FileNotFoundException: https://api.github.com/repos/apache/beam/statuses/60f90c8dcb229c35a82c7be15e64a89678bae058
at com.squareup.okhttp.internal.huc.HttpURLConnectionImpl.getInputStream(HttpURLConnectionImpl.java:243)
at com.squareup.okhttp.internal.huc.DelegatingHttpsURLConnection.getInputStream(DelegatingHttpsURLConnection.java:210)
at com.squareup.okhttp.internal.huc.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:25)
at org.kohsuke.github.Requester.parse(Requester.java:612)
at org.kohsuke.github.Requester.parse(Requester.java:594)
at org.kohsuke.github.Requester._to(Requester.java:272)
Caused: org.kohsuke.github.GHFileNotFoundException: {"message":"Not Found","documentation_url":"https://developer.github.com/v3/repos/statuses/#create-a-status"}
at org.kohsuke.github.Requester.handleApiError(Requester.java:686)
at org.kohsuke.github.Requester._to(Requester.java:293)
at org.kohsuke.github.Requester.to(Requester.java:234)
at org.kohsuke.github.GHRepository.createCommitStatus(GHRepository.java:1075)
at org.jenkinsci.plugins.github.status.GitHubCommitStatusSetter.perform(GitHubCommitStatusSetter.java:160)
Caused: org.jenkinsci.plugins.github.common.CombineErrorHandler$ErrorHandlingException
at org.jenkinsci.plugins.github.common.CombineErrorHandler.handle(CombineErrorHandler.java:74)
at org.jenkinsci.plugins.github.status.GitHubCommitStatusSetter.perform(GitHubCommitStatusSetter.java:164)
at com.cloudbees.jenkins.GitHubCommitNotifier.perform(GitHubCommitNotifier.java:151)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
at hudson.model.Run.execute(Run.java:1749)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Build step 'Set build status on GitHub commit [deprecated]' marked build as failure
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user jasonkuster@google.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user schroederc@google.com
Not sending mail to unregistered user shoyer@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1245
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1245/display/redirect?page=changes>
Changes:
[github] Explictly delcare globals defined elsewhere
[mairbek] Introduced SpannerWriteResult that
[mairbek] Addressed comments
[mairbek] Happy checkstyle
[github] Adding a microbenchmark for side input iterables. (#5294)
[apilloud] Enable githubCommitNotifier for post commits
[tgroh] Migrate the `portable` subpackage to Portability
------------------------------------------
[...truncated 5.12 KB...]
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1693927279099916115.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/aa/063eca6a416f397bd99552c534c6d11d57f58f2e94c14780f3bbf818c4cf/monotonic-1.5-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.5 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1423956943309251479.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-09 00:22:47,596 487ca9f1 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/487ca9f1/pkb.log>
2018-05-09 00:22:47,596 487ca9f1 MainThread INFO PerfKitBenchmarker version: v1.12.0-580-g36cfc82
2018-05-09 00:22:47,597 487ca9f1 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-09 00:22:47,873 487ca9f1 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-09 00:22:48,031 487ca9f1 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-09 00:22:48,035 487ca9f1 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-09 00:22:48,035 487ca9f1 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-09 00:22:53,199 487ca9f1 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/487ca9f1/beam/gradlew> clean install -xcheck --stacktrace -DintegrationTestRunner=dataflow
2018-05-09 00:26:48,573 487ca9f1 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-09 00:26:48,584 487ca9f1 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/487ca9f1/beam/sdks/python/target>
2018-05-09 00:26:48,585 487ca9f1 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-09 00:26:48,588 487ca9f1 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 781, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/487ca9f1/beam/sdks/python/target>
2018-05-09 00:26:48,588 487ca9f1 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-09 00:26:48,630 487ca9f1 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-09 00:26:48,631 487ca9f1 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/487ca9f1/pkb.log>
2018-05-09 00:26:48,631 487ca9f1 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/487ca9f1/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user jasonkuster@google.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user schroederc@google.com
Not sending mail to unregistered user shoyer@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1244
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1244/display/redirect?page=changes>
Changes:
[schroederc] Move textio.FileSystem and its registry to a separate package
[schroederc] Move filesystem implementations under beam/io/filesystem package
[wcn] Fix missing return in datastore connector.
[swegner] Create ValidatesRunner task for DirectRunner.
[swegner] Convert some tests from ValidatesRunner -> NeedsRunner
[apilloud] [SQL] Cleanout unneeded sqlEnv
[apilloud] [SQL] Hide details of BeamSqlEnv from rel test
[apilloud] [SQL] Simplify TableProvider interface
[apilloud] [SQL] Return map from TableProvider.getTables
[apilloud] [SQL] Add tables via TableStore in CalciteSchema
[apilloud] [SQL] Move Create and Drop Table inline with ddl
[tgroh] Suppress Thread Leak Test
[tgroh] Rename ExecutorServiceParlallelExecutorTest#test
------------------------------------------
[...truncated 4.74 KB...]
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4320043418822733879.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/aa/063eca6a416f397bd99552c534c6d11d57f58f2e94c14780f3bbf818c4cf/monotonic-1.5-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.5 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1124193938476074160.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-08 18:00:37,274 663fc479 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/663fc479/pkb.log>
2018-05-08 18:00:37,275 663fc479 MainThread INFO PerfKitBenchmarker version: v1.12.0-580-g36cfc82
2018-05-08 18:00:37,275 663fc479 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-08 18:00:37,484 663fc479 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-08 18:00:37,631 663fc479 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-08 18:00:37,635 663fc479 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-08 18:00:37,635 663fc479 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-08 18:00:42,535 663fc479 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/663fc479/beam/gradlew> clean install -xcheck --stacktrace -DintegrationTestRunner=dataflow
2018-05-08 18:04:53,410 663fc479 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-08 18:04:53,420 663fc479 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/663fc479/beam/sdks/python/target>
2018-05-08 18:04:53,421 663fc479 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-08 18:04:53,422 663fc479 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 781, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/663fc479/beam/sdks/python/target>
2018-05-08 18:04:53,423 663fc479 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-08 18:04:53,448 663fc479 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-08 18:04:53,449 663fc479 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/663fc479/pkb.log>
2018-05-08 18:04:53,449 663fc479 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/663fc479/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user jasonkuster@google.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user shoyer@google.com
Not sending mail to unregistered user schroederc@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1243
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1243/display/redirect>
------------------------------------------
[...truncated 4.52 KB...]
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins477075854833080956.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/aa/063eca6a416f397bd99552c534c6d11d57f58f2e94c14780f3bbf818c4cf/monotonic-1.5-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.5 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5332734419671954790.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-08 12:00:43,852 9097e4d9 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/9097e4d9/pkb.log>
2018-05-08 12:00:43,853 9097e4d9 MainThread INFO PerfKitBenchmarker version: v1.12.0-578-g1836465
2018-05-08 12:00:43,854 9097e4d9 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-08 12:00:44,083 9097e4d9 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-08 12:00:44,245 9097e4d9 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-08 12:00:44,249 9097e4d9 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-08 12:00:44,249 9097e4d9 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-08 12:00:49,441 9097e4d9 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/9097e4d9/beam/gradlew> clean install -xcheck --stacktrace -DintegrationTestRunner=dataflow
2018-05-08 12:05:02,447 9097e4d9 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-08 12:05:02,457 9097e4d9 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/9097e4d9/beam/sdks/python/target>
2018-05-08 12:05:02,458 9097e4d9 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-08 12:05:02,460 9097e4d9 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 781, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/9097e4d9/beam/sdks/python/target>
2018-05-08 12:05:02,460 9097e4d9 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-08 12:05:02,488 9097e4d9 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-08 12:05:02,489 9097e4d9 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/9097e4d9/pkb.log>
2018-05-08 12:05:02,490 9097e4d9 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/9097e4d9/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user jasonkuster@google.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user shoyer@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1242
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1242/display/redirect?page=changes>
Changes:
[daniel.o.programmer] Change to docstring in filesystem.py
[github] Moving Python PostCommit into a Gradle task. (#5289)
------------------------------------------
[...truncated 4.53 KB...]
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3370853666288046184.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/aa/063eca6a416f397bd99552c534c6d11d57f58f2e94c14780f3bbf818c4cf/monotonic-1.5-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.5 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins261813762147085139.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-08 06:00:38,217 469eec03 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/469eec03/pkb.log>
2018-05-08 06:00:38,218 469eec03 MainThread INFO PerfKitBenchmarker version: v1.12.0-578-g1836465
2018-05-08 06:00:38,219 469eec03 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-08 06:00:38,621 469eec03 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-08 06:00:38,779 469eec03 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-08 06:00:38,782 469eec03 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-08 06:00:38,783 469eec03 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-08 06:00:44,181 469eec03 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/469eec03/beam/gradlew> clean install -xcheck --stacktrace -DintegrationTestRunner=dataflow
2018-05-08 06:05:28,127 469eec03 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-08 06:05:28,136 469eec03 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/469eec03/beam/sdks/python/target>
2018-05-08 06:05:28,137 469eec03 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-08 06:05:28,139 469eec03 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 781, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/469eec03/beam/sdks/python/target>
2018-05-08 06:05:28,139 469eec03 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-08 06:05:28,198 469eec03 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-08 06:05:28,199 469eec03 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/469eec03/pkb.log>
2018-05-08 06:05:28,199 469eec03 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/469eec03/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user jasonkuster@google.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user shoyer@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1241
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1241/display/redirect?page=changes>
Changes:
[thw] [BEAM-4246] Remove Maven build directory dependency in Apex runner.
[thw] Upgrade Apex version to 3.7.0 to get rid of poisened netlet 1.3.0
------------------------------------------
[...truncated 4.55 KB...]
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4753164093500792055.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/aa/063eca6a416f397bd99552c534c6d11d57f58f2e94c14780f3bbf818c4cf/monotonic-1.5-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.5 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6459374300070367387.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-08 00:00:40,684 74b2afca MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/74b2afca/pkb.log>
2018-05-08 00:00:40,684 74b2afca MainThread INFO PerfKitBenchmarker version: v1.12.0-578-g1836465
2018-05-08 00:00:40,685 74b2afca MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-08 00:00:41,194 74b2afca MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-08 00:00:41,381 74b2afca MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-08 00:00:41,385 74b2afca MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-08 00:00:41,386 74b2afca MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-08 00:00:47,591 74b2afca MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/74b2afca/beam/gradlew> clean install -xcheck --stacktrace -DintegrationTestRunner=dataflow
2018-05-08 00:10:08,648 74b2afca MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-08 00:10:08,662 74b2afca MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/74b2afca/beam/sdks/python/target>
2018-05-08 00:10:08,664 74b2afca MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-08 00:10:08,666 74b2afca MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 781, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/74b2afca/beam/sdks/python/target>
2018-05-08 00:10:08,667 74b2afca MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-08 00:10:08,705 74b2afca MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-08 00:10:08,706 74b2afca MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/74b2afca/pkb.log>
2018-05-08 00:10:08,707 74b2afca MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/74b2afca/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user jasonkuster@google.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user shoyer@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1240
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1240/display/redirect>
------------------------------------------
[...truncated 4.54 KB...]
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4661909573531510816.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/aa/063eca6a416f397bd99552c534c6d11d57f58f2e94c14780f3bbf818c4cf/monotonic-1.5-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.5 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins14739166260006659.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-07 18:01:02,030 f72fcf8f MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/f72fcf8f/pkb.log>
2018-05-07 18:01:02,031 f72fcf8f MainThread INFO PerfKitBenchmarker version: v1.12.0-578-g1836465
2018-05-07 18:01:02,033 f72fcf8f MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-07 18:01:02,453 f72fcf8f MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-07 18:01:02,707 f72fcf8f MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-07 18:01:02,711 f72fcf8f MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-07 18:01:02,712 f72fcf8f MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-07 18:01:09,683 f72fcf8f MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/f72fcf8f/beam/gradlew> clean install -xcheck --stacktrace -DintegrationTestRunner=dataflow
2018-05-07 18:06:44,809 f72fcf8f MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-07 18:06:44,822 f72fcf8f MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/f72fcf8f/beam/sdks/python/target>
2018-05-07 18:06:44,824 f72fcf8f MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-07 18:06:44,826 f72fcf8f MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 781, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/f72fcf8f/beam/sdks/python/target>
2018-05-07 18:06:44,827 f72fcf8f MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-07 18:06:44,884 f72fcf8f MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-07 18:06:44,885 f72fcf8f MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/f72fcf8f/pkb.log>
2018-05-07 18:06:44,886 f72fcf8f MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/f72fcf8f/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user jasonkuster@google.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user shoyer@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1239
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1239/display/redirect>
------------------------------------------
[...truncated 4.54 KB...]
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4998656870473128651.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/aa/063eca6a416f397bd99552c534c6d11d57f58f2e94c14780f3bbf818c4cf/monotonic-1.5-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.5 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1042977088663791323.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-07 12:00:38,770 065bb00f MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/065bb00f/pkb.log>
2018-05-07 12:00:38,770 065bb00f MainThread INFO PerfKitBenchmarker version: v1.12.0-577-g0d6a9aa
2018-05-07 12:00:38,771 065bb00f MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-07 12:00:39,017 065bb00f MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-07 12:00:39,295 065bb00f MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-07 12:00:39,302 065bb00f MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-07 12:00:39,303 065bb00f MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-07 12:00:45,149 065bb00f MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/065bb00f/beam/gradlew> clean install -xcheck --stacktrace -DintegrationTestRunner=dataflow
2018-05-07 12:05:52,049 065bb00f MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-07 12:05:52,062 065bb00f MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/065bb00f/beam/sdks/python/target>
2018-05-07 12:05:52,063 065bb00f MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-07 12:05:52,065 065bb00f MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 781, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/065bb00f/beam/sdks/python/target>
2018-05-07 12:05:52,066 065bb00f MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-07 12:05:52,105 065bb00f MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-07 12:05:52,106 065bb00f MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/065bb00f/pkb.log>
2018-05-07 12:05:52,107 065bb00f MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/065bb00f/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user jasonkuster@google.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user shoyer@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1238
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1238/display/redirect>
------------------------------------------
[...truncated 4.54 KB...]
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2000152233350451449.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/aa/063eca6a416f397bd99552c534c6d11d57f58f2e94c14780f3bbf818c4cf/monotonic-1.5-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.5 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins9107637541295296419.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-07 06:00:41,294 8eb482dc MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/8eb482dc/pkb.log>
2018-05-07 06:00:41,294 8eb482dc MainThread INFO PerfKitBenchmarker version: v1.12.0-577-g0d6a9aa
2018-05-07 06:00:41,295 8eb482dc MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-07 06:00:41,501 8eb482dc MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-07 06:00:41,664 8eb482dc MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-07 06:00:41,668 8eb482dc MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-07 06:00:41,669 8eb482dc MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-07 06:00:46,971 8eb482dc MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/8eb482dc/beam/gradlew> clean install -xcheck --stacktrace -DintegrationTestRunner=dataflow
2018-05-07 06:04:55,495 8eb482dc MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-07 06:04:55,506 8eb482dc MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/8eb482dc/beam/sdks/python/target>
2018-05-07 06:04:55,507 8eb482dc MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-07 06:04:55,509 8eb482dc MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 781, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/8eb482dc/beam/sdks/python/target>
2018-05-07 06:04:55,509 8eb482dc MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-07 06:04:55,527 8eb482dc MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-07 06:04:55,528 8eb482dc MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/8eb482dc/pkb.log>
2018-05-07 06:04:55,529 8eb482dc MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/8eb482dc/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user jasonkuster@google.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user shoyer@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1237
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1237/display/redirect>
------------------------------------------
[...truncated 4.54 KB...]
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7554523155013107655.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/aa/063eca6a416f397bd99552c534c6d11d57f58f2e94c14780f3bbf818c4cf/monotonic-1.5-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.5 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3655198860506791688.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-07 00:00:35,661 b22c5959 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/b22c5959/pkb.log>
2018-05-07 00:00:35,661 b22c5959 MainThread INFO PerfKitBenchmarker version: v1.12.0-577-g0d6a9aa
2018-05-07 00:00:35,662 b22c5959 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-07 00:00:36,042 b22c5959 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-07 00:00:36,232 b22c5959 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-07 00:00:36,235 b22c5959 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-07 00:00:36,236 b22c5959 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-07 00:00:41,922 b22c5959 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/b22c5959/beam/gradlew> clean install -xcheck --stacktrace -DintegrationTestRunner=dataflow
2018-05-07 00:06:13,506 b22c5959 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-07 00:06:13,520 b22c5959 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/b22c5959/beam/sdks/python/target>
2018-05-07 00:06:13,522 b22c5959 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-07 00:06:13,525 b22c5959 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 781, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/b22c5959/beam/sdks/python/target>
2018-05-07 00:06:13,526 b22c5959 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-07 00:06:13,558 b22c5959 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-07 00:06:13,559 b22c5959 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/b22c5959/pkb.log>
2018-05-07 00:06:13,560 b22c5959 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/b22c5959/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user jasonkuster@google.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user shoyer@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1236
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1236/display/redirect>
------------------------------------------
[...truncated 4.54 KB...]
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4485151787855792818.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/aa/063eca6a416f397bd99552c534c6d11d57f58f2e94c14780f3bbf818c4cf/monotonic-1.5-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.5 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5147281323271792884.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-06 18:00:41,791 37dc18da MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/37dc18da/pkb.log>
2018-05-06 18:00:41,792 37dc18da MainThread INFO PerfKitBenchmarker version: v1.12.0-577-g0d6a9aa
2018-05-06 18:00:41,793 37dc18da MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-06 18:00:42,113 37dc18da MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-06 18:00:42,332 37dc18da MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-06 18:00:42,337 37dc18da MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-06 18:00:42,338 37dc18da MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-06 18:00:48,161 37dc18da MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/37dc18da/beam/gradlew> clean install -xcheck --stacktrace -DintegrationTestRunner=dataflow
2018-05-06 18:06:43,404 37dc18da MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-06 18:06:43,416 37dc18da MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/37dc18da/beam/sdks/python/target>
2018-05-06 18:06:43,418 37dc18da MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-06 18:06:43,420 37dc18da MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 781, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/37dc18da/beam/sdks/python/target>
2018-05-06 18:06:43,421 37dc18da MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-06 18:06:43,436 37dc18da MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-06 18:06:43,437 37dc18da MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/37dc18da/pkb.log>
2018-05-06 18:06:43,437 37dc18da MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/37dc18da/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user jasonkuster@google.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user shoyer@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1235
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1235/display/redirect>
------------------------------------------
[...truncated 4.54 KB...]
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4200929610893318071.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/aa/063eca6a416f397bd99552c534c6d11d57f58f2e94c14780f3bbf818c4cf/monotonic-1.5-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.5 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1217229940661743471.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-06 12:00:37,795 e296e3cd MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e296e3cd/pkb.log>
2018-05-06 12:00:37,796 e296e3cd MainThread INFO PerfKitBenchmarker version: v1.12.0-577-g0d6a9aa
2018-05-06 12:00:37,797 e296e3cd MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-06 12:00:37,995 e296e3cd MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-06 12:00:38,166 e296e3cd MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-06 12:00:38,169 e296e3cd MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-06 12:00:38,170 e296e3cd MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-06 12:00:43,481 e296e3cd MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e296e3cd/beam/gradlew> clean install -xcheck --stacktrace -DintegrationTestRunner=dataflow
2018-05-06 12:05:17,129 e296e3cd MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-06 12:05:17,141 e296e3cd MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e296e3cd/beam/sdks/python/target>
2018-05-06 12:05:17,142 e296e3cd MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-06 12:05:17,144 e296e3cd MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 781, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e296e3cd/beam/sdks/python/target>
2018-05-06 12:05:17,145 e296e3cd MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-06 12:05:17,193 e296e3cd MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-06 12:05:17,194 e296e3cd MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e296e3cd/pkb.log>
2018-05-06 12:05:17,195 e296e3cd MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e296e3cd/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user jasonkuster@google.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user shoyer@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1234
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1234/display/redirect>
------------------------------------------
[...truncated 4.54 KB...]
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5934797366180056880.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/aa/063eca6a416f397bd99552c534c6d11d57f58f2e94c14780f3bbf818c4cf/monotonic-1.5-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.5 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4496668234875030015.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-06 06:00:33,170 a572d4d3 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/a572d4d3/pkb.log>
2018-05-06 06:00:33,170 a572d4d3 MainThread INFO PerfKitBenchmarker version: v1.12.0-577-g0d6a9aa
2018-05-06 06:00:33,171 a572d4d3 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-06 06:00:33,607 a572d4d3 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-06 06:00:33,771 a572d4d3 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-06 06:00:33,774 a572d4d3 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-06 06:00:33,775 a572d4d3 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-06 06:00:38,921 a572d4d3 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/a572d4d3/beam/gradlew> clean install -xcheck --stacktrace -DintegrationTestRunner=dataflow
2018-05-06 06:05:14,371 a572d4d3 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-06 06:05:14,381 a572d4d3 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/a572d4d3/beam/sdks/python/target>
2018-05-06 06:05:14,383 a572d4d3 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-06 06:05:14,384 a572d4d3 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 781, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/a572d4d3/beam/sdks/python/target>
2018-05-06 06:05:14,385 a572d4d3 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-06 06:05:14,443 a572d4d3 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-06 06:05:14,444 a572d4d3 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/a572d4d3/pkb.log>
2018-05-06 06:05:14,444 a572d4d3 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/a572d4d3/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user jasonkuster@google.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user shoyer@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1233
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1233/display/redirect>
------------------------------------------
[...truncated 4.54 KB...]
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins238767545631536820.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/aa/063eca6a416f397bd99552c534c6d11d57f58f2e94c14780f3bbf818c4cf/monotonic-1.5-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.5 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1651119597549796430.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-06 00:00:34,879 f0963711 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/f0963711/pkb.log>
2018-05-06 00:00:34,880 f0963711 MainThread INFO PerfKitBenchmarker version: v1.12.0-577-g0d6a9aa
2018-05-06 00:00:34,882 f0963711 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-06 00:00:35,138 f0963711 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-06 00:00:35,315 f0963711 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-06 00:00:35,318 f0963711 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-06 00:00:35,319 f0963711 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-06 00:00:41,315 f0963711 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/f0963711/beam/gradlew> clean install -xcheck --stacktrace -DintegrationTestRunner=dataflow
2018-05-06 00:05:59,334 f0963711 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-06 00:05:59,347 f0963711 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/f0963711/beam/sdks/python/target>
2018-05-06 00:05:59,349 f0963711 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-06 00:05:59,351 f0963711 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 781, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/f0963711/beam/sdks/python/target>
2018-05-06 00:05:59,352 f0963711 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-06 00:05:59,407 f0963711 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-06 00:05:59,408 f0963711 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/f0963711/pkb.log>
2018-05-06 00:05:59,408 f0963711 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/f0963711/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user jasonkuster@google.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user shoyer@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1232
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1232/display/redirect>
------------------------------------------
[...truncated 4.54 KB...]
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6226459328513373971.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/aa/063eca6a416f397bd99552c534c6d11d57f58f2e94c14780f3bbf818c4cf/monotonic-1.5-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.5 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7432409411430361103.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-05 18:02:22,058 0edebfde MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0edebfde/pkb.log>
2018-05-05 18:02:22,059 0edebfde MainThread INFO PerfKitBenchmarker version: v1.12.0-577-g0d6a9aa
2018-05-05 18:02:22,060 0edebfde MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-05 18:02:22,337 0edebfde MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-05 18:02:22,505 0edebfde MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-05 18:02:22,509 0edebfde MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-05 18:02:22,509 0edebfde MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-05 18:02:28,461 0edebfde MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0edebfde/beam/gradlew> clean install -xcheck --stacktrace -DintegrationTestRunner=dataflow
2018-05-05 18:07:40,899 0edebfde MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-05 18:07:40,909 0edebfde MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0edebfde/beam/sdks/python/target>
2018-05-05 18:07:40,911 0edebfde MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-05 18:07:40,913 0edebfde MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 781, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0edebfde/beam/sdks/python/target>
2018-05-05 18:07:40,914 0edebfde MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-05 18:07:40,930 0edebfde MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-05 18:07:40,931 0edebfde MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0edebfde/pkb.log>
2018-05-05 18:07:40,932 0edebfde MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0edebfde/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user jasonkuster@google.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user shoyer@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1231
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1231/display/redirect?page=changes>
Changes:
[apilloud] Revert "Merge pull request #5220: [BEAM-3983][SQL] Add BigQuery table
[apilloud] Revert "Merge pull request #4947: [BEAM-3983] Add utils for converting
------------------------------------------
[...truncated 4.54 KB...]
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3294619523682552139.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/aa/063eca6a416f397bd99552c534c6d11d57f58f2e94c14780f3bbf818c4cf/monotonic-1.5-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.5 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7187232489825388123.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-05 12:04:52,705 7d7d4d49 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/7d7d4d49/pkb.log>
2018-05-05 12:04:52,705 7d7d4d49 MainThread INFO PerfKitBenchmarker version: v1.12.0-577-g0d6a9aa
2018-05-05 12:04:52,706 7d7d4d49 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-05 12:04:53,198 7d7d4d49 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-05 12:04:53,370 7d7d4d49 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-05 12:04:53,374 7d7d4d49 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-05 12:04:53,374 7d7d4d49 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-05 12:04:58,462 7d7d4d49 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/7d7d4d49/beam/gradlew> clean install -xcheck --stacktrace -DintegrationTestRunner=dataflow
2018-05-05 12:09:49,399 7d7d4d49 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-05 12:09:49,409 7d7d4d49 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/7d7d4d49/beam/sdks/python/target>
2018-05-05 12:09:49,411 7d7d4d49 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-05 12:09:49,413 7d7d4d49 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 781, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/7d7d4d49/beam/sdks/python/target>
2018-05-05 12:09:49,413 7d7d4d49 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-05 12:09:49,443 7d7d4d49 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-05 12:09:49,443 7d7d4d49 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/7d7d4d49/pkb.log>
2018-05-05 12:09:49,444 7d7d4d49 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/7d7d4d49/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user jasonkuster@google.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user shoyer@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1230
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1230/display/redirect?page=changes>
Changes:
[coheigea] Combine nested if statements
------------------------------------------
[...truncated 4.49 KB...]
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7079395939582489318.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/aa/063eca6a416f397bd99552c534c6d11d57f58f2e94c14780f3bbf818c4cf/monotonic-1.5-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.5 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2685674630389068120.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-05 06:02:25,650 180360c0 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/180360c0/pkb.log>
2018-05-05 06:02:25,651 180360c0 MainThread INFO PerfKitBenchmarker version: v1.12.0-577-g0d6a9aa
2018-05-05 06:02:25,652 180360c0 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-05 06:02:25,990 180360c0 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-05 06:02:26,201 180360c0 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-05 06:02:26,206 180360c0 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-05 06:02:26,206 180360c0 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-05 06:02:31,567 180360c0 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/180360c0/beam/gradlew> clean install -xcheck --stacktrace -DintegrationTestRunner=dataflow
2018-05-05 06:06:23,434 180360c0 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-05 06:06:23,442 180360c0 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/180360c0/beam/sdks/python/target>
2018-05-05 06:06:23,443 180360c0 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-05 06:06:23,445 180360c0 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 781, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/180360c0/beam/sdks/python/target>
2018-05-05 06:06:23,445 180360c0 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-05 06:06:23,477 180360c0 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-05 06:06:23,478 180360c0 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/180360c0/pkb.log>
2018-05-05 06:06:23,478 180360c0 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/180360c0/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user jasonkuster@google.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user shoyer@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1229
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1229/display/redirect?page=changes>
Changes:
[mariagh] Update CoGroupByKey docstring
[sidhom] Use wildcard generics rather than Object in portable translator
[sidhom] [BEAM-4231] Runner utility for coder instantiation
------------------------------------------
[...truncated 4.51 KB...]
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins75125069342373121.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/aa/063eca6a416f397bd99552c534c6d11d57f58f2e94c14780f3bbf818c4cf/monotonic-1.5-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.5 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4842221025192894472.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-05 00:09:14,174 6bbbab9e MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/6bbbab9e/pkb.log>
2018-05-05 00:09:14,175 6bbbab9e MainThread INFO PerfKitBenchmarker version: v1.12.0-577-g0d6a9aa
2018-05-05 00:09:14,176 6bbbab9e MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-05 00:09:14,554 6bbbab9e MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-05 00:09:14,821 6bbbab9e MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-05 00:09:14,827 6bbbab9e MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-05 00:09:14,828 6bbbab9e MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-05 00:09:21,768 6bbbab9e MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/6bbbab9e/beam/gradlew> clean install -xcheck --stacktrace -DintegrationTestRunner=dataflow
2018-05-05 00:15:23,678 6bbbab9e MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-05 00:15:23,696 6bbbab9e MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/6bbbab9e/beam/sdks/python/target>
2018-05-05 00:15:23,698 6bbbab9e MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-05 00:15:23,701 6bbbab9e MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 781, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 647, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 527, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/6bbbab9e/beam/sdks/python/target>
2018-05-05 00:15:23,701 6bbbab9e MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-05 00:15:23,735 6bbbab9e MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-05 00:15:23,736 6bbbab9e MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/6bbbab9e/pkb.log>
2018-05-05 00:15:23,738 6bbbab9e MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/6bbbab9e/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user jasonkuster@google.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user shoyer@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1228
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1228/display/redirect?page=changes>
Changes:
[swegner] Remove Gradle worker cap.
[swegner] Re-enable additional Jenkins worker parallelism for Dataflow
[swegner] Adjust Jenkins Gradle memory usage based on max-workers
[swegner] Hard-code jenkins machines total memory until we find a working API
------------------------------------------
[...truncated 4.49 KB...]
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6915006597034856160.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/aa/063eca6a416f397bd99552c534c6d11d57f58f2e94c14780f3bbf818c4cf/monotonic-1.5-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.5 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6503675509702015280.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-04 18:04:17,845 8c2cc485 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/8c2cc485/pkb.log>
2018-05-04 18:04:17,846 8c2cc485 MainThread INFO PerfKitBenchmarker version: v1.12.0-573-gb2a0a68
2018-05-04 18:04:17,847 8c2cc485 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-04 18:04:18,272 8c2cc485 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-04 18:04:18,536 8c2cc485 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-04 18:04:18,541 8c2cc485 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-04 18:04:18,542 8c2cc485 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-04 18:04:25,240 8c2cc485 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/8c2cc485/beam/gradlew> clean install -xcheck --stacktrace -DintegrationTestRunner=dataflow
2018-05-04 18:09:36,636 8c2cc485 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-04 18:09:36,649 8c2cc485 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/8c2cc485/beam/sdks/python/target>
2018-05-04 18:09:36,651 8c2cc485 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-04 18:09:36,653 8c2cc485 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 780, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/8c2cc485/beam/sdks/python/target>
2018-05-04 18:09:36,653 8c2cc485 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-04 18:09:36,685 8c2cc485 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-04 18:09:36,686 8c2cc485 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/8c2cc485/pkb.log>
2018-05-04 18:09:36,686 8c2cc485 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/8c2cc485/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user jasonkuster@google.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user shoyer@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1227
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1227/display/redirect?page=changes>
Changes:
[iemejia] Make MetricsHttpSinkTest more concise and fix typo
------------------------------------------
[...truncated 4.52 KB...]
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5206099886400342274.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/aa/063eca6a416f397bd99552c534c6d11d57f58f2e94c14780f3bbf818c4cf/monotonic-1.5-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.5 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1882754806044232706.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-04 12:01:04,296 fa13bb7d MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/fa13bb7d/pkb.log>
2018-05-04 12:01:04,297 fa13bb7d MainThread INFO PerfKitBenchmarker version: v1.12.0-573-gb2a0a68
2018-05-04 12:01:04,297 fa13bb7d MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-04 12:01:04,788 fa13bb7d MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-04 12:01:04,948 fa13bb7d MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-04 12:01:04,952 fa13bb7d MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-04 12:01:04,952 fa13bb7d MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-04 12:01:11,002 fa13bb7d MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/fa13bb7d/beam/gradlew> clean install -xcheck --stacktrace -DintegrationTestRunner=dataflow
2018-05-04 12:05:41,317 fa13bb7d MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-04 12:05:41,324 fa13bb7d MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/fa13bb7d/beam/sdks/python/target>
2018-05-04 12:05:41,325 fa13bb7d MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-04 12:05:41,327 fa13bb7d MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 780, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/fa13bb7d/beam/sdks/python/target>
2018-05-04 12:05:41,327 fa13bb7d MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-04 12:05:41,381 fa13bb7d MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-04 12:05:41,382 fa13bb7d MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/fa13bb7d/pkb.log>
2018-05-04 12:05:41,382 fa13bb7d MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/fa13bb7d/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user jasonkuster@google.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user shoyer@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1226
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1226/display/redirect?page=changes>
Changes:
[tgroh] Add an Unsupported StateRequestHandler
[apilloud] [SQL] Use the same javacc and fmpp as calcite
[apilloud] [SQL] BeamQueryPlanner uses BeamSqlParserImpl
------------------------------------------
[...truncated 4.55 KB...]
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2383670572846156876.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/aa/063eca6a416f397bd99552c534c6d11d57f58f2e94c14780f3bbf818c4cf/monotonic-1.5-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.5 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3837318280685648430.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-04 06:00:45,206 38c943f1 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/38c943f1/pkb.log>
2018-05-04 06:00:45,207 38c943f1 MainThread INFO PerfKitBenchmarker version: v1.12.0-573-gb2a0a68
2018-05-04 06:00:45,207 38c943f1 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-04 06:00:45,501 38c943f1 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-04 06:00:45,658 38c943f1 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-04 06:00:45,661 38c943f1 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-04 06:00:45,662 38c943f1 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-04 06:00:50,828 38c943f1 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/38c943f1/beam/gradlew> clean install -xcheck --stacktrace -DintegrationTestRunner=dataflow
2018-05-04 06:05:16,462 38c943f1 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-04 06:05:16,470 38c943f1 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/38c943f1/beam/sdks/python/target>
2018-05-04 06:05:16,472 38c943f1 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-04 06:05:16,473 38c943f1 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 780, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/38c943f1/beam/sdks/python/target>
2018-05-04 06:05:16,474 38c943f1 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-04 06:05:16,501 38c943f1 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-04 06:05:16,502 38c943f1 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/38c943f1/pkb.log>
2018-05-04 06:05:16,502 38c943f1 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/38c943f1/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user jasonkuster@google.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user shoyer@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1225
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1225/display/redirect?page=changes>
Changes:
[apilloud] [SQL] Add BeamEnumerableConverter
[mairbek] Templatize host name in SpannerIO
[coheigea] A few small fixes to catch a few regressions - put String literals
[sidhom] Move SyntheticNodes to top-level construction package and rename
[sidhom] Pull Flink execution environment utilities into static methods
[sidhom] Refactor Runner/SDK wire coder creation into a shared utility
[sidhom] [BEAM-3972] Translate portable batch pipelines by proto
[sidhom] FIXUP: Use ModelCoders to create proto coders and use WireCoders to
[sidhom] FIXUP: Use RunnerApi in WireCoders to prepare for coder instantiation
[tgroh] Add additional methods to QueryablePipeline
[tgroh] Expand the ExecutableGraph interface
[tgroh] Add ProtoOverrides
[jasonkuster] Add labels to Go SDK Dataflow Runner.
[jasonkuster] Add verbose error message if unmarshaling fails for label flag.
------------------------------------------
[...truncated 4.55 KB...]
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins816417449346650119.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/aa/063eca6a416f397bd99552c534c6d11d57f58f2e94c14780f3bbf818c4cf/monotonic-1.5-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.5 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4561375221741013585.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-04 00:00:50,218 e9647bc1 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e9647bc1/pkb.log>
2018-05-04 00:00:50,219 e9647bc1 MainThread INFO PerfKitBenchmarker version: v1.12.0-573-gb2a0a68
2018-05-04 00:00:50,219 e9647bc1 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-04 00:00:50,477 e9647bc1 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-04 00:00:50,643 e9647bc1 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-04 00:00:50,647 e9647bc1 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-04 00:00:50,647 e9647bc1 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-04 00:00:55,795 e9647bc1 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e9647bc1/beam/gradlew> clean install -xcheck --stacktrace -DintegrationTestRunner=dataflow
2018-05-04 00:05:30,908 e9647bc1 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-04 00:05:30,917 e9647bc1 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e9647bc1/beam/sdks/python/target>
2018-05-04 00:05:30,918 e9647bc1 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-04 00:05:30,920 e9647bc1 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 780, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e9647bc1/beam/sdks/python/target>
2018-05-04 00:05:30,920 e9647bc1 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-04 00:05:30,940 e9647bc1 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-04 00:05:30,941 e9647bc1 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e9647bc1/pkb.log>
2018-05-04 00:05:30,941 e9647bc1 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e9647bc1/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user jasonkuster@google.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user shoyer@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1224
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1224/display/redirect>
------------------------------------------
[...truncated 4.46 KB...]
Using cached https://files.pythonhosted.org/packages/a2/71/8273a7eeed0aff6a854237ab5453bc9aa67deb49df4832801c21f0ff3782/contextlib2-0.5.5-py2.py3-none-any.whl
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins603971570518380326.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9f/ff/68860dec8b90424028ca54df8b9fc8cd1c47ef58710bd5e591ec8c619243/monotonic-1.4-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.4 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7785240956336259259.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-03 18:00:47,479 e3be401d MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e3be401d/pkb.log>
2018-05-03 18:00:47,479 e3be401d MainThread INFO PerfKitBenchmarker version: v1.12.0-567-g59dbd5d
2018-05-03 18:00:47,480 e3be401d MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-03 18:00:47,977 e3be401d MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-03 18:00:48,134 e3be401d MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-03 18:00:48,137 e3be401d MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-03 18:00:48,138 e3be401d MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-03 18:00:53,456 e3be401d MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e3be401d/beam/gradlew> clean install -xcheck --stacktrace -DintegrationTestRunner=dataflow
2018-05-03 18:05:30,260 e3be401d MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-03 18:05:30,270 e3be401d MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e3be401d/beam/sdks/python/target>
2018-05-03 18:05:30,271 e3be401d MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-03 18:05:30,273 e3be401d MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 780, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e3be401d/beam/sdks/python/target>
2018-05-03 18:05:30,273 e3be401d MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-03 18:05:30,329 e3be401d MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-03 18:05:30,329 e3be401d MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e3be401d/pkb.log>
2018-05-03 18:05:30,330 e3be401d MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e3be401d/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user shoyer@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1223
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1223/display/redirect?page=changes>
Changes:
[echauchot] [BEAM-4138] Support runners that do not support committed metrics in
------------------------------------------
[...truncated 4.46 KB...]
Using cached https://files.pythonhosted.org/packages/a2/71/8273a7eeed0aff6a854237ab5453bc9aa67deb49df4832801c21f0ff3782/contextlib2-0.5.5-py2.py3-none-any.whl
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3976313595851427305.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9f/ff/68860dec8b90424028ca54df8b9fc8cd1c47ef58710bd5e591ec8c619243/monotonic-1.4-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.4 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7804519026070567558.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-03 12:00:45,724 b284b6be MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/b284b6be/pkb.log>
2018-05-03 12:00:45,724 b284b6be MainThread INFO PerfKitBenchmarker version: v1.12.0-567-g59dbd5d
2018-05-03 12:00:45,725 b284b6be MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-03 12:00:46,110 b284b6be MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-03 12:00:46,274 b284b6be MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-03 12:00:46,291 b284b6be MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-03 12:00:46,292 b284b6be MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-03 12:00:51,514 b284b6be MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/b284b6be/beam/gradlew> clean install -xcheck --stacktrace -DintegrationTestRunner=dataflow
2018-05-03 12:05:13,777 b284b6be MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-03 12:05:13,785 b284b6be MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/b284b6be/beam/sdks/python/target>
2018-05-03 12:05:13,786 b284b6be MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-03 12:05:13,788 b284b6be MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 780, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/b284b6be/beam/sdks/python/target>
2018-05-03 12:05:13,788 b284b6be MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-03 12:05:13,839 b284b6be MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-03 12:05:13,840 b284b6be MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/b284b6be/pkb.log>
2018-05-03 12:05:13,840 b284b6be MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/b284b6be/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user shoyer@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1222
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1222/display/redirect?page=changes>
Changes:
[tgroh] Add KV and Window Coder Utilities to ModelCoders
[tgroh] Use ModelCoders in ProcessBundleDescriptors
[sidhom] Remove StateRequestHandler from JobBundleFactory
------------------------------------------
[...truncated 4.45 KB...]
Using cached https://files.pythonhosted.org/packages/a2/71/8273a7eeed0aff6a854237ab5453bc9aa67deb49df4832801c21f0ff3782/contextlib2-0.5.5-py2.py3-none-any.whl
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins513585465902491747.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9f/ff/68860dec8b90424028ca54df8b9fc8cd1c47ef58710bd5e591ec8c619243/monotonic-1.4-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.4 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8807632777923104948.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-03 06:01:29,557 4cad8c05 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/4cad8c05/pkb.log>
2018-05-03 06:01:29,558 4cad8c05 MainThread INFO PerfKitBenchmarker version: v1.12.0-567-g59dbd5d
2018-05-03 06:01:29,558 4cad8c05 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-03 06:01:29,886 4cad8c05 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-03 06:01:30,037 4cad8c05 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-03 06:01:30,040 4cad8c05 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-03 06:01:30,041 4cad8c05 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-03 06:01:35,386 4cad8c05 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/4cad8c05/beam/gradlew> clean install -xcheck --stacktrace -DintegrationTestRunner=dataflow
2018-05-03 06:06:07,692 4cad8c05 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-03 06:06:07,702 4cad8c05 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/4cad8c05/beam/sdks/python/target>
2018-05-03 06:06:07,703 4cad8c05 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-03 06:06:07,704 4cad8c05 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 780, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/4cad8c05/beam/sdks/python/target>
2018-05-03 06:06:07,705 4cad8c05 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-03 06:06:07,763 4cad8c05 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-03 06:06:07,764 4cad8c05 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/4cad8c05/pkb.log>
2018-05-03 06:06:07,764 4cad8c05 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/4cad8c05/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user shoyer@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1221
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1221/display/redirect?page=changes>
Changes:
[axelmagn] Create ArtifactSource Interface
[tgroh] Remove unused Test Utilities, Test
[axelmagn] Create RemoteBundle interface
[axelmagn] Create JobInfo for job declaration by operators.
[mairbek] Wait for input collection
[mairbek] Expose DatabaseAdminClient
[mairbek] Consistently use lower case table name and column names
[axelmagn] Create BundleFactory chain.
[mairbek] Addressed comments
------------------------------------------
[...truncated 4.42 KB...]
Using cached https://files.pythonhosted.org/packages/a2/71/8273a7eeed0aff6a854237ab5453bc9aa67deb49df4832801c21f0ff3782/contextlib2-0.5.5-py2.py3-none-any.whl
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4772389610378115721.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9f/ff/68860dec8b90424028ca54df8b9fc8cd1c47ef58710bd5e591ec8c619243/monotonic-1.4-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.4 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7671905243203665238.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-03 00:13:24,711 1cb54242 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/1cb54242/pkb.log>
2018-05-03 00:13:24,711 1cb54242 MainThread INFO PerfKitBenchmarker version: v1.12.0-550-g5a2f0ec
2018-05-03 00:13:24,712 1cb54242 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-03 00:13:25,073 1cb54242 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-03 00:13:25,228 1cb54242 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-03 00:13:25,232 1cb54242 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-03 00:13:25,232 1cb54242 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-03 00:13:30,023 1cb54242 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/1cb54242/beam/gradlew> clean install -xcheck --stacktrace -DintegrationTestRunner=dataflow
2018-05-03 00:17:59,583 1cb54242 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-03 00:17:59,594 1cb54242 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/1cb54242/beam/sdks/python/target>
2018-05-03 00:17:59,595 1cb54242 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-03 00:17:59,597 1cb54242 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 780, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/1cb54242/beam/sdks/python/target>
2018-05-03 00:17:59,597 1cb54242 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-03 00:17:59,611 1cb54242 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-03 00:17:59,612 1cb54242 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/1cb54242/pkb.log>
2018-05-03 00:17:59,612 1cb54242 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/1cb54242/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user shoyer@google.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user mairbek@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1220
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1220/display/redirect?page=changes>
Changes:
[github] Count compressed records with a long to avoid overflow
[alan] [BEAM-4218] Fix failing javadoc build
[github] Fix DynamicDestinations documentation
------------------------------------------
[...truncated 4.42 KB...]
Using cached https://files.pythonhosted.org/packages/a2/71/8273a7eeed0aff6a854237ab5453bc9aa67deb49df4832801c21f0ff3782/contextlib2-0.5.5-py2.py3-none-any.whl
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins998352039491693429.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9f/ff/68860dec8b90424028ca54df8b9fc8cd1c47ef58710bd5e591ec8c619243/monotonic-1.4-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.4 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4098664051267110337.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-02 18:00:40,051 64882461 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/64882461/pkb.log>
2018-05-02 18:00:40,051 64882461 MainThread INFO PerfKitBenchmarker version: v1.12.0-537-gb9fbad5
2018-05-02 18:00:40,052 64882461 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-02 18:00:40,341 64882461 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-02 18:00:40,498 64882461 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-02 18:00:40,501 64882461 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-02 18:00:40,502 64882461 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-02 18:00:45,506 64882461 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/64882461/beam/gradlew> clean install -xcheck --stacktrace -DintegrationTestRunner=dataflow
2018-05-02 18:05:08,594 64882461 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-02 18:05:08,604 64882461 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/64882461/beam/sdks/python/target>
2018-05-02 18:05:08,605 64882461 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-02 18:05:08,607 64882461 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 780, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/64882461/beam/sdks/python/target>
2018-05-02 18:05:08,608 64882461 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-02 18:05:08,665 64882461 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-02 18:05:08,666 64882461 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/64882461/pkb.log>
2018-05-02 18:05:08,666 64882461 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/64882461/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user shoyer@google.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user mairbek@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1219
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1219/display/redirect>
------------------------------------------
[...truncated 4.47 KB...]
Using cached https://files.pythonhosted.org/packages/a2/71/8273a7eeed0aff6a854237ab5453bc9aa67deb49df4832801c21f0ff3782/contextlib2-0.5.5-py2.py3-none-any.whl
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins39168637975386711.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9f/ff/68860dec8b90424028ca54df8b9fc8cd1c47ef58710bd5e591ec8c619243/monotonic-1.4-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.4 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8709922746120296298.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-02 12:00:35,427 1dccc1c9 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/1dccc1c9/pkb.log>
2018-05-02 12:00:35,428 1dccc1c9 MainThread INFO PerfKitBenchmarker version: v1.12.0-536-g43c0135
2018-05-02 12:00:35,428 1dccc1c9 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-02 12:00:35,653 1dccc1c9 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-02 12:00:35,809 1dccc1c9 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-02 12:00:35,812 1dccc1c9 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-02 12:00:35,812 1dccc1c9 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-02 12:00:40,756 1dccc1c9 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/1dccc1c9/beam/gradlew> clean install -xcheck --stacktrace -DintegrationTestRunner=dataflow
2018-05-02 12:04:30,243 1dccc1c9 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-02 12:04:30,252 1dccc1c9 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/1dccc1c9/beam/sdks/python/target>
2018-05-02 12:04:30,253 1dccc1c9 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-02 12:04:30,255 1dccc1c9 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 780, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/1dccc1c9/beam/sdks/python/target>
2018-05-02 12:04:30,255 1dccc1c9 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-02 12:04:30,299 1dccc1c9 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-02 12:04:30,300 1dccc1c9 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/1dccc1c9/pkb.log>
2018-05-02 12:04:30,300 1dccc1c9 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/1dccc1c9/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user shoyer@google.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user mairbek@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1218
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1218/display/redirect?page=changes>
Changes:
[github] Update perf.go: only trigger on processing bundles
------------------------------------------
[...truncated 4.47 KB...]
Using cached https://files.pythonhosted.org/packages/a2/71/8273a7eeed0aff6a854237ab5453bc9aa67deb49df4832801c21f0ff3782/contextlib2-0.5.5-py2.py3-none-any.whl
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5866105267809553540.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9f/ff/68860dec8b90424028ca54df8b9fc8cd1c47ef58710bd5e591ec8c619243/monotonic-1.4-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.4 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6644645470118232010.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-02 06:00:56,861 23dbc788 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/23dbc788/pkb.log>
2018-05-02 06:00:56,862 23dbc788 MainThread INFO PerfKitBenchmarker version: v1.12.0-536-g43c0135
2018-05-02 06:00:56,863 23dbc788 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-02 06:00:57,260 23dbc788 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-02 06:00:57,414 23dbc788 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-02 06:00:57,418 23dbc788 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-02 06:00:57,418 23dbc788 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-02 06:01:02,501 23dbc788 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/23dbc788/beam/gradlew> clean install -xcheck --stacktrace -DintegrationTestRunner=dataflow
2018-05-02 06:05:34,626 23dbc788 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-02 06:05:34,635 23dbc788 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/23dbc788/beam/sdks/python/target>
2018-05-02 06:05:34,636 23dbc788 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-02 06:05:34,637 23dbc788 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 780, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/23dbc788/beam/sdks/python/target>
2018-05-02 06:05:34,638 23dbc788 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-02 06:05:34,696 23dbc788 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-02 06:05:34,697 23dbc788 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/23dbc788/pkb.log>
2018-05-02 06:05:34,697 23dbc788 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/23dbc788/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user shoyer@google.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user mairbek@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1217
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1217/display/redirect?page=changes>
Changes:
[swegner] Findbugs cleanup; no functional changes
[github] [BEAM-3042] Refactor of TransformIOCounters (performance, inheritance).
[kirpichov] [BEAM-4144] Fixes SplittableParDoProcessFnTest
------------------------------------------
[...truncated 4.45 KB...]
Using cached https://files.pythonhosted.org/packages/a2/71/8273a7eeed0aff6a854237ab5453bc9aa67deb49df4832801c21f0ff3782/contextlib2-0.5.5-py2.py3-none-any.whl
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5533562723208021089.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9f/ff/68860dec8b90424028ca54df8b9fc8cd1c47ef58710bd5e591ec8c619243/monotonic-1.4-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.4 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1949013470430979385.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-02 00:06:43,078 a99e8695 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/a99e8695/pkb.log>
2018-05-02 00:06:43,079 a99e8695 MainThread INFO PerfKitBenchmarker version: v1.12.0-536-g43c0135
2018-05-02 00:06:43,080 a99e8695 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-02 00:06:43,444 a99e8695 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-02 00:06:43,604 a99e8695 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-02 00:06:43,643 a99e8695 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-02 00:06:43,644 a99e8695 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-02 00:06:48,571 a99e8695 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/a99e8695/beam/gradlew> clean install -xcheck --stacktrace -DintegrationTestRunner=dataflow
2018-05-02 00:10:26,494 a99e8695 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-02 00:10:26,501 a99e8695 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/a99e8695/beam/sdks/python/target>
2018-05-02 00:10:26,502 a99e8695 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-02 00:10:26,504 a99e8695 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 780, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 190, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 267, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 295, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/a99e8695/beam/sdks/python/target>
2018-05-02 00:10:26,504 a99e8695 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-02 00:10:26,533 a99e8695 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-02 00:10:26,534 a99e8695 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/a99e8695/pkb.log>
2018-05-02 00:10:26,534 a99e8695 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/a99e8695/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user shoyer@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Build failed in Jenkins: beam_PerformanceTests_Python #1216
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1216/display/redirect?page=changes>
Changes:
[tgroh] Mark some ReduceFnRunner arguments Nullable
[Pablo] Add documentaton for quickstart tasks
------------------------------------------
[...truncated 4.42 KB...]
Using cached https://files.pythonhosted.org/packages/a2/71/8273a7eeed0aff6a854237ab5453bc9aa67deb49df4832801c21f0ff3782/contextlib2-0.5.5-py2.py3-none-any.whl
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3393256317851533590.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9f/ff/68860dec8b90424028ca54df8b9fc8cd1c47ef58710bd5e591ec8c619243/monotonic-1.4-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.4 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6332065632938212553.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-01 18:08:40,795 8aa06d8f MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/8aa06d8f/pkb.log>
2018-05-01 18:08:40,795 8aa06d8f MainThread INFO PerfKitBenchmarker version: v1.12.0-535-g5be59b8
2018-05-01 18:08:40,796 8aa06d8f MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-01 18:08:41,024 8aa06d8f MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-01 18:08:41,207 8aa06d8f MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-01 18:08:41,211 8aa06d8f MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-01 18:08:41,211 8aa06d8f MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-01 18:08:46,299 8aa06d8f MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-05-01 18:14:45,446 8aa06d8f MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-01 18:14:45,467 8aa06d8f MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/8aa06d8f/beam/sdks/python/target/apache-beam-2.5.0.dev0.tar.gz>
2018-05-01 18:14:45,701 8aa06d8f MainThread beam_integration_benchmark(1/1) INFO Ran: {python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/8aa06d8f/beam/sdks/python/target/apache-beam-2.5.0.dev0.tar.gz}> ReturnCode:1
STDOUT:
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:397: UserWarning: Normalizing '2.5.0.dev' to '2.5.0.dev0'
normalized_version,
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: invalid command 'nosetests'
2018-05-01 18:14:45,702 8aa06d8f MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-05-01 18:14:45,704 8aa06d8f MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-01 18:14:45,706 8aa06d8f MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 780, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-05-01 18:14:45,707 8aa06d8f MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-01 18:14:45,728 8aa06d8f MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-01 18:14:45,729 8aa06d8f MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/8aa06d8f/pkb.log>
2018-05-01 18:14:45,729 8aa06d8f MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/8aa06d8f/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user shoyer@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Build failed in Jenkins: beam_PerformanceTests_Python #1215
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1215/display/redirect>
------------------------------------------
[...truncated 4.44 KB...]
Using cached https://files.pythonhosted.org/packages/a2/71/8273a7eeed0aff6a854237ab5453bc9aa67deb49df4832801c21f0ff3782/contextlib2-0.5.5-py2.py3-none-any.whl
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4799797187760619386.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9f/ff/68860dec8b90424028ca54df8b9fc8cd1c47ef58710bd5e591ec8c619243/monotonic-1.4-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.4 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1493983922878122074.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-01 12:00:50,244 d6149ea5 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/d6149ea5/pkb.log>
2018-05-01 12:00:50,245 d6149ea5 MainThread INFO PerfKitBenchmarker version: v1.12.0-534-gb60e653
2018-05-01 12:00:50,246 d6149ea5 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-01 12:00:50,501 d6149ea5 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-01 12:00:50,659 d6149ea5 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-01 12:00:50,662 d6149ea5 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-01 12:00:50,663 d6149ea5 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-01 12:00:56,299 d6149ea5 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-05-01 12:06:57,039 d6149ea5 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-01 12:06:57,062 d6149ea5 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/d6149ea5/beam/sdks/python/target/apache-beam-2.5.0.dev0.tar.gz>
2018-05-01 12:06:57,315 d6149ea5 MainThread beam_integration_benchmark(1/1) INFO Ran: {python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/d6149ea5/beam/sdks/python/target/apache-beam-2.5.0.dev0.tar.gz}> ReturnCode:1
STDOUT:
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:397: UserWarning: Normalizing '2.5.0.dev' to '2.5.0.dev0'
normalized_version,
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: invalid command 'nosetests'
2018-05-01 12:06:57,316 d6149ea5 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-05-01 12:06:57,317 d6149ea5 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-01 12:06:57,318 d6149ea5 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 780, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-05-01 12:06:57,319 d6149ea5 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-01 12:06:57,368 d6149ea5 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-01 12:06:57,369 d6149ea5 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/d6149ea5/pkb.log>
2018-05-01 12:06:57,369 d6149ea5 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/d6149ea5/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user shoyer@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Build failed in Jenkins: beam_PerformanceTests_Python #1214
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1214/display/redirect?page=changes>
Changes:
[thw] [BEAM-4131] Include SDK into Python SDK harness container.
[thw] cleanup
------------------------------------------
[...truncated 4.44 KB...]
Using cached https://files.pythonhosted.org/packages/a2/71/8273a7eeed0aff6a854237ab5453bc9aa67deb49df4832801c21f0ff3782/contextlib2-0.5.5-py2.py3-none-any.whl
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7273163152504492280.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9f/ff/68860dec8b90424028ca54df8b9fc8cd1c47ef58710bd5e591ec8c619243/monotonic-1.4-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.4 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4175359635929884124.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-01 06:00:52,985 3b56a02d MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/3b56a02d/pkb.log>
2018-05-01 06:00:52,986 3b56a02d MainThread INFO PerfKitBenchmarker version: v1.12.0-534-gb60e653
2018-05-01 06:00:52,986 3b56a02d MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-01 06:00:53,341 3b56a02d MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-01 06:00:53,606 3b56a02d MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-01 06:00:53,611 3b56a02d MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-01 06:00:53,612 3b56a02d MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-01 06:00:59,644 3b56a02d MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-05-01 06:07:11,308 3b56a02d MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-01 06:07:11,325 3b56a02d MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/3b56a02d/beam/sdks/python/target/apache-beam-2.5.0.dev0.tar.gz>
2018-05-01 06:07:11,584 3b56a02d MainThread beam_integration_benchmark(1/1) INFO Ran: {python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/3b56a02d/beam/sdks/python/target/apache-beam-2.5.0.dev0.tar.gz}> ReturnCode:1
STDOUT:
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:397: UserWarning: Normalizing '2.5.0.dev' to '2.5.0.dev0'
normalized_version,
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: invalid command 'nosetests'
2018-05-01 06:07:11,585 3b56a02d MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-05-01 06:07:11,586 3b56a02d MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-01 06:07:11,587 3b56a02d MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 780, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-05-01 06:07:11,588 3b56a02d MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-01 06:07:11,629 3b56a02d MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-01 06:07:11,630 3b56a02d MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/3b56a02d/pkb.log>
2018-05-01 06:07:11,631 3b56a02d MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/3b56a02d/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user shoyer@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Build failed in Jenkins: beam_PerformanceTests_Python #1213
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1213/display/redirect?page=changes>
Changes:
[herohde] [BEAM-4175] Fix direct ouput pardo issue
[kenn] Add errorprone to basic Gradle config
[kenn] Fix suppressed failure in ResourceIdTester
[kenn] Fix erroneous tests in ViewTest
[kenn] Add missing @Test in CollectionCoderTest
[kenn] Sickbay broken LocalResourceIdTest case
[kenn] Remove inaccurate GuardedBy in both copies of DirectMetrics
[kenn] Fix suppressed failure in RetryHttpRequestInitializerTest
[kenn] Remove unnecessary type arguments in ReduceFnRunnerTest
[kenn] Add missing @Test in SplittableParDoProcessFnTest
[kenn] Remove unnecessary type argument in PipelineTranslationTest
[kenn] Fix compile-type constant int overflow in S3FileSystem
[kenn] Disable findbugs for JdbcIO where it gave false positives, now that we
[kenn] Sickbay broken GcsResourceIdTest case
[kenn] Sickbay broken HadoopResourceIdTest case
[kenn] Sickbay broken SplittableParDoProcessFnTest case
[kenn] Add missing @Test in S3ResourceIdTest
[kenn] Fix typo in CassandraIOIT
[kenn] Fix nonsensical @RunWith in ExampleEchoPipelineTest
[kenn] Fix missing @Test in both copies of WatermarkManagerTest
[kenn] Fix misuse of Arrays.asList in KinesisMockWriteTest
[kenn] Remove extraneous type variables in MongoDBGridFSIOTest
[kenn] Sickbay broken S3ResourceIdTest cases
[kenn] Sickbay broken WatermarkManagerTest case
[apilloud] [BEAM-3983][SQL] Add BigQuery table provider
[iemejia] Fix gradle script to build the docker image of 'sdks/java/container'
[Pablo] Creation of utils.py whit CountingSource class on it.
[Pablo] Changed import from examples.snippets to io.utils
------------------------------------------
[...truncated 4.43 KB...]
Using cached https://files.pythonhosted.org/packages/a2/71/8273a7eeed0aff6a854237ab5453bc9aa67deb49df4832801c21f0ff3782/contextlib2-0.5.5-py2.py3-none-any.whl
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2017567295102981354.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9f/ff/68860dec8b90424028ca54df8b9fc8cd1c47ef58710bd5e591ec8c619243/monotonic-1.4-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.4 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3564933175583089852.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-05-01 00:00:42,979 5e1e8535 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/5e1e8535/pkb.log>
2018-05-01 00:00:42,979 5e1e8535 MainThread INFO PerfKitBenchmarker version: v1.12.0-534-gb60e653
2018-05-01 00:00:42,980 5e1e8535 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-05-01 00:00:43,326 5e1e8535 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-05-01 00:00:43,479 5e1e8535 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-05-01 00:00:43,483 5e1e8535 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-05-01 00:00:43,483 5e1e8535 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-05-01 00:00:48,973 5e1e8535 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-05-01 00:07:01,540 5e1e8535 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-05-01 00:07:01,558 5e1e8535 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/5e1e8535/beam/sdks/python/target/apache-beam-2.5.0.dev0.tar.gz>
2018-05-01 00:07:01,812 5e1e8535 MainThread beam_integration_benchmark(1/1) INFO Ran: {python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/5e1e8535/beam/sdks/python/target/apache-beam-2.5.0.dev0.tar.gz}> ReturnCode:1
STDOUT:
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:397: UserWarning: Normalizing '2.5.0.dev' to '2.5.0.dev0'
normalized_version,
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: invalid command 'nosetests'
2018-05-01 00:07:01,813 5e1e8535 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-05-01 00:07:01,814 5e1e8535 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-05-01 00:07:01,816 5e1e8535 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 780, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-05-01 00:07:01,816 5e1e8535 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-05-01 00:07:01,874 5e1e8535 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-05-01 00:07:01,875 5e1e8535 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/5e1e8535/pkb.log>
2018-05-01 00:07:01,875 5e1e8535 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/5e1e8535/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user shoyer@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Build failed in Jenkins: beam_PerformanceTests_Python #1212
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1212/display/redirect?page=changes>
Changes:
[ankurgoenka] fixing generic errors
[echauchot] [BEAM-3119] ensure metrics thread is shutdown
[echauchot] [BEAM-4088] Rebase on master, fix conflicts and adapt test to master
[echauchot] [BEAM-4088] Add a test with a real pipeline that has a DoFn with metrics
[echauchot] [BEAM-4088] Blocks until the threads in metricsExecutorService is done,
[pgerver] Enhance awsCredentialsProvider option description
------------------------------------------
[...truncated 4.44 KB...]
Using cached https://files.pythonhosted.org/packages/a2/71/8273a7eeed0aff6a854237ab5453bc9aa67deb49df4832801c21f0ff3782/contextlib2-0.5.5-py2.py3-none-any.whl
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4920621408066048505.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9f/ff/68860dec8b90424028ca54df8b9fc8cd1c47ef58710bd5e591ec8c619243/monotonic-1.4-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.4 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins9167633024371039651.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-04-30 18:01:49,703 8cdf7846 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/8cdf7846/pkb.log>
2018-04-30 18:01:49,703 8cdf7846 MainThread INFO PerfKitBenchmarker version: v1.12.0-533-g60f6470
2018-04-30 18:01:49,704 8cdf7846 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-04-30 18:01:50,029 8cdf7846 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-04-30 18:01:50,196 8cdf7846 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-04-30 18:01:50,199 8cdf7846 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-04-30 18:01:50,200 8cdf7846 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-04-30 18:01:56,399 8cdf7846 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-04-30 18:08:33,304 8cdf7846 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-04-30 18:08:33,320 8cdf7846 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/8cdf7846/beam/sdks/python/target/apache-beam-2.5.0.dev0.tar.gz>
2018-04-30 18:08:33,548 8cdf7846 MainThread beam_integration_benchmark(1/1) INFO Ran: {python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/8cdf7846/beam/sdks/python/target/apache-beam-2.5.0.dev0.tar.gz}> ReturnCode:1
STDOUT:
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:397: UserWarning: Normalizing '2.5.0.dev' to '2.5.0.dev0'
normalized_version,
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: invalid command 'nosetests'
2018-04-30 18:08:33,549 8cdf7846 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-04-30 18:08:33,550 8cdf7846 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-04-30 18:08:33,552 8cdf7846 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 780, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-04-30 18:08:33,552 8cdf7846 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-04-30 18:08:33,591 8cdf7846 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-04-30 18:08:33,592 8cdf7846 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/8cdf7846/pkb.log>
2018-04-30 18:08:33,592 8cdf7846 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/8cdf7846/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user shoyer@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Build failed in Jenkins: beam_PerformanceTests_Python #1211
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1211/display/redirect?page=changes>
Changes:
[aljoscha.krettek] [BEAM-3909] Add tests for Flink DoFnOperator side-input checkpointing
------------------------------------------
[...truncated 4.32 KB...]
Collecting contextlib2>=0.5.1 (from -r PerfKitBenchmarker/requirements.txt (line 24))
Using cached https://files.pythonhosted.org/packages/a2/71/8273a7eeed0aff6a854237ab5453bc9aa67deb49df4832801c21f0ff3782/contextlib2-0.5.5-py2.py3-none-any.whl
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2645658287835764866.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9f/ff/68860dec8b90424028ca54df8b9fc8cd1c47ef58710bd5e591ec8c619243/monotonic-1.4-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.4 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1348428526960774711.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-04-30 12:01:36,892 176e9506 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/176e9506/pkb.log>
2018-04-30 12:01:36,893 176e9506 MainThread INFO PerfKitBenchmarker version: v1.12.0-533-g60f6470
2018-04-30 12:01:36,894 176e9506 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-04-30 12:01:37,253 176e9506 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-04-30 12:01:37,395 176e9506 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-04-30 12:01:37,398 176e9506 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-04-30 12:01:37,399 176e9506 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-04-30 12:01:42,327 176e9506 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-04-30 12:07:47,478 176e9506 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-04-30 12:07:47,557 176e9506 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/176e9506/beam/sdks/python/target/apache-beam-2.5.0.dev0.tar.gz>
2018-04-30 12:07:47,804 176e9506 MainThread beam_integration_benchmark(1/1) INFO Ran: {python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/176e9506/beam/sdks/python/target/apache-beam-2.5.0.dev0.tar.gz}> ReturnCode:1
STDOUT:
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:397: UserWarning: Normalizing '2.5.0.dev' to '2.5.0.dev0'
normalized_version,
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: invalid command 'nosetests'
2018-04-30 12:07:47,804 176e9506 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-04-30 12:07:47,805 176e9506 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-04-30 12:07:47,807 176e9506 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 780, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-04-30 12:07:47,807 176e9506 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-04-30 12:07:47,830 176e9506 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-04-30 12:07:47,830 176e9506 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/176e9506/pkb.log>
2018-04-30 12:07:47,830 176e9506 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/176e9506/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user shoyer@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Build failed in Jenkins: beam_PerformanceTests_Python #1210
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1210/display/redirect?page=changes>
Changes:
[chamikara] [BEAM-3973] Adds a parameter to the Cloud Spanner read connector that
------------------------------------------
[...truncated 4.33 KB...]
Collecting contextlib2>=0.5.1 (from -r PerfKitBenchmarker/requirements.txt (line 24))
Using cached https://files.pythonhosted.org/packages/a2/71/8273a7eeed0aff6a854237ab5453bc9aa67deb49df4832801c21f0ff3782/contextlib2-0.5.5-py2.py3-none-any.whl
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1598592255176044864.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9f/ff/68860dec8b90424028ca54df8b9fc8cd1c47ef58710bd5e591ec8c619243/monotonic-1.4-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.4 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6373523809561692719.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-04-30 06:00:38,958 f947e7a5 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/f947e7a5/pkb.log>
2018-04-30 06:00:38,959 f947e7a5 MainThread INFO PerfKitBenchmarker version: v1.12.0-533-g60f6470
2018-04-30 06:00:38,959 f947e7a5 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-04-30 06:00:39,272 f947e7a5 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-04-30 06:00:39,425 f947e7a5 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-04-30 06:00:39,428 f947e7a5 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-04-30 06:00:39,429 f947e7a5 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-04-30 06:00:44,339 f947e7a5 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-04-30 06:06:48,997 f947e7a5 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-04-30 06:06:49,014 f947e7a5 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/f947e7a5/beam/sdks/python/target/apache-beam-2.5.0.dev0.tar.gz>
2018-04-30 06:06:49,253 f947e7a5 MainThread beam_integration_benchmark(1/1) INFO Ran: {python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/f947e7a5/beam/sdks/python/target/apache-beam-2.5.0.dev0.tar.gz}> ReturnCode:1
STDOUT:
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:397: UserWarning: Normalizing '2.5.0.dev' to '2.5.0.dev0'
normalized_version,
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: invalid command 'nosetests'
2018-04-30 06:06:49,253 f947e7a5 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-04-30 06:06:49,254 f947e7a5 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-04-30 06:06:49,256 f947e7a5 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 780, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-04-30 06:06:49,256 f947e7a5 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-04-30 06:06:49,312 f947e7a5 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-04-30 06:06:49,312 f947e7a5 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/f947e7a5/pkb.log>
2018-04-30 06:06:49,312 f947e7a5 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/f947e7a5/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user shoyer@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Build failed in Jenkins: beam_PerformanceTests_Python #1209
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1209/display/redirect>
------------------------------------------
[...truncated 4.28 KB...]
Collecting contextlib2>=0.5.1 (from -r PerfKitBenchmarker/requirements.txt (line 24))
Using cached https://files.pythonhosted.org/packages/a2/71/8273a7eeed0aff6a854237ab5453bc9aa67deb49df4832801c21f0ff3782/contextlib2-0.5.5-py2.py3-none-any.whl
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8861034256233226150.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9f/ff/68860dec8b90424028ca54df8b9fc8cd1c47ef58710bd5e591ec8c619243/monotonic-1.4-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.4 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7321882618682103576.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-04-30 00:00:49,141 f919251a MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/f919251a/pkb.log>
2018-04-30 00:00:49,142 f919251a MainThread INFO PerfKitBenchmarker version: v1.12.0-533-g60f6470
2018-04-30 00:00:49,142 f919251a MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-04-30 00:00:49,310 f919251a MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-04-30 00:00:49,460 f919251a MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-04-30 00:00:49,464 f919251a MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-04-30 00:00:49,464 f919251a MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-04-30 00:00:55,132 f919251a MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-04-30 00:06:49,467 f919251a MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-04-30 00:06:49,492 f919251a MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/f919251a/beam/sdks/python/target/apache-beam-2.5.0.dev0.tar.gz>
2018-04-30 00:06:49,730 f919251a MainThread beam_integration_benchmark(1/1) INFO Ran: {python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/f919251a/beam/sdks/python/target/apache-beam-2.5.0.dev0.tar.gz}> ReturnCode:1
STDOUT:
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:397: UserWarning: Normalizing '2.5.0.dev' to '2.5.0.dev0'
normalized_version,
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: invalid command 'nosetests'
2018-04-30 00:06:49,731 f919251a MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-04-30 00:06:49,732 f919251a MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-04-30 00:06:49,734 f919251a MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 780, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-04-30 00:06:49,735 f919251a MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-04-30 00:06:49,753 f919251a MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-04-30 00:06:49,754 f919251a MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/f919251a/pkb.log>
2018-04-30 00:06:49,754 f919251a MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/f919251a/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user shoyer@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Build failed in Jenkins: beam_PerformanceTests_Python #1208
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1208/display/redirect>
------------------------------------------
[...truncated 4.28 KB...]
Collecting contextlib2>=0.5.1 (from -r PerfKitBenchmarker/requirements.txt (line 24))
Using cached https://files.pythonhosted.org/packages/a2/71/8273a7eeed0aff6a854237ab5453bc9aa67deb49df4832801c21f0ff3782/contextlib2-0.5.5-py2.py3-none-any.whl
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7138293126766063020.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9f/ff/68860dec8b90424028ca54df8b9fc8cd1c47ef58710bd5e591ec8c619243/monotonic-1.4-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.4 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins624080849028614251.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-04-29 18:00:35,491 aef9c1ec MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/aef9c1ec/pkb.log>
2018-04-29 18:00:35,491 aef9c1ec MainThread INFO PerfKitBenchmarker version: v1.12.0-533-g60f6470
2018-04-29 18:00:35,492 aef9c1ec MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-04-29 18:00:35,967 aef9c1ec MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-04-29 18:00:36,113 aef9c1ec MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-04-29 18:00:36,117 aef9c1ec MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-04-29 18:00:36,117 aef9c1ec MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-04-29 18:00:41,310 aef9c1ec MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-04-29 18:06:45,852 aef9c1ec MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-04-29 18:06:45,870 aef9c1ec MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/aef9c1ec/beam/sdks/python/target/apache-beam-2.5.0.dev0.tar.gz>
2018-04-29 18:06:46,099 aef9c1ec MainThread beam_integration_benchmark(1/1) INFO Ran: {python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/aef9c1ec/beam/sdks/python/target/apache-beam-2.5.0.dev0.tar.gz}> ReturnCode:1
STDOUT:
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:397: UserWarning: Normalizing '2.5.0.dev' to '2.5.0.dev0'
normalized_version,
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: invalid command 'nosetests'
2018-04-29 18:06:46,099 aef9c1ec MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-04-29 18:06:46,100 aef9c1ec MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-04-29 18:06:46,102 aef9c1ec MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 780, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-04-29 18:06:46,102 aef9c1ec MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-04-29 18:06:46,126 aef9c1ec MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-04-29 18:06:46,127 aef9c1ec MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/aef9c1ec/pkb.log>
2018-04-29 18:06:46,127 aef9c1ec MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/aef9c1ec/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user shoyer@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Build failed in Jenkins: beam_PerformanceTests_Python #1207
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1207/display/redirect>
------------------------------------------
[...truncated 72.94 KB...]
[INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (enforce-banned-dependencies) @ beam-sdks-go-container ---
[INFO]
[INFO] --- maven-dependency-plugin:3.0.2:unpack (copy-dependency) @ beam-sdks-go-container ---
[INFO] Configured Artifact: org.apache.beam:beam-sdks-go:pkg-sources:2.5.0-SNAPSHOT:zip
[INFO] Unpacking <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/7e877a07/beam/sdks/go/target/beam-sdks-go-2.5.0-SNAPSHOT-pkg-sources.zip> to <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/7e877a07/beam/sdks/go/container/target/src/github.com/apache/beam/sdks/go> with includes "" and excludes ""
[INFO]
[INFO] --- maven-resources-plugin:3.0.2:copy-resources (copy-go-pkg-source) @ beam-sdks-go-container ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/7e877a07/beam/sdks/go/container/pkg>
[INFO]
[INFO] --- maven-resources-plugin:3.0.2:copy-resources (copy-go-cmd-source) @ beam-sdks-go-container ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO]
[INFO] --- maven-remote-resources-plugin:1.5:process (process-resource-bundles) @ beam-sdks-go-container ---
[INFO]
[INFO] --- mvn-golang-wrapper:2.1.7:get (go-get-imports) @ beam-sdks-go-container ---
[INFO] Prepared command line : bin/go get -u google.golang.org/grpc golang.org/x/oauth2/google google.golang.org/api/storage/v1
[ERROR]
[ERROR] ---------Exec.Err---------
[ERROR] # cd /home/jenkins/.mvnGoLang/.go_path/src/github.com/golang/protobuf; git pull --ff-only
[ERROR] Your configuration specifies to merge with the ref 'refs/heads/master'
[ERROR] from the remote, but no such ref was fetched.
[ERROR] package github.com/golang/protobuf/proto: exit status 1
[ERROR]
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Beam :: Parent .............................. SUCCESS [ 2.472 s]
[INFO] Apache Beam :: SDKs :: Java :: Build Tools ......... SUCCESS [ 1.962 s]
[INFO] Apache Beam :: Model ............................... SUCCESS [ 0.029 s]
[INFO] Apache Beam :: Model :: Pipeline ................... SUCCESS [ 7.329 s]
[INFO] Apache Beam :: Model :: Job Management ............. SUCCESS [ 3.223 s]
[INFO] Apache Beam :: Model :: Fn Execution ............... SUCCESS [ 3.623 s]
[INFO] Apache Beam :: SDKs ................................ SUCCESS [ 0.105 s]
[INFO] Apache Beam :: SDKs :: Go .......................... SUCCESS [ 23.420 s]
[INFO] Apache Beam :: SDKs :: Go :: Container ............. FAILURE [ 14.522 s]
[INFO] Apache Beam :: SDKs :: Java ........................ SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Core ................ SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Fn Execution ........ SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions .......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Google Cloud Platform Core SKIPPED
[INFO] Apache Beam :: Runners ............................. SKIPPED
[INFO] Apache Beam :: Runners :: Core Construction Java ... SKIPPED
[INFO] Apache Beam :: Runners :: Core Java ................ SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Harness ............. SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Container ........... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO .................. SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Amazon Web Services SKIPPED
[INFO] Apache Beam :: Runners :: Local Java Core .......... SKIPPED
[INFO] Apache Beam :: Runners :: Direct Java .............. SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: AMQP .......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Common ........ SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Cassandra ..... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch . SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: Common SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: 2.x SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: 5.x SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: XML ........... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Protobuf SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Google Cloud Platform SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Dataflow .... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: File-based-io-tests SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop Common . SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop File System SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: JDBC .......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop Input Format SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: HBase ......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: HCatalog ...... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: JMS ........... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Kafka ......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Kinesis ....... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: MongoDB ....... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: MQTT .......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Redis ......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Solr .......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Tika .......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes .... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Starter SKIPPED
[INFO] Apache Beam :: Examples ............................ SKIPPED
[INFO] Apache Beam :: Examples :: Java .................... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Examples SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Jackson SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Join library SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Sketching SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Sorter SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: SQL ... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Nexmark ............. SKIPPED
[INFO] Apache Beam :: SDKs :: Python ...................... SKIPPED
[INFO] Apache Beam :: SDKs :: Python :: Container ......... SKIPPED
[INFO] Apache Beam :: Runners :: Java Fn Execution ........ SKIPPED
[INFO] Apache Beam :: Runners :: Java Local Artifact Service SKIPPED
[INFO] Apache Beam :: Runners :: Reference ................ SKIPPED
[INFO] Apache Beam :: Runners :: Reference :: Java ........ SKIPPED
[INFO] Apache Beam :: Runners :: Reference :: Job Orchestrator SKIPPED
[INFO] Apache Beam :: Runners :: Flink .................... SKIPPED
[INFO] Apache Beam :: Runners :: Gearpump ................. SKIPPED
[INFO] Apache Beam :: Runners :: Spark .................... SKIPPED
[INFO] Apache Beam :: Runners :: Apex ..................... SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform .... SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform :: GCE metadata provisioning SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform :: GCS artifact proxy SKIPPED
[INFO] Apache Beam :: Runners :: Extensions Java .......... SKIPPED
[INFO] Apache Beam :: Runners :: Extensions Java :: Metrics SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Aggregated Javadoc .. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 58.176 s
[INFO] Finished at: 2018-04-29T12:01:39Z
[INFO] Final Memory: 105M/3203M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal com.igormaznitsa:mvn-golang-wrapper:2.1.7:get (go-get-imports) on project beam-sdks-go-container: Process exit code : 1 -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :beam-sdks-go-container
STDERR:
2018-04-29 12:01:39,612 7e877a07 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-04-29 12:01:39,621 7e877a07 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 175, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 265, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 290, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/7e877a07/beam/sdks/python/target>
2018-04-29 12:01:39,622 7e877a07 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-04-29 12:01:39,624 7e877a07 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 780, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 175, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 265, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 290, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/7e877a07/beam/sdks/python/target>
2018-04-29 12:01:39,624 7e877a07 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-04-29 12:01:39,663 7e877a07 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-04-29 12:01:39,664 7e877a07 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/7e877a07/pkb.log>
2018-04-29 12:01:39,664 7e877a07 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/7e877a07/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user shoyer@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Build failed in Jenkins: beam_PerformanceTests_Python #1206
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1206/display/redirect>
------------------------------------------
[...truncated 4.28 KB...]
Collecting contextlib2>=0.5.1 (from -r PerfKitBenchmarker/requirements.txt (line 24))
Using cached https://files.pythonhosted.org/packages/a2/71/8273a7eeed0aff6a854237ab5453bc9aa67deb49df4832801c21f0ff3782/contextlib2-0.5.5-py2.py3-none-any.whl
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins285657690710648429.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9f/ff/68860dec8b90424028ca54df8b9fc8cd1c47ef58710bd5e591ec8c619243/monotonic-1.4-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.4 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins486216521547812524.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-04-29 06:00:33,577 0efc01c9 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0efc01c9/pkb.log>
2018-04-29 06:00:33,578 0efc01c9 MainThread INFO PerfKitBenchmarker version: v1.12.0-533-g60f6470
2018-04-29 06:00:33,578 0efc01c9 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-04-29 06:00:34,029 0efc01c9 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-04-29 06:00:34,184 0efc01c9 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-04-29 06:00:34,187 0efc01c9 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-04-29 06:00:34,188 0efc01c9 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-04-29 06:00:39,125 0efc01c9 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-04-29 06:06:43,380 0efc01c9 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-04-29 06:06:43,399 0efc01c9 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0efc01c9/beam/sdks/python/target/apache-beam-2.5.0.dev0.tar.gz>
2018-04-29 06:06:43,641 0efc01c9 MainThread beam_integration_benchmark(1/1) INFO Ran: {python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0efc01c9/beam/sdks/python/target/apache-beam-2.5.0.dev0.tar.gz}> ReturnCode:1
STDOUT:
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:397: UserWarning: Normalizing '2.5.0.dev' to '2.5.0.dev0'
normalized_version,
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: invalid command 'nosetests'
2018-04-29 06:06:43,641 0efc01c9 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-04-29 06:06:43,643 0efc01c9 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-04-29 06:06:43,644 0efc01c9 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 780, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-04-29 06:06:43,645 0efc01c9 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-04-29 06:06:43,681 0efc01c9 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-04-29 06:06:43,682 0efc01c9 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0efc01c9/pkb.log>
2018-04-29 06:06:43,683 0efc01c9 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0efc01c9/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user shoyer@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Build failed in Jenkins: beam_PerformanceTests_Python #1205
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1205/display/redirect>
------------------------------------------
[...truncated 160.06 KB...]
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (enforce) @ beam-sdks-java-container ---
[INFO] Adding ignore: module-info
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (enforce-banned-dependencies) @ beam-sdks-java-container ---
[INFO]
[INFO] --- maven-dependency-plugin:3.0.2:unpack (copy-dependency) @ beam-sdks-java-container ---
[INFO] Configured Artifact: org.apache.beam:beam-sdks-go:pkg-sources:2.5.0-SNAPSHOT:zip
[INFO] Unpacking <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/1042e0d1/beam/sdks/go/target/beam-sdks-go-2.5.0-SNAPSHOT-pkg-sources.zip> to <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/1042e0d1/beam/sdks/java/container/target/src/github.com/apache/beam/sdks/go> with includes "" and excludes ""
[INFO]
[INFO] --- maven-resources-plugin:3.0.2:copy-resources (copy-go-cmd-source) @ beam-sdks-java-container ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO]
[INFO] --- maven-remote-resources-plugin:1.5:process (process-resource-bundles) @ beam-sdks-java-container ---
[INFO]
[INFO] --- mvn-golang-wrapper:2.1.7:get (go-get-imports) @ beam-sdks-java-container ---
[INFO] Prepared command line : bin/go get -u google.golang.org/grpc golang.org/x/oauth2/google google.golang.org/api/storage/v1
[ERROR]
[ERROR] ---------Exec.Err---------
[ERROR] # cd /home/jenkins/.mvnGoLang/.go_path/src/cloud.google.com/go; git pull --ff-only
[ERROR] Your configuration specifies to merge with the ref 'refs/heads/master'
[ERROR] from the remote, but no such ref was fetched.
[ERROR] package cloud.google.com/go/compute/metadata: exit status 1
[ERROR]
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Beam :: Parent .............................. SUCCESS [ 3.019 s]
[INFO] Apache Beam :: SDKs :: Java :: Build Tools ......... SUCCESS [ 1.977 s]
[INFO] Apache Beam :: Model ............................... SUCCESS [ 0.029 s]
[INFO] Apache Beam :: Model :: Pipeline ................... SUCCESS [ 7.072 s]
[INFO] Apache Beam :: Model :: Job Management ............. SUCCESS [ 3.408 s]
[INFO] Apache Beam :: Model :: Fn Execution ............... SUCCESS [ 3.400 s]
[INFO] Apache Beam :: SDKs ................................ SUCCESS [ 0.103 s]
[INFO] Apache Beam :: SDKs :: Go .......................... SUCCESS [ 23.311 s]
[INFO] Apache Beam :: SDKs :: Go :: Container ............. SUCCESS [ 22.609 s]
[INFO] Apache Beam :: SDKs :: Java ........................ SUCCESS [ 0.086 s]
[INFO] Apache Beam :: SDKs :: Java :: Core ................ SUCCESS [ 24.093 s]
[INFO] Apache Beam :: SDKs :: Java :: Fn Execution ........ SUCCESS [ 3.168 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions .......... SUCCESS [ 0.045 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Google Cloud Platform Core SUCCESS [ 2.772 s]
[INFO] Apache Beam :: Runners ............................. SUCCESS [ 0.047 s]
[INFO] Apache Beam :: Runners :: Core Construction Java ... SUCCESS [ 4.002 s]
[INFO] Apache Beam :: Runners :: Core Java ................ SUCCESS [ 3.787 s]
[INFO] Apache Beam :: SDKs :: Java :: Harness ............. SUCCESS [ 7.719 s]
[INFO] Apache Beam :: SDKs :: Java :: Container ........... FAILURE [ 3.869 s]
[INFO] Apache Beam :: SDKs :: Java :: IO .................. SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Amazon Web Services SKIPPED
[INFO] Apache Beam :: Runners :: Local Java Core .......... SKIPPED
[INFO] Apache Beam :: Runners :: Direct Java .............. SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: AMQP .......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Common ........ SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Cassandra ..... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch . SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: Common SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: 2.x SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: 5.x SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: XML ........... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Protobuf SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Google Cloud Platform SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Dataflow .... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: File-based-io-tests SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop Common . SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop File System SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: JDBC .......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop Input Format SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: HBase ......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: HCatalog ...... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: JMS ........... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Kafka ......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Kinesis ....... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: MongoDB ....... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: MQTT .......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Redis ......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Solr .......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Tika .......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes .... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Starter SKIPPED
[INFO] Apache Beam :: Examples ............................ SKIPPED
[INFO] Apache Beam :: Examples :: Java .................... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Examples SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Jackson SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Join library SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Sketching SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Sorter SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: SQL ... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Nexmark ............. SKIPPED
[INFO] Apache Beam :: SDKs :: Python ...................... SKIPPED
[INFO] Apache Beam :: SDKs :: Python :: Container ......... SKIPPED
[INFO] Apache Beam :: Runners :: Java Fn Execution ........ SKIPPED
[INFO] Apache Beam :: Runners :: Java Local Artifact Service SKIPPED
[INFO] Apache Beam :: Runners :: Reference ................ SKIPPED
[INFO] Apache Beam :: Runners :: Reference :: Java ........ SKIPPED
[INFO] Apache Beam :: Runners :: Reference :: Job Orchestrator SKIPPED
[INFO] Apache Beam :: Runners :: Flink .................... SKIPPED
[INFO] Apache Beam :: Runners :: Gearpump ................. SKIPPED
[INFO] Apache Beam :: Runners :: Spark .................... SKIPPED
[INFO] Apache Beam :: Runners :: Apex ..................... SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform .... SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform :: GCE metadata provisioning SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform :: GCS artifact proxy SKIPPED
[INFO] Apache Beam :: Runners :: Extensions Java .......... SKIPPED
[INFO] Apache Beam :: Runners :: Extensions Java :: Metrics SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Aggregated Javadoc .. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 01:56 min
[INFO] Finished at: 2018-04-29T00:02:36Z
[INFO] Final Memory: 183M/6369M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal com.igormaznitsa:mvn-golang-wrapper:2.1.7:get (go-get-imports) on project beam-sdks-java-container: Process exit code : 1 -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :beam-sdks-java-container
STDERR:
2018-04-29 00:02:37,108 1042e0d1 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-04-29 00:02:37,117 1042e0d1 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 175, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 265, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 290, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/1042e0d1/beam/sdks/python/target>
2018-04-29 00:02:37,118 1042e0d1 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-04-29 00:02:37,119 1042e0d1 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 780, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 175, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 265, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 290, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/1042e0d1/beam/sdks/python/target>
2018-04-29 00:02:37,120 1042e0d1 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-04-29 00:02:37,132 1042e0d1 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-04-29 00:02:37,133 1042e0d1 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/1042e0d1/pkb.log>
2018-04-29 00:02:37,133 1042e0d1 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/1042e0d1/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user shoyer@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Build failed in Jenkins: beam_PerformanceTests_Python #1204
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1204/display/redirect>
------------------------------------------
[...truncated 4.28 KB...]
Collecting contextlib2>=0.5.1 (from -r PerfKitBenchmarker/requirements.txt (line 24))
Using cached https://files.pythonhosted.org/packages/a2/71/8273a7eeed0aff6a854237ab5453bc9aa67deb49df4832801c21f0ff3782/contextlib2-0.5.5-py2.py3-none-any.whl
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5948604961949767722.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9f/ff/68860dec8b90424028ca54df8b9fc8cd1c47ef58710bd5e591ec8c619243/monotonic-1.4-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.4 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6685805350601234296.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-04-28 18:01:26,430 9e536eb1 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/9e536eb1/pkb.log>
2018-04-28 18:01:26,431 9e536eb1 MainThread INFO PerfKitBenchmarker version: v1.12.0-533-g60f6470
2018-04-28 18:01:26,431 9e536eb1 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-04-28 18:01:26,801 9e536eb1 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-04-28 18:01:26,969 9e536eb1 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-04-28 18:01:26,973 9e536eb1 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-04-28 18:01:26,974 9e536eb1 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-04-28 18:01:32,036 9e536eb1 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-04-28 18:07:38,725 9e536eb1 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-04-28 18:07:38,743 9e536eb1 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/9e536eb1/beam/sdks/python/target/apache-beam-2.5.0.dev0.tar.gz>
2018-04-28 18:07:38,978 9e536eb1 MainThread beam_integration_benchmark(1/1) INFO Ran: {python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/9e536eb1/beam/sdks/python/target/apache-beam-2.5.0.dev0.tar.gz}> ReturnCode:1
STDOUT:
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:397: UserWarning: Normalizing '2.5.0.dev' to '2.5.0.dev0'
normalized_version,
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: invalid command 'nosetests'
2018-04-28 18:07:38,979 9e536eb1 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-04-28 18:07:38,980 9e536eb1 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-04-28 18:07:38,981 9e536eb1 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 780, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-04-28 18:07:38,982 9e536eb1 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-04-28 18:07:39,018 9e536eb1 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-04-28 18:07:39,019 9e536eb1 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/9e536eb1/pkb.log>
2018-04-28 18:07:39,019 9e536eb1 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/9e536eb1/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user shoyer@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Build failed in Jenkins: beam_PerformanceTests_Python #1203
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1203/display/redirect?page=changes>
Changes:
[matzew] It is OK per JMS_1.1 spec that JMSMessageId is actually null
------------------------------------------
[...truncated 4.28 KB...]
Collecting contextlib2>=0.5.1 (from -r PerfKitBenchmarker/requirements.txt (line 24))
Using cached https://files.pythonhosted.org/packages/a2/71/8273a7eeed0aff6a854237ab5453bc9aa67deb49df4832801c21f0ff3782/contextlib2-0.5.5-py2.py3-none-any.whl
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins753502001620833192.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.1.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9f/ff/68860dec8b90424028ca54df8b9fc8cd1c47ef58710bd5e591ec8c619243/monotonic-1.4-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.4 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4266955144491287279.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-04-28 12:02:05,158 e5439967 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e5439967/pkb.log>
2018-04-28 12:02:05,158 e5439967 MainThread INFO PerfKitBenchmarker version: v1.12.0-533-g60f6470
2018-04-28 12:02:05,159 e5439967 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-04-28 12:02:05,588 e5439967 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-04-28 12:02:05,815 e5439967 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-04-28 12:02:05,819 e5439967 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-04-28 12:02:05,820 e5439967 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-04-28 12:02:13,388 e5439967 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-04-28 12:09:11,391 e5439967 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-04-28 12:09:11,411 e5439967 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e5439967/beam/sdks/python/target/apache-beam-2.5.0.dev0.tar.gz>
2018-04-28 12:09:11,720 e5439967 MainThread beam_integration_benchmark(1/1) INFO Ran: {python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e5439967/beam/sdks/python/target/apache-beam-2.5.0.dev0.tar.gz}> ReturnCode:1
STDOUT:
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:397: UserWarning: Normalizing '2.5.0.dev' to '2.5.0.dev0'
normalized_version,
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: invalid command 'nosetests'
2018-04-28 12:09:11,721 e5439967 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-04-28 12:09:11,722 e5439967 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-04-28 12:09:11,739 e5439967 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 780, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-04-28 12:09:11,739 e5439967 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-04-28 12:09:11,767 e5439967 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-04-28 12:09:11,769 e5439967 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e5439967/pkb.log>
2018-04-28 12:09:11,769 e5439967 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e5439967/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user shoyer@google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Build failed in Jenkins: beam_PerformanceTests_Python #1202
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1202/display/redirect>
------------------------------------------
[...truncated 4.29 KB...]
Collecting contextlib2>=0.5.1 (from -r PerfKitBenchmarker/requirements.txt (line 24))
Using cached https://files.pythonhosted.org/packages/a2/71/8273a7eeed0aff6a854237ab5453bc9aa67deb49df4832801c21f0ff3782/contextlib2-0.5.5-py2.py3-none-any.whl
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5526531205315124375.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.0.1)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9f/ff/68860dec8b90424028ca54df8b9fc8cd1c47ef58710bd5e591ec8c619243/monotonic-1.4-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.4 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2464835549121233097.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-04-28 06:00:49,047 a647df5c MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/a647df5c/pkb.log>
2018-04-28 06:00:49,047 a647df5c MainThread INFO PerfKitBenchmarker version: v1.12.0-533-g60f6470
2018-04-28 06:00:49,048 a647df5c MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-04-28 06:00:49,283 a647df5c MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-04-28 06:00:49,460 a647df5c MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-04-28 06:00:49,464 a647df5c MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-04-28 06:00:49,464 a647df5c MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-04-28 06:00:55,117 a647df5c MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-04-28 06:07:38,832 a647df5c MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-04-28 06:07:38,863 a647df5c MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/a647df5c/beam/sdks/python/target/apache-beam-2.5.0.dev0.tar.gz>
2018-04-28 06:07:39,154 a647df5c MainThread beam_integration_benchmark(1/1) INFO Ran: {python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/a647df5c/beam/sdks/python/target/apache-beam-2.5.0.dev0.tar.gz}> ReturnCode:1
STDOUT:
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:397: UserWarning: Normalizing '2.5.0.dev' to '2.5.0.dev0'
normalized_version,
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: invalid command 'nosetests'
2018-04-28 06:07:39,155 a647df5c MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-04-28 06:07:39,156 a647df5c MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-04-28 06:07:39,159 a647df5c MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 780, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-04-28 06:07:39,160 a647df5c MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-04-28 06:07:39,205 a647df5c MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-04-28 06:07:39,206 a647df5c MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/a647df5c/pkb.log>
2018-04-28 06:07:39,206 a647df5c MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/a647df5c/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user shoyer@google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Build failed in Jenkins: beam_PerformanceTests_Python #1201
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1201/display/redirect?page=changes>
Changes:
[herohde] Rename Window to WindowingStrategy
[herohde] Add WindowInto and window representations
[herohde] Update EventTime to match Beam spec
[herohde] Simplify Combine code
[herohde] Support windowing in Go direct runner
[herohde] Support Go windowing on Dataflow
[herohde] CR: addressed reviewer comments
[coheigea] Remove some unnecessary autoboxing calls
[coheigea] Casting some calls to long to prevent loss of precision
[herohde] CR: simplify Dataflow encoding and comment fixes
[github] Bump Dataflow ValidatesRunner Timeout
[github] Update comment to reflect new timeout.
[kirpichov] Fixes mapping outputs in Dataflow streaming WriteFiles override
------------------------------------------
[...truncated 4.29 KB...]
Collecting contextlib2>=0.5.1 (from -r PerfKitBenchmarker/requirements.txt (line 24))
Using cached https://files.pythonhosted.org/packages/a2/71/8273a7eeed0aff6a854237ab5453bc9aa67deb49df4832801c21f0ff3782/contextlib2-0.5.5-py2.py3-none-any.whl
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5189339784178250188.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.0.1)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9f/ff/68860dec8b90424028ca54df8b9fc8cd1c47ef58710bd5e591ec8c619243/monotonic-1.4-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.4 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7561471626734465025.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-04-28 00:01:03,404 8b56126e MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/8b56126e/pkb.log>
2018-04-28 00:01:03,404 8b56126e MainThread INFO PerfKitBenchmarker version: v1.12.0-533-g60f6470
2018-04-28 00:01:03,406 8b56126e MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-04-28 00:01:03,768 8b56126e MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-04-28 00:01:04,054 8b56126e MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-04-28 00:01:04,058 8b56126e MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-04-28 00:01:04,059 8b56126e MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-04-28 00:01:12,127 8b56126e MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-04-28 00:08:25,557 8b56126e MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-04-28 00:08:25,586 8b56126e MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/8b56126e/beam/sdks/python/target/apache-beam-2.5.0.dev0.tar.gz>
2018-04-28 00:08:25,913 8b56126e MainThread beam_integration_benchmark(1/1) INFO Ran: {python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/8b56126e/beam/sdks/python/target/apache-beam-2.5.0.dev0.tar.gz}> ReturnCode:1
STDOUT:
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:397: UserWarning: Normalizing '2.5.0.dev' to '2.5.0.dev0'
normalized_version,
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: invalid command 'nosetests'
2018-04-28 00:08:25,914 8b56126e MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-04-28 00:08:25,916 8b56126e MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-04-28 00:08:25,918 8b56126e MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 780, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-04-28 00:08:25,919 8b56126e MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-04-28 00:08:25,955 8b56126e MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-04-28 00:08:25,956 8b56126e MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/8b56126e/pkb.log>
2018-04-28 00:08:25,957 8b56126e MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/8b56126e/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user shoyer@google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Build failed in Jenkins: beam_PerformanceTests_Python #1200
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1200/display/redirect?page=changes>
Changes:
[robertwb] Augment Python SDK worker to handle unwindowed coders.
[alex] [BEAM-3485] Fix split generation in CassandraIO
[katarzyna.kucharczyk] [BEAM-3997] Add HCatalogIOIT
------------------------------------------
[...truncated 4.47 KB...]
Using cached https://files.pythonhosted.org/packages/a2/71/8273a7eeed0aff6a854237ab5453bc9aa67deb49df4832801c21f0ff3782/contextlib2-0.5.5-py2.py3-none-any.whl
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3443737298246626167.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.0.1)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9f/ff/68860dec8b90424028ca54df8b9fc8cd1c47ef58710bd5e591ec8c619243/monotonic-1.4-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.4 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7574088253466885124.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-04-27 18:00:56,989 3853517c MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/3853517c/pkb.log>
2018-04-27 18:00:56,989 3853517c MainThread INFO PerfKitBenchmarker version: v1.12.0-521-g06cb008
2018-04-27 18:00:56,990 3853517c MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-04-27 18:00:57,377 3853517c MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-04-27 18:00:57,378 3853517c MainThread INFO Initializing the edw service decoder
2018-04-27 18:00:57,685 3853517c MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-04-27 18:00:57,691 3853517c MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-04-27 18:00:57,691 3853517c MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-04-27 18:01:04,466 3853517c MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-04-27 18:08:07,066 3853517c MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-04-27 18:08:07,085 3853517c MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/3853517c/beam/sdks/python/target/apache-beam-2.5.0.dev0.tar.gz>
2018-04-27 18:08:07,358 3853517c MainThread beam_integration_benchmark(1/1) INFO Ran: {python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/3853517c/beam/sdks/python/target/apache-beam-2.5.0.dev0.tar.gz}> ReturnCode:1
STDOUT:
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:397: UserWarning: Normalizing '2.5.0.dev' to '2.5.0.dev0'
normalized_version,
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: invalid command 'nosetests'
2018-04-27 18:08:07,359 3853517c MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-04-27 18:08:07,360 3853517c MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-04-27 18:08:07,363 3853517c MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 780, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-04-27 18:08:07,363 3853517c MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-04-27 18:08:07,415 3853517c MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-04-27 18:08:07,416 3853517c MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/3853517c/pkb.log>
2018-04-27 18:08:07,416 3853517c MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/3853517c/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user shoyer@google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Build failed in Jenkins: beam_PerformanceTests_Python #1199
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1199/display/redirect>
------------------------------------------
[...truncated 4.28 KB...]
Collecting contextlib2>=0.5.1 (from -r PerfKitBenchmarker/requirements.txt (line 24))
Using cached https://files.pythonhosted.org/packages/a2/71/8273a7eeed0aff6a854237ab5453bc9aa67deb49df4832801c21f0ff3782/contextlib2-0.5.5-py2.py3-none-any.whl
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8073374633193439424.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.0.1)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9f/ff/68860dec8b90424028ca54df8b9fc8cd1c47ef58710bd5e591ec8c619243/monotonic-1.4-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.4 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7830583081970257988.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-04-27 12:01:03,243 e1a8a9ba MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e1a8a9ba/pkb.log>
2018-04-27 12:01:03,243 e1a8a9ba MainThread INFO PerfKitBenchmarker version: v1.12.0-521-g06cb008
2018-04-27 12:01:03,245 e1a8a9ba MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-04-27 12:01:03,589 e1a8a9ba MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-04-27 12:01:03,590 e1a8a9ba MainThread INFO Initializing the edw service decoder
2018-04-27 12:01:03,854 e1a8a9ba MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-04-27 12:01:03,861 e1a8a9ba MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-04-27 12:01:03,862 e1a8a9ba MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-04-27 12:01:12,291 e1a8a9ba MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-04-27 12:08:29,073 e1a8a9ba MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-04-27 12:08:29,095 e1a8a9ba MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e1a8a9ba/beam/sdks/python/target/apache-beam-2.5.0.dev0.tar.gz>
2018-04-27 12:08:29,365 e1a8a9ba MainThread beam_integration_benchmark(1/1) INFO Ran: {python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e1a8a9ba/beam/sdks/python/target/apache-beam-2.5.0.dev0.tar.gz}> ReturnCode:1
STDOUT:
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:397: UserWarning: Normalizing '2.5.0.dev' to '2.5.0.dev0'
normalized_version,
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: invalid command 'nosetests'
2018-04-27 12:08:29,366 e1a8a9ba MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-04-27 12:08:29,367 e1a8a9ba MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-04-27 12:08:29,370 e1a8a9ba MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 780, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-04-27 12:08:29,370 e1a8a9ba MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-04-27 12:08:29,401 e1a8a9ba MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-04-27 12:08:29,402 e1a8a9ba MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e1a8a9ba/pkb.log>
2018-04-27 12:08:29,402 e1a8a9ba MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e1a8a9ba/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user shoyer@google.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Build failed in Jenkins: beam_PerformanceTests_Python #1198
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1198/display/redirect?page=changes>
Changes:
[Pablo] Transitioning MapTask objects to namecontext.
[Pablo] Removing NameContext creation from MapTaskExecutor.
------------------------------------------
[...truncated 4.28 KB...]
Collecting contextlib2>=0.5.1 (from -r PerfKitBenchmarker/requirements.txt (line 24))
Using cached https://files.pythonhosted.org/packages/a2/71/8273a7eeed0aff6a854237ab5453bc9aa67deb49df4832801c21f0ff3782/contextlib2-0.5.5-py2.py3-none-any.whl
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7065021449708473200.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.0.1)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9f/ff/68860dec8b90424028ca54df8b9fc8cd1c47ef58710bd5e591ec8c619243/monotonic-1.4-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.4 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins9200863054617720647.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-04-27 06:01:06,086 744a92f2 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/744a92f2/pkb.log>
2018-04-27 06:01:06,086 744a92f2 MainThread INFO PerfKitBenchmarker version: v1.12.0-521-g06cb008
2018-04-27 06:01:06,087 744a92f2 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-04-27 06:01:06,456 744a92f2 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-04-27 06:01:06,457 744a92f2 MainThread INFO Initializing the edw service decoder
2018-04-27 06:01:06,673 744a92f2 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-04-27 06:01:06,677 744a92f2 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-04-27 06:01:06,677 744a92f2 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-04-27 06:01:13,281 744a92f2 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-04-27 06:07:46,034 744a92f2 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-04-27 06:07:46,054 744a92f2 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/744a92f2/beam/sdks/python/target/apache-beam-2.5.0.dev0.tar.gz>
2018-04-27 06:07:46,344 744a92f2 MainThread beam_integration_benchmark(1/1) INFO Ran: {python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/744a92f2/beam/sdks/python/target/apache-beam-2.5.0.dev0.tar.gz}> ReturnCode:1
STDOUT:
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:397: UserWarning: Normalizing '2.5.0.dev' to '2.5.0.dev0'
normalized_version,
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: invalid command 'nosetests'
2018-04-27 06:07:46,345 744a92f2 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-04-27 06:07:46,347 744a92f2 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-04-27 06:07:46,349 744a92f2 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 780, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-04-27 06:07:46,350 744a92f2 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-04-27 06:07:46,364 744a92f2 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-04-27 06:07:46,365 744a92f2 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/744a92f2/pkb.log>
2018-04-27 06:07:46,365 744a92f2 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/744a92f2/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user shoyer@google.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Build failed in Jenkins: beam_PerformanceTests_Python #1197
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1197/display/redirect?page=changes>
Changes:
[rober] Move examples to godoc examples part 1.
------------------------------------------
[...truncated 4.29 KB...]
Collecting contextlib2>=0.5.1 (from -r PerfKitBenchmarker/requirements.txt (line 24))
Using cached https://files.pythonhosted.org/packages/a2/71/8273a7eeed0aff6a854237ab5453bc9aa67deb49df4832801c21f0ff3782/contextlib2-0.5.5-py2.py3-none-any.whl
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6465108651070753407.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.0.1)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9f/ff/68860dec8b90424028ca54df8b9fc8cd1c47ef58710bd5e591ec8c619243/monotonic-1.4-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.4 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8552309696080878985.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-04-27 00:00:41,242 06bb95be MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/06bb95be/pkb.log>
2018-04-27 00:00:41,242 06bb95be MainThread INFO PerfKitBenchmarker version: v1.12.0-521-g06cb008
2018-04-27 00:00:41,243 06bb95be MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-04-27 00:00:41,561 06bb95be MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-04-27 00:00:41,562 06bb95be MainThread INFO Initializing the edw service decoder
2018-04-27 00:00:41,748 06bb95be MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-04-27 00:00:41,751 06bb95be MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-04-27 00:00:41,752 06bb95be MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-04-27 00:00:47,267 06bb95be MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-04-27 00:22:37,549 06bb95be MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-04-27 00:22:37,572 06bb95be MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/06bb95be/beam/sdks/python/target/apache-beam-2.5.0.dev0.tar.gz>
2018-04-27 00:22:37,820 06bb95be MainThread beam_integration_benchmark(1/1) INFO Ran: {python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/06bb95be/beam/sdks/python/target/apache-beam-2.5.0.dev0.tar.gz}> ReturnCode:1
STDOUT:
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:397: UserWarning: Normalizing '2.5.0.dev' to '2.5.0.dev0'
normalized_version,
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: invalid command 'nosetests'
2018-04-27 00:22:37,821 06bb95be MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-04-27 00:22:37,823 06bb95be MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-04-27 00:22:37,825 06bb95be MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 780, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-04-27 00:22:37,825 06bb95be MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-04-27 00:22:37,879 06bb95be MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-04-27 00:22:37,880 06bb95be MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/06bb95be/pkb.log>
2018-04-27 00:22:37,880 06bb95be MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/06bb95be/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user shoyer@google.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Build failed in Jenkins: beam_PerformanceTests_Python #1196
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1196/display/redirect?page=changes>
Changes:
[coheigea] Simplify single-line lambdas
[coheigea] Removing duplicate dependency from nexmark pom
[tgroh] For the DirectRunner implementation into a "Portable" package
[coheigea] Upgrade Tika due to multiple CVEs
[tgroh] Clean up Portable Direct Runner package
[thw] [BEAM-4071] fix Maven build
[lcwik] Removing some unused local variables (#4684)
------------------------------------------
[...truncated 4.27 KB...]
Collecting contextlib2>=0.5.1 (from -r PerfKitBenchmarker/requirements.txt (line 24))
Using cached https://files.pythonhosted.org/packages/a2/71/8273a7eeed0aff6a854237ab5453bc9aa67deb49df4832801c21f0ff3782/contextlib2-0.5.5-py2.py3-none-any.whl
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5209153894782645156.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.0.1)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9f/ff/68860dec8b90424028ca54df8b9fc8cd1c47ef58710bd5e591ec8c619243/monotonic-1.4-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.4 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6661445516490448989.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-04-26 18:00:45,663 2a56a636 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/2a56a636/pkb.log>
2018-04-26 18:00:45,664 2a56a636 MainThread INFO PerfKitBenchmarker version: v1.12.0-514-g92e9b3f
2018-04-26 18:00:45,665 2a56a636 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-04-26 18:00:46,023 2a56a636 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-04-26 18:00:46,023 2a56a636 MainThread INFO Initializing the edw service decoder
2018-04-26 18:00:46,220 2a56a636 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-04-26 18:00:46,223 2a56a636 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-04-26 18:00:46,224 2a56a636 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-04-26 18:00:52,077 2a56a636 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-04-26 18:23:01,831 2a56a636 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-04-26 18:23:01,858 2a56a636 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/2a56a636/beam/sdks/python/target/apache-beam-2.5.0.dev0.tar.gz>
2018-04-26 18:23:02,125 2a56a636 MainThread beam_integration_benchmark(1/1) INFO Ran: {python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/2a56a636/beam/sdks/python/target/apache-beam-2.5.0.dev0.tar.gz}> ReturnCode:1
STDOUT:
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:397: UserWarning: Normalizing '2.5.0.dev' to '2.5.0.dev0'
normalized_version,
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: invalid command 'nosetests'
2018-04-26 18:23:02,126 2a56a636 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-04-26 18:23:02,127 2a56a636 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-04-26 18:23:02,129 2a56a636 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 780, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-04-26 18:23:02,130 2a56a636 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-04-26 18:23:02,171 2a56a636 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-04-26 18:23:02,172 2a56a636 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/2a56a636/pkb.log>
2018-04-26 18:23:02,172 2a56a636 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/2a56a636/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user shoyer@google.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Build failed in Jenkins: beam_PerformanceTests_Python #1195
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1195/display/redirect?page=changes>
Changes:
[coheigea] Use Collections.emptyList() instead of Collections.EMPTY_LIST
------------------------------------------
[...truncated 816.84 KB...]
[INFO] Apache Beam :: Model ............................... SUCCESS [ 0.033 s]
[INFO] Apache Beam :: Model :: Pipeline ................... SUCCESS [ 8.215 s]
[INFO] Apache Beam :: Model :: Job Management ............. SUCCESS [ 3.514 s]
[INFO] Apache Beam :: Model :: Fn Execution ............... SUCCESS [ 3.758 s]
[INFO] Apache Beam :: SDKs ................................ SUCCESS [ 0.116 s]
[INFO] Apache Beam :: SDKs :: Go .......................... SUCCESS [ 25.209 s]
[INFO] Apache Beam :: SDKs :: Go :: Container ............. SUCCESS [ 17.459 s]
[INFO] Apache Beam :: SDKs :: Java ........................ SUCCESS [ 0.117 s]
[INFO] Apache Beam :: SDKs :: Java :: Core ................ SUCCESS [ 27.648 s]
[INFO] Apache Beam :: SDKs :: Java :: Fn Execution ........ SUCCESS [ 3.166 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions .......... SUCCESS [ 0.062 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Google Cloud Platform Core SUCCESS [ 3.381 s]
[INFO] Apache Beam :: Runners ............................. SUCCESS [ 0.062 s]
[INFO] Apache Beam :: Runners :: Core Construction Java ... SUCCESS [ 5.013 s]
[INFO] Apache Beam :: Runners :: Core Java ................ SUCCESS [ 4.613 s]
[INFO] Apache Beam :: SDKs :: Java :: Harness ............. SUCCESS [ 8.462 s]
[INFO] Apache Beam :: SDKs :: Java :: Container ........... SUCCESS [ 7.218 s]
[INFO] Apache Beam :: SDKs :: Java :: IO .................. SUCCESS [ 0.049 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Amazon Web Services SUCCESS [ 2.880 s]
[INFO] Apache Beam :: Runners :: Local Java Core .......... SUCCESS [ 1.530 s]
[INFO] Apache Beam :: Runners :: Direct Java .............. SUCCESS [ 10.684 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: AMQP .......... SUCCESS [ 2.303 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Common ........ SUCCESS [ 2.074 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Cassandra ..... SUCCESS [ 2.490 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch . SUCCESS [ 1.958 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests SUCCESS [ 0.768 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: Common SUCCESS [ 0.614 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: 2.x SUCCESS [ 1.492 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: 5.x SUCCESS [ 1.944 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: XML ........... SUCCESS [ 2.399 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Protobuf SUCCESS [ 2.906 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Google Cloud Platform SUCCESS [ 4.825 s]
[INFO] Apache Beam :: Runners :: Google Cloud Dataflow .... SUCCESS [ 8.989 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: File-based-io-tests SUCCESS [ 1.989 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop Common . SUCCESS [ 3.025 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop File System SUCCESS [ 4.074 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: JDBC .......... SUCCESS [ 2.981 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop Input Format SUCCESS [ 9.254 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: HBase ......... SUCCESS [ 4.952 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: HCatalog ...... SUCCESS [ 8.581 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: JMS ........... SUCCESS [ 2.176 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Kafka ......... SUCCESS [ 3.361 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Kinesis ....... SUCCESS [ 1.558 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: MongoDB ....... SUCCESS [ 3.078 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: MQTT .......... SUCCESS [ 2.226 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Redis ......... SUCCESS [ 2.525 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Solr .......... SUCCESS [ 4.586 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Tika .......... SUCCESS [ 3.235 s]
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes .... SUCCESS [ 0.022 s]
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Starter SUCCESS [ 8.027 s]
[INFO] Apache Beam :: Examples ............................ SUCCESS [ 0.028 s]
[INFO] Apache Beam :: Examples :: Java .................... SUCCESS [ 3.525 s]
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Examples SUCCESS [ 27.572 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Jackson SUCCESS [ 1.708 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Join library SUCCESS [ 2.183 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Sketching SUCCESS [ 2.409 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Sorter SUCCESS [ 2.507 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: SQL ... SUCCESS [ 13.888 s]
[INFO] Apache Beam :: SDKs :: Java :: Nexmark ............. SUCCESS [ 15.878 s]
[INFO] Apache Beam :: SDKs :: Python ...................... SUCCESS [ 12.000 s]
[INFO] Apache Beam :: SDKs :: Python :: Container ......... SUCCESS [ 8.024 s]
[INFO] Apache Beam :: Runners :: Java Fn Execution ........ SUCCESS [ 3.257 s]
[INFO] Apache Beam :: Runners :: Java Local Artifact Service SUCCESS [ 2.032 s]
[INFO] Apache Beam :: Runners :: Reference ................ SUCCESS [ 0.025 s]
[INFO] Apache Beam :: Runners :: Reference :: Java ........ FAILURE [ 0.569 s]
[INFO] Apache Beam :: Runners :: Reference :: Job Orchestrator SKIPPED
[INFO] Apache Beam :: Runners :: Flink .................... SKIPPED
[INFO] Apache Beam :: Runners :: Gearpump ................. SKIPPED
[INFO] Apache Beam :: Runners :: Spark .................... SKIPPED
[INFO] Apache Beam :: Runners :: Apex ..................... SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform .... SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform :: GCE metadata provisioning SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform :: GCS artifact proxy SKIPPED
[INFO] Apache Beam :: Runners :: Extensions Java .......... SKIPPED
[INFO] Apache Beam :: Runners :: Extensions Java :: Metrics SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Aggregated Javadoc .. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 05:34 min
[INFO] Finished at: 2018-04-26T12:06:31Z
[INFO] Final Memory: 883M/9770M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.7.0:compile (default-compile) on project beam-runners-reference-java: Compilation failure: Compilation failure:
[ERROR] <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/3058de06/beam/runners/reference/java/src/main/java/org/apache/beam/runners/reference/testing/InProcessManagedChannelFactory.java>:[23,38] package org.apache.beam.sdk.fn.channel does not exist
[ERROR] <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/3058de06/beam/runners/reference/java/src/main/java/org/apache/beam/runners/reference/testing/InProcessManagedChannelFactory.java>:[30,53] cannot find symbol
[ERROR] symbol: class ManagedChannelFactory
[ERROR] <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/3058de06/beam/runners/reference/java/src/main/java/org/apache/beam/runners/reference/PortableRunner.java>:[49,38] package org.apache.beam.sdk.fn.channel does not exist
[ERROR] <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/3058de06/beam/runners/reference/java/src/main/java/org/apache/beam/runners/reference/PortableRunner.java>:[69,17] cannot find symbol
[ERROR] symbol: class ManagedChannelFactory
[ERROR] location: class org.apache.beam.runners.reference.PortableRunner
[ERROR] <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/3058de06/beam/runners/reference/java/src/main/java/org/apache/beam/runners/reference/PortableRunner.java>:[82,57] cannot find symbol
[ERROR] symbol: class ManagedChannelFactory
[ERROR] location: class org.apache.beam.runners.reference.PortableRunner
[ERROR] <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/3058de06/beam/runners/reference/java/src/main/java/org/apache/beam/runners/reference/PortableRunner.java>:[128,7] cannot find symbol
[ERROR] symbol: class ManagedChannelFactory
[ERROR] location: class org.apache.beam.runners.reference.PortableRunner
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :beam-runners-reference-java
STDERR: log4j:WARN No appenders could be found for logger (freemarker.cache).
log4j:WARN Please initialize the log4j system properly.
Warning: Lookahead adequacy checking not being performed since option LOOKAHEAD is more than 1. Set option FORCE_LA_CHECK to true to force checking.
/usr/local/lib/python2.7/dist-packages/setuptools/dist.py:397: UserWarning: Normalizing '2.5.0.dev' to '2.5.0.dev0'
normalized_version,
/usr/local/lib/python2.7/dist-packages/setuptools/dist.py:397: UserWarning: Normalizing '2.5.0.dev' to '2.5.0.dev0'
normalized_version,
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/3058de06/beam/sdks/python/gen_protos.py>:49: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
WARNING:root:Installing grpcio-tools into <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/3058de06/beam/sdks/python/.eggs/grpcio-wheels>
WARNING:root:Installing grpcio-tools took 3.64 seconds.
The script virtualenv is installed in '<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/3058de06/beam/sdks/python/target/python/bin'> which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
The scripts tox and tox-quickstart are installed in '<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/3058de06/beam/sdks/python/target/python/bin'> which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/3058de06/beam/sdks/python/target/python/lib/python2.7/site-packages/setuptools/dist.py>:397: UserWarning: Normalizing '2.5.0.dev' to '2.5.0.dev0'
normalized_version,
2018-04-26 12:06:33,035 3058de06 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-04-26 12:06:33,057 3058de06 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/3058de06/beam/sdks/python/target/apache-beam-2.5.0.dev0.tar.gz>
2018-04-26 12:06:33,342 3058de06 MainThread beam_integration_benchmark(1/1) INFO Ran: {python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/3058de06/beam/sdks/python/target/apache-beam-2.5.0.dev0.tar.gz}> ReturnCode:1
STDOUT:
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:397: UserWarning: Normalizing '2.5.0.dev' to '2.5.0.dev0'
normalized_version,
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: invalid command 'nosetests'
2018-04-26 12:06:33,342 3058de06 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-04-26 12:06:33,343 3058de06 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-04-26 12:06:33,345 3058de06 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 780, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-04-26 12:06:33,346 3058de06 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-04-26 12:06:33,382 3058de06 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-04-26 12:06:33,383 3058de06 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/3058de06/pkb.log>
2018-04-26 12:06:33,383 3058de06 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/3058de06/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user shoyer@google.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Build failed in Jenkins: beam_PerformanceTests_Python #1194
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1194/display/redirect?page=changes>
Changes:
[tgroh] Add OutputDeduplicator
[tgroh] Update ExecutableStageMatcher to accept Matchers
[tgroh] Use OutputDeduplicator in GreedyPipelineFuser
[swegner] Upgrade Gradle to 4.7
[swegner] Upgrade Gradle protobuf plugin to latest.
[tgroh] fixup! Add OutputDeduplicator
------------------------------------------
[...truncated 816.79 KB...]
[INFO] Apache Beam :: Model ............................... SUCCESS [ 0.035 s]
[INFO] Apache Beam :: Model :: Pipeline ................... SUCCESS [ 8.533 s]
[INFO] Apache Beam :: Model :: Job Management ............. SUCCESS [ 3.907 s]
[INFO] Apache Beam :: Model :: Fn Execution ............... SUCCESS [ 3.906 s]
[INFO] Apache Beam :: SDKs ................................ SUCCESS [ 0.132 s]
[INFO] Apache Beam :: SDKs :: Go .......................... SUCCESS [ 31.284 s]
[INFO] Apache Beam :: SDKs :: Go :: Container ............. SUCCESS [ 22.075 s]
[INFO] Apache Beam :: SDKs :: Java ........................ SUCCESS [ 0.154 s]
[INFO] Apache Beam :: SDKs :: Java :: Core ................ SUCCESS [ 26.405 s]
[INFO] Apache Beam :: SDKs :: Java :: Fn Execution ........ SUCCESS [ 3.083 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions .......... SUCCESS [ 0.050 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Google Cloud Platform Core SUCCESS [ 3.428 s]
[INFO] Apache Beam :: Runners ............................. SUCCESS [ 0.060 s]
[INFO] Apache Beam :: Runners :: Core Construction Java ... SUCCESS [ 4.812 s]
[INFO] Apache Beam :: Runners :: Core Java ................ SUCCESS [ 4.796 s]
[INFO] Apache Beam :: SDKs :: Java :: Harness ............. SUCCESS [ 8.793 s]
[INFO] Apache Beam :: SDKs :: Java :: Container ........... SUCCESS [ 8.932 s]
[INFO] Apache Beam :: SDKs :: Java :: IO .................. SUCCESS [ 0.054 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Amazon Web Services SUCCESS [ 2.953 s]
[INFO] Apache Beam :: Runners :: Local Java Core .......... SUCCESS [ 1.606 s]
[INFO] Apache Beam :: Runners :: Direct Java .............. SUCCESS [ 11.405 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: AMQP .......... SUCCESS [ 2.618 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Common ........ SUCCESS [ 1.865 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Cassandra ..... SUCCESS [ 2.769 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch . SUCCESS [ 1.959 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests SUCCESS [ 0.553 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: Common SUCCESS [ 0.638 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: 2.x SUCCESS [ 1.601 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: 5.x SUCCESS [ 1.990 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: XML ........... SUCCESS [ 4.223 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Protobuf SUCCESS [ 2.117 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Google Cloud Platform SUCCESS [ 4.934 s]
[INFO] Apache Beam :: Runners :: Google Cloud Dataflow .... SUCCESS [ 11.940 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: File-based-io-tests SUCCESS [ 2.123 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop Common . SUCCESS [ 3.187 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop File System SUCCESS [ 3.302 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: JDBC .......... SUCCESS [ 4.767 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop Input Format SUCCESS [ 8.049 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: HBase ......... SUCCESS [ 6.612 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: HCatalog ...... SUCCESS [ 7.833 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: JMS ........... SUCCESS [ 3.272 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Kafka ......... SUCCESS [ 2.675 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Kinesis ....... SUCCESS [ 1.614 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: MongoDB ....... SUCCESS [ 3.357 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: MQTT .......... SUCCESS [ 2.192 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Redis ......... SUCCESS [ 2.052 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Solr .......... SUCCESS [ 4.998 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Tika .......... SUCCESS [ 3.233 s]
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes .... SUCCESS [ 0.028 s]
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Starter SUCCESS [ 6.954 s]
[INFO] Apache Beam :: Examples ............................ SUCCESS [ 0.028 s]
[INFO] Apache Beam :: Examples :: Java .................... SUCCESS [ 3.841 s]
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Examples SUCCESS [ 28.156 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Jackson SUCCESS [ 1.933 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Join library SUCCESS [ 2.213 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Sketching SUCCESS [ 2.786 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Sorter SUCCESS [ 2.653 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: SQL ... SUCCESS [ 14.327 s]
[INFO] Apache Beam :: SDKs :: Java :: Nexmark ............. SUCCESS [ 17.786 s]
[INFO] Apache Beam :: SDKs :: Python ...................... SUCCESS [ 12.597 s]
[INFO] Apache Beam :: SDKs :: Python :: Container ......... SUCCESS [ 7.331 s]
[INFO] Apache Beam :: Runners :: Java Fn Execution ........ SUCCESS [ 3.937 s]
[INFO] Apache Beam :: Runners :: Java Local Artifact Service SUCCESS [ 2.059 s]
[INFO] Apache Beam :: Runners :: Reference ................ SUCCESS [ 0.022 s]
[INFO] Apache Beam :: Runners :: Reference :: Java ........ FAILURE [ 0.551 s]
[INFO] Apache Beam :: Runners :: Reference :: Job Orchestrator SKIPPED
[INFO] Apache Beam :: Runners :: Flink .................... SKIPPED
[INFO] Apache Beam :: Runners :: Gearpump ................. SKIPPED
[INFO] Apache Beam :: Runners :: Spark .................... SKIPPED
[INFO] Apache Beam :: Runners :: Apex ..................... SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform .... SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform :: GCE metadata provisioning SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform :: GCS artifact proxy SKIPPED
[INFO] Apache Beam :: Runners :: Extensions Java .......... SKIPPED
[INFO] Apache Beam :: Runners :: Extensions Java :: Metrics SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Aggregated Javadoc .. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 05:57 min
[INFO] Finished at: 2018-04-26T06:07:40Z
[INFO] Final Memory: 644M/9607M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.7.0:compile (default-compile) on project beam-runners-reference-java: Compilation failure: Compilation failure:
[ERROR] <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e6f070a9/beam/runners/reference/java/src/main/java/org/apache/beam/runners/reference/testing/InProcessManagedChannelFactory.java>:[23,38] package org.apache.beam.sdk.fn.channel does not exist
[ERROR] <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e6f070a9/beam/runners/reference/java/src/main/java/org/apache/beam/runners/reference/testing/InProcessManagedChannelFactory.java>:[30,53] cannot find symbol
[ERROR] symbol: class ManagedChannelFactory
[ERROR] <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e6f070a9/beam/runners/reference/java/src/main/java/org/apache/beam/runners/reference/PortableRunner.java>:[49,38] package org.apache.beam.sdk.fn.channel does not exist
[ERROR] <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e6f070a9/beam/runners/reference/java/src/main/java/org/apache/beam/runners/reference/PortableRunner.java>:[69,17] cannot find symbol
[ERROR] symbol: class ManagedChannelFactory
[ERROR] location: class org.apache.beam.runners.reference.PortableRunner
[ERROR] <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e6f070a9/beam/runners/reference/java/src/main/java/org/apache/beam/runners/reference/PortableRunner.java>:[82,57] cannot find symbol
[ERROR] symbol: class ManagedChannelFactory
[ERROR] location: class org.apache.beam.runners.reference.PortableRunner
[ERROR] <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e6f070a9/beam/runners/reference/java/src/main/java/org/apache/beam/runners/reference/PortableRunner.java>:[128,7] cannot find symbol
[ERROR] symbol: class ManagedChannelFactory
[ERROR] location: class org.apache.beam.runners.reference.PortableRunner
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :beam-runners-reference-java
STDERR: log4j:WARN No appenders could be found for logger (freemarker.cache).
log4j:WARN Please initialize the log4j system properly.
Warning: Lookahead adequacy checking not being performed since option LOOKAHEAD is more than 1. Set option FORCE_LA_CHECK to true to force checking.
/usr/local/lib/python2.7/dist-packages/setuptools/dist.py:397: UserWarning: Normalizing '2.5.0.dev' to '2.5.0.dev0'
normalized_version,
/usr/local/lib/python2.7/dist-packages/setuptools/dist.py:397: UserWarning: Normalizing '2.5.0.dev' to '2.5.0.dev0'
normalized_version,
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e6f070a9/beam/sdks/python/gen_protos.py>:49: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
WARNING:root:Installing grpcio-tools into <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e6f070a9/beam/sdks/python/.eggs/grpcio-wheels>
WARNING:root:Installing grpcio-tools took 3.74 seconds.
The script virtualenv is installed in '<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e6f070a9/beam/sdks/python/target/python/bin'> which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
The scripts tox and tox-quickstart are installed in '<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e6f070a9/beam/sdks/python/target/python/bin'> which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e6f070a9/beam/sdks/python/target/python/lib/python2.7/site-packages/setuptools/dist.py>:397: UserWarning: Normalizing '2.5.0.dev' to '2.5.0.dev0'
normalized_version,
2018-04-26 06:07:42,374 e6f070a9 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-04-26 06:07:42,396 e6f070a9 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e6f070a9/beam/sdks/python/target/apache-beam-2.5.0.dev0.tar.gz>
2018-04-26 06:07:42,666 e6f070a9 MainThread beam_integration_benchmark(1/1) INFO Ran: {python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e6f070a9/beam/sdks/python/target/apache-beam-2.5.0.dev0.tar.gz}> ReturnCode:1
STDOUT:
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:397: UserWarning: Normalizing '2.5.0.dev' to '2.5.0.dev0'
normalized_version,
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: invalid command 'nosetests'
2018-04-26 06:07:42,667 e6f070a9 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-04-26 06:07:42,668 e6f070a9 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-04-26 06:07:42,671 e6f070a9 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 780, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-04-26 06:07:42,672 e6f070a9 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-04-26 06:07:42,725 e6f070a9 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-04-26 06:07:42,726 e6f070a9 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e6f070a9/pkb.log>
2018-04-26 06:07:42,726 e6f070a9 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e6f070a9/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user boyuanz@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user shoyer@google.com
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Build failed in Jenkins: beam_PerformanceTests_Python #1193
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1193/display/redirect?page=changes>
Changes:
[sidhom] Add in-process ManagedChannelFactory and update tests to use it
[sidhom] Add common pipeline options for portable runners
[sidhom] Set artifact names explicitly while staging to service
[sidhom] Add CloseableResource to wrap non-closeable resources that must be
[sidhom] [BEAM-4071] Add Portable Runner Job API shim
[aaltay] [BEAM-3255] Enabling gradle-based release process (#5187)
[ekirpichov] Moves standard URNs into .proto files
[ekirpichov] Refactors common_urns.py
[ekirpichov] Fixes pylint errors
[ekirpichov] Fixes compile errors
[ekirpichov] Workaround for protobuf compiler bug
[ekirpichov] Fixes more references to common_urns
------------------------------------------
[...truncated 816.84 KB...]
[INFO] Apache Beam :: Model ............................... SUCCESS [ 0.029 s]
[INFO] Apache Beam :: Model :: Pipeline ................... SUCCESS [ 5.413 s]
[INFO] Apache Beam :: Model :: Job Management ............. SUCCESS [ 2.890 s]
[INFO] Apache Beam :: Model :: Fn Execution ............... SUCCESS [ 3.534 s]
[INFO] Apache Beam :: SDKs ................................ SUCCESS [ 0.145 s]
[INFO] Apache Beam :: SDKs :: Go .......................... SUCCESS [ 22.851 s]
[INFO] Apache Beam :: SDKs :: Go :: Container ............. SUCCESS [ 19.565 s]
[INFO] Apache Beam :: SDKs :: Java ........................ SUCCESS [ 0.133 s]
[INFO] Apache Beam :: SDKs :: Java :: Core ................ SUCCESS [ 21.859 s]
[INFO] Apache Beam :: SDKs :: Java :: Fn Execution ........ SUCCESS [ 2.874 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions .......... SUCCESS [ 0.047 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Google Cloud Platform Core SUCCESS [ 3.290 s]
[INFO] Apache Beam :: Runners ............................. SUCCESS [ 0.061 s]
[INFO] Apache Beam :: Runners :: Core Construction Java ... SUCCESS [ 4.354 s]
[INFO] Apache Beam :: Runners :: Core Java ................ SUCCESS [ 4.227 s]
[INFO] Apache Beam :: SDKs :: Java :: Harness ............. SUCCESS [ 8.126 s]
[INFO] Apache Beam :: SDKs :: Java :: Container ........... SUCCESS [ 7.438 s]
[INFO] Apache Beam :: SDKs :: Java :: IO .................. SUCCESS [ 0.060 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Amazon Web Services SUCCESS [ 2.670 s]
[INFO] Apache Beam :: Runners :: Local Java Core .......... SUCCESS [ 1.578 s]
[INFO] Apache Beam :: Runners :: Direct Java .............. SUCCESS [ 10.542 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: AMQP .......... SUCCESS [ 2.410 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Common ........ SUCCESS [ 1.776 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Cassandra ..... SUCCESS [ 2.721 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch . SUCCESS [ 1.900 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests SUCCESS [ 0.525 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: Common SUCCESS [ 0.589 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: 2.x SUCCESS [ 1.819 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: 5.x SUCCESS [ 1.877 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: XML ........... SUCCESS [ 2.270 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Protobuf SUCCESS [ 2.497 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Google Cloud Platform SUCCESS [ 4.969 s]
[INFO] Apache Beam :: Runners :: Google Cloud Dataflow .... SUCCESS [ 8.706 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: File-based-io-tests SUCCESS [ 2.182 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop Common . SUCCESS [ 3.026 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop File System SUCCESS [ 3.657 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: JDBC .......... SUCCESS [ 3.051 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop Input Format SUCCESS [ 8.544 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: HBase ......... SUCCESS [ 5.564 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: HCatalog ...... SUCCESS [ 9.062 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: JMS ........... SUCCESS [ 2.212 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Kafka ......... SUCCESS [ 2.666 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Kinesis ....... SUCCESS [ 1.835 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: MongoDB ....... SUCCESS [ 3.124 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: MQTT .......... SUCCESS [ 2.194 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Redis ......... SUCCESS [ 2.542 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Solr .......... SUCCESS [ 4.551 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Tika .......... SUCCESS [ 3.131 s]
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes .... SUCCESS [ 0.021 s]
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Starter SUCCESS [ 7.019 s]
[INFO] Apache Beam :: Examples ............................ SUCCESS [ 0.029 s]
[INFO] Apache Beam :: Examples :: Java .................... SUCCESS [ 3.666 s]
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Examples SUCCESS [ 21.453 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Jackson SUCCESS [ 2.214 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Join library SUCCESS [ 1.875 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Sketching SUCCESS [ 2.332 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Sorter SUCCESS [ 2.450 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: SQL ... SUCCESS [ 13.326 s]
[INFO] Apache Beam :: SDKs :: Java :: Nexmark ............. SUCCESS [ 16.418 s]
[INFO] Apache Beam :: SDKs :: Python ...................... SUCCESS [ 12.394 s]
[INFO] Apache Beam :: SDKs :: Python :: Container ......... SUCCESS [ 7.195 s]
[INFO] Apache Beam :: Runners :: Java Fn Execution ........ SUCCESS [ 3.208 s]
[INFO] Apache Beam :: Runners :: Java Local Artifact Service SUCCESS [ 2.607 s]
[INFO] Apache Beam :: Runners :: Reference ................ SUCCESS [ 0.023 s]
[INFO] Apache Beam :: Runners :: Reference :: Java ........ FAILURE [ 0.578 s]
[INFO] Apache Beam :: Runners :: Reference :: Job Orchestrator SKIPPED
[INFO] Apache Beam :: Runners :: Flink .................... SKIPPED
[INFO] Apache Beam :: Runners :: Gearpump ................. SKIPPED
[INFO] Apache Beam :: Runners :: Spark .................... SKIPPED
[INFO] Apache Beam :: Runners :: Apex ..................... SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform .... SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform :: GCE metadata provisioning SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform :: GCS artifact proxy SKIPPED
[INFO] Apache Beam :: Runners :: Extensions Java .......... SKIPPED
[INFO] Apache Beam :: Runners :: Extensions Java :: Metrics SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Aggregated Javadoc .. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 05:13 min
[INFO] Finished at: 2018-04-26T00:14:02Z
[INFO] Final Memory: 532M/11154M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.7.0:compile (default-compile) on project beam-runners-reference-java: Compilation failure: Compilation failure:
[ERROR] <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/34dc0d65/beam/runners/reference/java/src/main/java/org/apache/beam/runners/reference/testing/InProcessManagedChannelFactory.java>:[23,38] package org.apache.beam.sdk.fn.channel does not exist
[ERROR] <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/34dc0d65/beam/runners/reference/java/src/main/java/org/apache/beam/runners/reference/testing/InProcessManagedChannelFactory.java>:[30,53] cannot find symbol
[ERROR] symbol: class ManagedChannelFactory
[ERROR] <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/34dc0d65/beam/runners/reference/java/src/main/java/org/apache/beam/runners/reference/PortableRunner.java>:[49,38] package org.apache.beam.sdk.fn.channel does not exist
[ERROR] <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/34dc0d65/beam/runners/reference/java/src/main/java/org/apache/beam/runners/reference/PortableRunner.java>:[69,17] cannot find symbol
[ERROR] symbol: class ManagedChannelFactory
[ERROR] location: class org.apache.beam.runners.reference.PortableRunner
[ERROR] <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/34dc0d65/beam/runners/reference/java/src/main/java/org/apache/beam/runners/reference/PortableRunner.java>:[82,57] cannot find symbol
[ERROR] symbol: class ManagedChannelFactory
[ERROR] location: class org.apache.beam.runners.reference.PortableRunner
[ERROR] <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/34dc0d65/beam/runners/reference/java/src/main/java/org/apache/beam/runners/reference/PortableRunner.java>:[128,7] cannot find symbol
[ERROR] symbol: class ManagedChannelFactory
[ERROR] location: class org.apache.beam.runners.reference.PortableRunner
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :beam-runners-reference-java
STDERR: log4j:WARN No appenders could be found for logger (freemarker.cache).
log4j:WARN Please initialize the log4j system properly.
Warning: Lookahead adequacy checking not being performed since option LOOKAHEAD is more than 1. Set option FORCE_LA_CHECK to true to force checking.
/usr/local/lib/python2.7/dist-packages/setuptools/dist.py:397: UserWarning: Normalizing '2.5.0.dev' to '2.5.0.dev0'
normalized_version,
/usr/local/lib/python2.7/dist-packages/setuptools/dist.py:397: UserWarning: Normalizing '2.5.0.dev' to '2.5.0.dev0'
normalized_version,
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/34dc0d65/beam/sdks/python/gen_protos.py>:49: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
WARNING:root:Installing grpcio-tools into <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/34dc0d65/beam/sdks/python/.eggs/grpcio-wheels>
WARNING:root:Installing grpcio-tools took 3.72 seconds.
The script virtualenv is installed in '<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/34dc0d65/beam/sdks/python/target/python/bin'> which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
The scripts tox and tox-quickstart are installed in '<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/34dc0d65/beam/sdks/python/target/python/bin'> which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/34dc0d65/beam/sdks/python/target/python/lib/python2.7/site-packages/setuptools/dist.py>:397: UserWarning: Normalizing '2.5.0.dev' to '2.5.0.dev0'
normalized_version,
2018-04-26 00:14:04,049 34dc0d65 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-04-26 00:14:04,068 34dc0d65 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/34dc0d65/beam/sdks/python/target/apache-beam-2.5.0.dev0.tar.gz>
2018-04-26 00:14:04,352 34dc0d65 MainThread beam_integration_benchmark(1/1) INFO Ran: {python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/34dc0d65/beam/sdks/python/target/apache-beam-2.5.0.dev0.tar.gz}> ReturnCode:1
STDOUT:
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:397: UserWarning: Normalizing '2.5.0.dev' to '2.5.0.dev0'
normalized_version,
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: invalid command 'nosetests'
2018-04-26 00:14:04,353 34dc0d65 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-04-26 00:14:04,354 34dc0d65 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-04-26 00:14:04,356 34dc0d65 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 780, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-04-26 00:14:04,356 34dc0d65 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-04-26 00:14:04,392 34dc0d65 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-04-26 00:14:04,393 34dc0d65 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/34dc0d65/pkb.log>
2018-04-26 00:14:04,393 34dc0d65 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/34dc0d65/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user shoyer@google.com
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user boyuanz@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1192
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1192/display/redirect?page=changes>
Changes:
[relax] Add support for @Element injection in NewDoFn.
[relax] Add @Timestamp annotation to NewDoFn
[relax] Add support for OutputReceiver and MultiOutputReceiver. Currently we
[relax] Add support for timestamp and time domain to timers. Convert word count
[relax] Add some more tests.
[relax] Rewrite MapElements to use new-style.
[relax] Port common transforms to new style, excluding those that use
[relax] Fix javadoc.
[relax] Fix compilation error.
[relax] Fix CheckStyle errors.
[relax] Add proper downcast for element type.
[relax] Fix CheckStyle error.
[relax] Change MultiOutputReceiver to vend OutputReceivers.
[relax] Fix ParDoTest
[relax] Fix DoFnTester
[relax] Address CR comments.
[relax] Don't allow splitting a restriction into a different timestamp.
[relax] Add @Nullable annotations to DoFnTester.
[relax] Usage of the new parameters will break on Dataflow until Dataflow's
------------------------------------------
[...truncated 4.35 KB...]
Collecting contextlib2>=0.5.1 (from -r PerfKitBenchmarker/requirements.txt (line 24))
Using cached https://files.pythonhosted.org/packages/a2/71/8273a7eeed0aff6a854237ab5453bc9aa67deb49df4832801c21f0ff3782/contextlib2-0.5.5-py2.py3-none-any.whl
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins931873738335197626.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.0.1)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9f/ff/68860dec8b90424028ca54df8b9fc8cd1c47ef58710bd5e591ec8c619243/monotonic-1.4-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.4 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8495447775697176177.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-04-25 18:06:43,423 082316c7 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/082316c7/pkb.log>
2018-04-25 18:06:43,423 082316c7 MainThread INFO PerfKitBenchmarker version: v1.12.0-514-g92e9b3f
2018-04-25 18:06:43,424 082316c7 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-04-25 18:06:43,632 082316c7 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-04-25 18:06:43,633 082316c7 MainThread INFO Initializing the edw service decoder
2018-04-25 18:06:43,794 082316c7 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-04-25 18:06:43,797 082316c7 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-04-25 18:06:43,798 082316c7 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-04-25 18:06:49,055 082316c7 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-04-25 18:13:47,012 082316c7 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-04-25 18:13:47,031 082316c7 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/082316c7/beam/sdks/python/target/apache-beam-2.5.0.dev0.tar.gz>
2018-04-25 18:13:47,285 082316c7 MainThread beam_integration_benchmark(1/1) INFO Ran: {python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/082316c7/beam/sdks/python/target/apache-beam-2.5.0.dev0.tar.gz}> ReturnCode:1
STDOUT:
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:397: UserWarning: Normalizing '2.5.0.dev' to '2.5.0.dev0'
normalized_version,
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: invalid command 'nosetests'
2018-04-25 18:13:47,286 082316c7 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-04-25 18:13:47,287 082316c7 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-04-25 18:13:47,289 082316c7 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 780, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-04-25 18:13:47,290 082316c7 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-04-25 18:13:47,322 082316c7 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-04-25 18:13:47,323 082316c7 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/082316c7/pkb.log>
2018-04-25 18:13:47,323 082316c7 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/082316c7/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user shoyer@google.com
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user boyuanz@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1191
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1191/display/redirect?page=changes>
Changes:
[szewinho] [BEAM-4153] Fixing performance test of spark, added option to trigger
------------------------------------------
[...truncated 4.27 KB...]
Collecting contextlib2>=0.5.1 (from -r PerfKitBenchmarker/requirements.txt (line 24))
Using cached https://files.pythonhosted.org/packages/a2/71/8273a7eeed0aff6a854237ab5453bc9aa67deb49df4832801c21f0ff3782/contextlib2-0.5.5-py2.py3-none-any.whl
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3539505579878232616.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.0.1)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9f/ff/68860dec8b90424028ca54df8b9fc8cd1c47ef58710bd5e591ec8c619243/monotonic-1.4-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.4 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2268663210287148591.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-04-25 12:01:05,082 2b20ea63 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/2b20ea63/pkb.log>
2018-04-25 12:01:05,083 2b20ea63 MainThread INFO PerfKitBenchmarker version: v1.12.0-511-g5db1fe4
2018-04-25 12:01:05,084 2b20ea63 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-04-25 12:01:05,357 2b20ea63 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-04-25 12:01:05,358 2b20ea63 MainThread INFO Initializing the edw service decoder
2018-04-25 12:01:05,570 2b20ea63 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-04-25 12:01:05,574 2b20ea63 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-04-25 12:01:05,574 2b20ea63 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-04-25 12:01:13,183 2b20ea63 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-04-25 12:08:20,711 2b20ea63 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-04-25 12:08:20,733 2b20ea63 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/2b20ea63/beam/sdks/python/target/apache-beam-2.5.0.dev0.tar.gz>
2018-04-25 12:08:21,055 2b20ea63 MainThread beam_integration_benchmark(1/1) INFO Ran: {python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/2b20ea63/beam/sdks/python/target/apache-beam-2.5.0.dev0.tar.gz}> ReturnCode:1
STDOUT:
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:397: UserWarning: Normalizing '2.5.0.dev' to '2.5.0.dev0'
normalized_version,
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: invalid command 'nosetests'
2018-04-25 12:08:21,056 2b20ea63 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-04-25 12:08:21,057 2b20ea63 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-04-25 12:08:21,060 2b20ea63 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 780, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-04-25 12:08:21,061 2b20ea63 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-04-25 12:08:21,085 2b20ea63 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-04-25 12:08:21,086 2b20ea63 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/2b20ea63/pkb.log>
2018-04-25 12:08:21,087 2b20ea63 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/2b20ea63/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user shoyer@google.com
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user boyuanz@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1190
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1190/display/redirect?page=changes>
Changes:
[kedin] Add JsonToRow Transform
[kedin] Convert JsonToRow from MapElements.via() to ParDo
------------------------------------------
[...truncated 4.29 KB...]
Collecting contextlib2>=0.5.1 (from -r PerfKitBenchmarker/requirements.txt (line 24))
Using cached https://files.pythonhosted.org/packages/a2/71/8273a7eeed0aff6a854237ab5453bc9aa67deb49df4832801c21f0ff3782/contextlib2-0.5.5-py2.py3-none-any.whl
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins62554953466333455.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.0.1)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9f/ff/68860dec8b90424028ca54df8b9fc8cd1c47ef58710bd5e591ec8c619243/monotonic-1.4-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.4 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3216440880046143805.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-04-25 06:01:01,465 8b17c8bb MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/8b17c8bb/pkb.log>
2018-04-25 06:01:01,465 8b17c8bb MainThread INFO PerfKitBenchmarker version: v1.12.0-511-g5db1fe4
2018-04-25 06:01:01,466 8b17c8bb MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-04-25 06:01:01,705 8b17c8bb MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-04-25 06:01:01,705 8b17c8bb MainThread INFO Initializing the edw service decoder
2018-04-25 06:01:01,891 8b17c8bb MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-04-25 06:01:01,895 8b17c8bb MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-04-25 06:01:01,896 8b17c8bb MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-04-25 06:01:10,140 8b17c8bb MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-04-25 06:08:19,006 8b17c8bb MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-04-25 06:08:19,025 8b17c8bb MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/8b17c8bb/beam/sdks/python/target/apache-beam-2.5.0.dev0.tar.gz>
2018-04-25 06:08:19,280 8b17c8bb MainThread beam_integration_benchmark(1/1) INFO Ran: {python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/8b17c8bb/beam/sdks/python/target/apache-beam-2.5.0.dev0.tar.gz}> ReturnCode:1
STDOUT:
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:397: UserWarning: Normalizing '2.5.0.dev' to '2.5.0.dev0'
normalized_version,
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: invalid command 'nosetests'
2018-04-25 06:08:19,281 8b17c8bb MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-04-25 06:08:19,282 8b17c8bb MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-04-25 06:08:19,284 8b17c8bb MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 780, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-04-25 06:08:19,284 8b17c8bb MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-04-25 06:08:19,308 8b17c8bb MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-04-25 06:08:19,309 8b17c8bb MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/8b17c8bb/pkb.log>
2018-04-25 06:08:19,309 8b17c8bb MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/8b17c8bb/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user shoyer@google.com
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user boyuanz@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1189
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1189/display/redirect?page=changes>
Changes:
[tgroh] Cleanups in GroupByKeyOnlyEvaluatorFactory
[tgroh] Add a CollectionT to Bundle
[tgroh] Use Bundle in WatermarkManager
[apilloud] [BEAM-3983] Add utils for converting to BigQuery types
[rangadi] Disable flaky unbounded pipeline test
[wcn] Drain source when user function processing fails.
[kedin] Add primitive java types support to Row generation logic, add example
[aaltay] Unpinning Python jobs from Jenkins machines. (#5214)
[kirpichov] [BEAM-4166] Invoke @Setup in FnApiDoFnRunner
[wcn] Allow request and init hooks to update the context.
------------------------------------------
[...truncated 4.30 KB...]
Collecting contextlib2>=0.5.1 (from -r PerfKitBenchmarker/requirements.txt (line 24))
Using cached https://files.pythonhosted.org/packages/a2/71/8273a7eeed0aff6a854237ab5453bc9aa67deb49df4832801c21f0ff3782/contextlib2-0.5.5-py2.py3-none-any.whl
Collecting pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
Installing collected packages: absl-py, colorama, colorlog, blinker, futures, pint, numpy, contextlib2, ntlm-auth, requests-ntlm, xmltodict, pywinrm
Successfully installed absl-py-0.2.0 blinker-1.4 colorama-0.3.9 colorlog-2.6.0 contextlib2-0.5.5 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7789730774423475199.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.0.1)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9f/ff/68860dec8b90424028ca54df8b9fc8cd1c47ef58710bd5e591ec8c619243/monotonic-1.4-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, ply, future, cachetools, google-auth, google-gax, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.4 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8872160785721448385.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-04-25 00:01:04,895 2f59fd20 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/2f59fd20/pkb.log>
2018-04-25 00:01:04,895 2f59fd20 MainThread INFO PerfKitBenchmarker version: v1.12.0-511-g5db1fe4
2018-04-25 00:01:04,896 2f59fd20 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-04-25 00:01:05,129 2f59fd20 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-04-25 00:01:05,130 2f59fd20 MainThread INFO Initializing the edw service decoder
2018-04-25 00:01:05,275 2f59fd20 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-04-25 00:01:05,278 2f59fd20 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-04-25 00:01:05,279 2f59fd20 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-04-25 00:01:11,597 2f59fd20 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-04-25 00:07:10,099 2f59fd20 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-04-25 00:07:10,124 2f59fd20 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/2f59fd20/beam/sdks/python/target/apache-beam-2.5.0.dev0.tar.gz>
2018-04-25 00:07:10,436 2f59fd20 MainThread beam_integration_benchmark(1/1) INFO Ran: {python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/2f59fd20/beam/sdks/python/target/apache-beam-2.5.0.dev0.tar.gz}> ReturnCode:1
STDOUT:
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:397: UserWarning: Normalizing '2.5.0.dev' to '2.5.0.dev0'
normalized_version,
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: invalid command 'nosetests'
2018-04-25 00:07:10,437 2f59fd20 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-04-25 00:07:10,438 2f59fd20 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-04-25 00:07:10,440 2f59fd20 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 780, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-04-25 00:07:10,441 2f59fd20 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-04-25 00:07:10,464 2f59fd20 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-04-25 00:07:10,465 2f59fd20 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/2f59fd20/pkb.log>
2018-04-25 00:07:10,465 2f59fd20 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/2f59fd20/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user shoyer@google.com
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user boyuanz@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1188
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1188/display/redirect?page=changes>
Changes:
[tgroh] Stop implementing EvaluatorFactory in Registry
[aromanenko.dev] [BEAM-4066] Moved anonymous classes into inner ones
[sidhom] [BEAM-4149] Set worker id to "" if it is not set in the request header
[sidhom] [BEAM-3327] Refactor ControlClientPool to allow client multiplexing
[sidhom] [BEAM-3327] Basic Docker environment factory
[iemejia] [BEAM-4018] Add a ByteKeyRangeTracker based on RestrictionTracker for
------------------------------------------
[...truncated 13.13 KB...]
Downloading https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Downloading https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl (330kB)
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Downloading https://files.pythonhosted.org/packages/2a/1f/4124f15e1132a2eeeaf616d825990bb1d395b4c2c37362654ea5cd89bb42/proto-google-cloud-datastore-v1-0.90.4.tar.gz
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Downloading https://files.pythonhosted.org/packages/73/d0/17ce873331aaf529ab238464a15fd7bdc1ba8d2c684789970a7fa8b505a8/googledatastore-7.0.1.tar.gz
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Downloading https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Downloading https://files.pythonhosted.org/packages/c0/a2/2eeffa0069830f00016196dfdd69491cf562372b5353f2e8e378b3c2cb0a/proto-google-cloud-pubsub-v1-0.15.4.tar.gz
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Downloading https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl (41kB)
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Downloading https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl (154kB)
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) (1.8.0)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.0.1)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Downloading https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Downloading https://files.pythonhosted.org/packages/00/03/d25bed04ec8d930bcfa488ba81a2ecbf7eb36ae3ffd7e8f5be0d036a89c9/googleapis-common-protos-1.5.3.tar.gz
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Downloading https://files.pythonhosted.org/packages/bc/a7/0225bd7a95e037a0afa90b2dd9534d0c79cd62283a5bddb30a3197579cbc/gapic-google-cloud-pubsub-v1-0.15.4.tar.gz
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Downloading https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl (52kB)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Downloading https://files.pythonhosted.org/packages/9f/ff/68860dec8b90424028ca54df8b9fc8cd1c47ef58710bd5e591ec8c619243/monotonic-1.4-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Downloading https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl (46kB)
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Downloading https://files.pythonhosted.org/packages/9b/28/f26f67381cb23e81271b8d66c00a846ad9d25a909ae1ae1df8222fad2744/grpc-google-iam-v1-0.11.4.tar.gz
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Downloading https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Downloading https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl (65kB)
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Downloading https://files.pythonhosted.org/packages/96/e0/430fcdb6b3ef1ae534d231397bee7e9304be14a47a267e82ebcb3323d0b5/ply-3.8.tar.gz (157kB)
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Downloading https://files.pythonhosted.org/packages/00/2b/8d082ddfed935f3608cc61140df6dcbf0edea1bc3ab52fb6c29ae3e81e85/future-0.16.0.tar.gz (824kB)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Downloading https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
Building wheels for collected packages: avro, dill, hdfs, httplib2, pyvcf, proto-google-cloud-datastore-v1, googledatastore, proto-google-cloud-pubsub-v1, googleapis-common-protos, gapic-google-cloud-pubsub-v1, grpc-google-iam-v1, ply, future
Running setup.py bdist_wheel for avro: started
Running setup.py bdist_wheel for avro: finished with status 'done'
Stored in directory: /home/jenkins/.cache/pip/wheels/bf/0b/75/1b3517b7d36ddc8ba5d22c0df5eb01e83979f34420066d643e
Running setup.py bdist_wheel for dill: started
Running setup.py bdist_wheel for dill: finished with status 'done'
Stored in directory: /home/jenkins/.cache/pip/wheels/17/74/0a/250b442a5196d5c0d80f7d4259a0a26a9c9af0e6b0ee98a335
Running setup.py bdist_wheel for hdfs: started
Running setup.py bdist_wheel for hdfs: finished with status 'done'
Stored in directory: /home/jenkins/.cache/pip/wheels/8c/89/59/b95e6bf2d5957541862f2cd109a052f26b61dc2fad2f04a1b4
Running setup.py bdist_wheel for httplib2: started
Running setup.py bdist_wheel for httplib2: finished with status 'done'
Stored in directory: /home/jenkins/.cache/pip/wheels/36/f2/49/5adbf90fba31e02a7784e1147d7f8b6c4af3718739e568c8cb
Running setup.py bdist_wheel for pyvcf: started
Running setup.py bdist_wheel for pyvcf: finished with status 'done'
Stored in directory: /home/jenkins/.cache/pip/wheels/81/91/41/3272543c0b9c61da9c525f24ee35bae6fe8f60d4858c66805d
Running setup.py bdist_wheel for proto-google-cloud-datastore-v1: started
Running setup.py bdist_wheel for proto-google-cloud-datastore-v1: finished with status 'done'
Stored in directory: /home/jenkins/.cache/pip/wheels/bd/ce/33/8b769968db3761c42c7a91d8a0dbbafc50acfa0750866c8abd
Running setup.py bdist_wheel for googledatastore: started
Running setup.py bdist_wheel for googledatastore: finished with status 'done'
Stored in directory: /home/jenkins/.cache/pip/wheels/93/fa/12/99c490525fff40bcd240504f694bd02848479bc2debc31daae
Running setup.py bdist_wheel for proto-google-cloud-pubsub-v1: started
Running setup.py bdist_wheel for proto-google-cloud-pubsub-v1: finished with status 'done'
Stored in directory: /home/jenkins/.cache/pip/wheels/d1/0d/9e/95e7192ab2625847ac40b2bc618800bf5b6c984cd572a83314
Running setup.py bdist_wheel for googleapis-common-protos: started
Running setup.py bdist_wheel for googleapis-common-protos: finished with status 'done'
Stored in directory: /home/jenkins/.cache/pip/wheels/62/45/af/649bbf07b6595fda010be1bda667cd56d0444d07afc6f8b687
Running setup.py bdist_wheel for gapic-google-cloud-pubsub-v1: started
Running setup.py bdist_wheel for gapic-google-cloud-pubsub-v1: finished with status 'done'
Stored in directory: /home/jenkins/.cache/pip/wheels/f4/2b/10/bdcbc9be2ae4e437232e118056f026025cf2cc46d6dcf0d69d
Running setup.py bdist_wheel for grpc-google-iam-v1: started
Running setup.py bdist_wheel for grpc-google-iam-v1: finished with status 'done'
Stored in directory: /home/jenkins/.cache/pip/wheels/b6/c6/31/c20321a5a3fde456fc375b7c2814135e6e98bc0d74c40239d9
Running setup.py bdist_wheel for ply: started
Running setup.py bdist_wheel for ply: finished with status 'done'
Stored in directory: /home/jenkins/.cache/pip/wheels/f2/21/c0/f0056cc96847933daa961a19eb59a2ecd0228fdbe3376e7a68
Running setup.py bdist_wheel for future: started
Running setup.py bdist_wheel for future: finished with status 'done'
Stored in directory: /home/jenkins/.cache/pip/wheels/bf/c9/a3/c538d90ef17cf7823fa51fc701a7a7a910a80f6a405bf15b1a
Successfully built avro dill hdfs httplib2 pyvcf proto-google-cloud-datastore-v1 googledatastore proto-google-cloud-pubsub-v1 googleapis-common-protos gapic-google-cloud-pubsub-v1 grpc-google-iam-v1 ply future
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, mock, pyasn1-modules, rsa, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, ply, future, cachetools, google-auth, google-gax, proto-google-cloud-pubsub-v1, grpc-google-iam-v1, gapic-google-cloud-pubsub-v1, google-auth-httplib2, google-cloud-core, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.4 nose-1.3.7 oauth2client-4.1.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5292819246503380425.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-04-24 18:01:50,754 2d61a523 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/2d61a523/pkb.log>
2018-04-24 18:01:50,755 2d61a523 MainThread INFO PerfKitBenchmarker version: v1.12.0-511-g5db1fe4
2018-04-24 18:01:50,755 2d61a523 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-04-24 18:01:50,992 2d61a523 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-04-24 18:01:50,992 2d61a523 MainThread INFO Initializing the edw service decoder
2018-04-24 18:01:51,141 2d61a523 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-04-24 18:01:51,144 2d61a523 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-04-24 18:01:51,144 2d61a523 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-04-24 18:01:56,361 2d61a523 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-04-24 18:22:48,174 2d61a523 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-04-24 18:22:48,192 2d61a523 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/2d61a523/beam/sdks/python/target/apache-beam-2.5.0.dev0.tar.gz>
2018-04-24 18:22:48,415 2d61a523 MainThread beam_integration_benchmark(1/1) INFO Ran: {python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/2d61a523/beam/sdks/python/target/apache-beam-2.5.0.dev0.tar.gz}> ReturnCode:1
STDOUT:
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:397: UserWarning: Normalizing '2.5.0.dev' to '2.5.0.dev0'
normalized_version,
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: invalid command 'nosetests'
2018-04-24 18:22:48,416 2d61a523 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-04-24 18:22:48,417 2d61a523 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-04-24 18:22:48,418 2d61a523 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 780, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-04-24 18:22:48,419 2d61a523 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-04-24 18:22:48,436 2d61a523 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-04-24 18:22:48,437 2d61a523 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/2d61a523/pkb.log>
2018-04-24 18:22:48,437 2d61a523 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/2d61a523/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user shoyer@google.com
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user boyuanz@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1187
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1187/display/redirect?page=changes>
Changes:
[Pablo] Python Metrics now rely on StateSampler state.
[robertwb] [BEAM-4097] Set environment for Python sdk function specs.
[robertwb] Logging around default docker image environment.
------------------------------------------
[...truncated 251.72 KB...]
[INFO] --- maven-surefire-plugin:2.21.0:test (default-test) @ beam-sdks-java-io-elasticsearch-tests-5 ---
[INFO] Tests are skipped.
[INFO]
[INFO] --- build-helper-maven-plugin:3.0.0:regex-properties (render-artifact-id) @ beam-sdks-java-io-elasticsearch-tests-5 ---
[INFO]
[INFO] --- maven-jar-plugin:3.0.2:jar (default-jar) @ beam-sdks-java-io-elasticsearch-tests-5 ---
[INFO] Building jar: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/1d4cacfd/beam/sdks/java/io/elasticsearch-tests/elasticsearch-tests-5/target/beam-sdks-java-io-elasticsearch-tests-5-2.5.0-SNAPSHOT.jar>
[INFO]
[INFO] --- maven-site-plugin:3.7:attach-descriptor (attach-descriptor) @ beam-sdks-java-io-elasticsearch-tests-5 ---
[INFO] Skipping because packaging 'jar' is not pom.
[INFO]
[INFO] --- maven-jar-plugin:3.0.2:test-jar (default-test-jar) @ beam-sdks-java-io-elasticsearch-tests-5 ---
[INFO] Building jar: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/1d4cacfd/beam/sdks/java/io/elasticsearch-tests/elasticsearch-tests-5/target/beam-sdks-java-io-elasticsearch-tests-5-2.5.0-SNAPSHOT-tests.jar>
[INFO]
[INFO] --- maven-shade-plugin:3.1.0:shade (bundle-and-repackage) @ beam-sdks-java-io-elasticsearch-tests-5 ---
[INFO] Replacing original artifact with shaded artifact.
[INFO] Replacing <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/1d4cacfd/beam/sdks/java/io/elasticsearch-tests/elasticsearch-tests-5/target/beam-sdks-java-io-elasticsearch-tests-5-2.5.0-SNAPSHOT.jar> with <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/1d4cacfd/beam/sdks/java/io/elasticsearch-tests/elasticsearch-tests-5/target/beam-sdks-java-io-elasticsearch-tests-5-2.5.0-SNAPSHOT-shaded.jar>
[INFO] Replacing original test artifact with shaded test artifact.
[INFO] Replacing <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/1d4cacfd/beam/sdks/java/io/elasticsearch-tests/elasticsearch-tests-5/target/beam-sdks-java-io-elasticsearch-tests-5-2.5.0-SNAPSHOT-tests.jar> with <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/1d4cacfd/beam/sdks/java/io/elasticsearch-tests/elasticsearch-tests-5/target/beam-sdks-java-io-elasticsearch-tests-5-2.5.0-SNAPSHOT-shaded-tests.jar>
[INFO]
[INFO] --- maven-dependency-plugin:3.0.2:analyze-only (default) @ beam-sdks-java-io-elasticsearch-tests-5 ---
[INFO] No dependency problems found
[INFO]
[INFO] --- maven-install-plugin:2.5.2:install (default-install) @ beam-sdks-java-io-elasticsearch-tests-5 ---
[INFO] Installing <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/1d4cacfd/beam/sdks/java/io/elasticsearch-tests/elasticsearch-tests-5/target/beam-sdks-java-io-elasticsearch-tests-5-2.5.0-SNAPSHOT.jar> to /home/jenkins/.m2/repository/org/apache/beam/beam-sdks-java-io-elasticsearch-tests-5/2.5.0-SNAPSHOT/beam-sdks-java-io-elasticsearch-tests-5-2.5.0-SNAPSHOT.jar
[INFO] Installing <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/1d4cacfd/beam/sdks/java/io/elasticsearch-tests/elasticsearch-tests-5/pom.xml> to /home/jenkins/.m2/repository/org/apache/beam/beam-sdks-java-io-elasticsearch-tests-5/2.5.0-SNAPSHOT/beam-sdks-java-io-elasticsearch-tests-5-2.5.0-SNAPSHOT.pom
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Beam :: Parent .............................. SUCCESS [ 9.881 s]
[INFO] Apache Beam :: SDKs :: Java :: Build Tools ......... SUCCESS [ 6.198 s]
[INFO] Apache Beam :: Model ............................... SUCCESS [ 0.165 s]
[INFO] Apache Beam :: Model :: Pipeline ................... SUCCESS [ 18.446 s]
[INFO] Apache Beam :: Model :: Job Management ............. SUCCESS [ 7.539 s]
[INFO] Apache Beam :: Model :: Fn Execution ............... SUCCESS [ 8.611 s]
[INFO] Apache Beam :: SDKs ................................ SUCCESS [ 0.370 s]
[INFO] Apache Beam :: SDKs :: Go .......................... SUCCESS [ 54.686 s]
[INFO] Apache Beam :: SDKs :: Go :: Container ............. SUCCESS [ 31.697 s]
[INFO] Apache Beam :: SDKs :: Java ........................ SUCCESS [ 0.116 s]
[INFO] Apache Beam :: SDKs :: Java :: Core ................ SUCCESS [ 34.721 s]
[INFO] Apache Beam :: SDKs :: Java :: Fn Execution ........ SUCCESS [ 3.633 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions .......... SUCCESS [ 0.039 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Google Cloud Platform Core SUCCESS [ 3.716 s]
[INFO] Apache Beam :: Runners ............................. SUCCESS [ 0.067 s]
[INFO] Apache Beam :: Runners :: Core Construction Java ... SUCCESS [ 5.166 s]
[INFO] Apache Beam :: Runners :: Core Java ................ SUCCESS [ 5.855 s]
[INFO] Apache Beam :: SDKs :: Java :: Harness ............. SUCCESS [ 8.574 s]
[INFO] Apache Beam :: SDKs :: Java :: Container ........... SUCCESS [ 7.763 s]
[INFO] Apache Beam :: SDKs :: Java :: IO .................. SUCCESS [ 0.052 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Amazon Web Services SUCCESS [ 2.947 s]
[INFO] Apache Beam :: Runners :: Local Java Core .......... SUCCESS [ 1.585 s]
[INFO] Apache Beam :: Runners :: Direct Java .............. SUCCESS [ 10.820 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: AMQP .......... SUCCESS [ 2.789 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Common ........ SUCCESS [ 1.747 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Cassandra ..... SUCCESS [ 2.584 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch . SUCCESS [ 2.112 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests SUCCESS [ 0.849 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: Common SUCCESS [ 0.611 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: 2.x SUCCESS [ 1.850 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: 5.x FAILURE [ 3.342 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: XML ........... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Protobuf SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Google Cloud Platform SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Dataflow .... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: File-based-io-tests SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop Common . SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop File System SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: JDBC .......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop Input Format SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: HBase ......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: HCatalog ...... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: JMS ........... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Kafka ......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Kinesis ....... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: MongoDB ....... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: MQTT .......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Redis ......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Solr .......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Tika .......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes .... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Starter SKIPPED
[INFO] Apache Beam :: Examples ............................ SKIPPED
[INFO] Apache Beam :: Examples :: Java .................... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Examples SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Jackson SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Join library SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Sketching SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Sorter SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: SQL ... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Nexmark ............. SKIPPED
[INFO] Apache Beam :: SDKs :: Python ...................... SKIPPED
[INFO] Apache Beam :: SDKs :: Python :: Container ......... SKIPPED
[INFO] Apache Beam :: Runners :: Java Fn Execution ........ SKIPPED
[INFO] Apache Beam :: Runners :: Java Local Artifact Service SKIPPED
[INFO] Apache Beam :: Runners :: Reference ................ SKIPPED
[INFO] Apache Beam :: Runners :: Reference :: Java ........ SKIPPED
[INFO] Apache Beam :: Runners :: Reference :: Job Orchestrator SKIPPED
[INFO] Apache Beam :: Runners :: Flink .................... SKIPPED
[INFO] Apache Beam :: Runners :: Gearpump ................. SKIPPED
[INFO] Apache Beam :: Runners :: Spark .................... SKIPPED
[INFO] Apache Beam :: Runners :: Apex ..................... SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform .... SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform :: GCE metadata provisioning SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform :: GCS artifact proxy SKIPPED
[INFO] Apache Beam :: Runners :: Extensions Java .......... SKIPPED
[INFO] Apache Beam :: Runners :: Extensions Java :: Metrics SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Aggregated Javadoc .. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 04:03 min
[INFO] Finished at: 2018-04-24T12:05:16Z
[INFO] Final Memory: 157M/1485M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-install-plugin:2.5.2:install (default-install) on project beam-sdks-java-io-elasticsearch-tests-5: Failed to install metadata org.apache.beam:beam-sdks-java-io-elasticsearch-tests-5/maven-metadata.xml: Could not parse metadata /home/jenkins/.m2/repository/org/apache/beam/beam-sdks-java-io-elasticsearch-tests-5/maven-metadata-local.xml: in epilog non whitespace content is not allowed but got s (position: END_TAG seen ...</metadata>\ns... @12:2) -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :beam-sdks-java-io-elasticsearch-tests-5
STDERR:
2018-04-24 12:05:17,060 1d4cacfd MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-04-24 12:05:17,079 1d4cacfd MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 175, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 265, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 290, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/1d4cacfd/beam/sdks/python/target>
2018-04-24 12:05:17,080 1d4cacfd MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-04-24 12:05:17,082 1d4cacfd MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 780, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 175, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 265, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 290, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/1d4cacfd/beam/sdks/python/target>
2018-04-24 12:05:17,082 1d4cacfd MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-04-24 12:05:17,126 1d4cacfd MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-04-24 12:05:17,127 1d4cacfd MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/1d4cacfd/pkb.log>
2018-04-24 12:05:17,128 1d4cacfd MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/1d4cacfd/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user shoyer@google.com
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user boyuanz@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1186
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1186/display/redirect?page=changes>
Changes:
[apilloud] [SQL] Embed BeamSqlTable in BeamCalciteTable
[owenzhang1990] [BEAM-4129] Run WordCount example on Gearpump runner with Gradle
[sidhom] Fix python lint error
------------------------------------------
[...truncated 30.28 KB...]
[INFO] --- maven-surefire-plugin:2.21.0:test (default-test) @ beam-sdks-java-build-tools ---
[INFO] Tests are skipped.
[INFO]
[INFO] --- build-helper-maven-plugin:3.0.0:regex-properties (render-artifact-id) @ beam-sdks-java-build-tools ---
[INFO]
[INFO] --- maven-jar-plugin:3.0.2:jar (default-jar) @ beam-sdks-java-build-tools ---
[INFO] Building jar: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/92a4e432/beam/sdks/java/build-tools/target/beam-sdks-java-build-tools-2.5.0-SNAPSHOT.jar>
[INFO]
[INFO] --- maven-site-plugin:3.7:attach-descriptor (attach-descriptor) @ beam-sdks-java-build-tools ---
[INFO] Skipping because packaging 'jar' is not pom.
[INFO]
[INFO] --- maven-jar-plugin:3.0.2:test-jar (default-test-jar) @ beam-sdks-java-build-tools ---
[INFO] Building jar: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/92a4e432/beam/sdks/java/build-tools/target/beam-sdks-java-build-tools-2.5.0-SNAPSHOT-tests.jar>
[INFO]
[INFO] --- maven-shade-plugin:3.1.0:shade (bundle-and-repackage) @ beam-sdks-java-build-tools ---
[INFO] Replacing original artifact with shaded artifact.
[INFO] Replacing <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/92a4e432/beam/sdks/java/build-tools/target/beam-sdks-java-build-tools-2.5.0-SNAPSHOT.jar> with <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/92a4e432/beam/sdks/java/build-tools/target/beam-sdks-java-build-tools-2.5.0-SNAPSHOT-shaded.jar>
[INFO] Replacing original test artifact with shaded test artifact.
[INFO] Replacing <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/92a4e432/beam/sdks/java/build-tools/target/beam-sdks-java-build-tools-2.5.0-SNAPSHOT-tests.jar> with <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/92a4e432/beam/sdks/java/build-tools/target/beam-sdks-java-build-tools-2.5.0-SNAPSHOT-shaded-tests.jar>
[INFO]
[INFO] --- maven-dependency-plugin:3.0.2:analyze-only (default) @ beam-sdks-java-build-tools ---
[INFO] No dependency problems found
[INFO]
[INFO] --- maven-install-plugin:2.5.2:install (default-install) @ beam-sdks-java-build-tools ---
[INFO] Installing <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/92a4e432/beam/sdks/java/build-tools/target/beam-sdks-java-build-tools-2.5.0-SNAPSHOT.jar> to /home/jenkins/.m2/repository/org/apache/beam/beam-sdks-java-build-tools/2.5.0-SNAPSHOT/beam-sdks-java-build-tools-2.5.0-SNAPSHOT.jar
[INFO] Installing <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/92a4e432/beam/sdks/java/build-tools/pom.xml> to /home/jenkins/.m2/repository/org/apache/beam/beam-sdks-java-build-tools/2.5.0-SNAPSHOT/beam-sdks-java-build-tools-2.5.0-SNAPSHOT.pom
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Beam :: Parent .............................. SUCCESS [ 9.385 s]
[INFO] Apache Beam :: SDKs :: Java :: Build Tools ......... FAILURE [ 5.965 s]
[INFO] Apache Beam :: Model ............................... SKIPPED
[INFO] Apache Beam :: Model :: Pipeline ................... SKIPPED
[INFO] Apache Beam :: Model :: Job Management ............. SKIPPED
[INFO] Apache Beam :: Model :: Fn Execution ............... SKIPPED
[INFO] Apache Beam :: SDKs ................................ SKIPPED
[INFO] Apache Beam :: SDKs :: Go .......................... SKIPPED
[INFO] Apache Beam :: SDKs :: Go :: Container ............. SKIPPED
[INFO] Apache Beam :: SDKs :: Java ........................ SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Core ................ SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Fn Execution ........ SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions .......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Google Cloud Platform Core SKIPPED
[INFO] Apache Beam :: Runners ............................. SKIPPED
[INFO] Apache Beam :: Runners :: Core Construction Java ... SKIPPED
[INFO] Apache Beam :: Runners :: Core Java ................ SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Harness ............. SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Container ........... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO .................. SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Amazon Web Services SKIPPED
[INFO] Apache Beam :: Runners :: Local Java Core .......... SKIPPED
[INFO] Apache Beam :: Runners :: Direct Java .............. SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: AMQP .......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Common ........ SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Cassandra ..... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch . SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: Common SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: 2.x SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: 5.x SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: XML ........... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Protobuf SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Google Cloud Platform SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Dataflow .... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: File-based-io-tests SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop Common . SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop File System SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: JDBC .......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop Input Format SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: HBase ......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: HCatalog ...... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: JMS ........... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Kafka ......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Kinesis ....... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: MongoDB ....... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: MQTT .......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Redis ......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Solr .......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Tika .......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes .... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Starter SKIPPED
[INFO] Apache Beam :: Examples ............................ SKIPPED
[INFO] Apache Beam :: Examples :: Java .................... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Examples SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Jackson SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Join library SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Sketching SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Sorter SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: SQL ... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Nexmark ............. SKIPPED
[INFO] Apache Beam :: SDKs :: Python ...................... SKIPPED
[INFO] Apache Beam :: SDKs :: Python :: Container ......... SKIPPED
[INFO] Apache Beam :: Runners :: Java Fn Execution ........ SKIPPED
[INFO] Apache Beam :: Runners :: Java Local Artifact Service SKIPPED
[INFO] Apache Beam :: Runners :: Reference ................ SKIPPED
[INFO] Apache Beam :: Runners :: Reference :: Java ........ SKIPPED
[INFO] Apache Beam :: Runners :: Reference :: Job Orchestrator SKIPPED
[INFO] Apache Beam :: Runners :: Flink .................... SKIPPED
[INFO] Apache Beam :: Runners :: Gearpump ................. SKIPPED
[INFO] Apache Beam :: Runners :: Spark .................... SKIPPED
[INFO] Apache Beam :: Runners :: Apex ..................... SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform .... SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform :: GCE metadata provisioning SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform :: GCS artifact proxy SKIPPED
[INFO] Apache Beam :: Runners :: Extensions Java .......... SKIPPED
[INFO] Apache Beam :: Runners :: Extensions Java :: Metrics SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Aggregated Javadoc .. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 21.604 s
[INFO] Finished at: 2018-04-24T06:01:16Z
[INFO] Final Memory: 69M/695M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-install-plugin:2.5.2:install (default-install) on project beam-sdks-java-build-tools: Failed to install metadata org.apache.beam:beam-sdks-java-build-tools/maven-metadata.xml: Could not read metadata /home/jenkins/.m2/repository/org/apache/beam/beam-sdks-java-build-tools/maven-metadata-local.xml: input contained no data -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :beam-sdks-java-build-tools
STDERR:
2018-04-24 06:01:17,369 92a4e432 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-04-24 06:01:17,375 92a4e432 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 175, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 265, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 290, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/92a4e432/beam/sdks/python/target>
2018-04-24 06:01:17,376 92a4e432 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-04-24 06:01:17,378 92a4e432 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 780, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 175, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 265, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 290, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/92a4e432/beam/sdks/python/target>
2018-04-24 06:01:17,378 92a4e432 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-04-24 06:01:17,415 92a4e432 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-04-24 06:01:17,416 92a4e432 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/92a4e432/pkb.log>
2018-04-24 06:01:17,416 92a4e432 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/92a4e432/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user owenzhang1990@gmail.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user shoyer@google.com
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user boyuanz@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1185
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1185/display/redirect?page=changes>
Changes:
[geet.kumar75] BEAM-4038: Support Kafka Headers in KafkaIO
[geet.kumar75] BEAM-4038: Update license information
[geet.kumar75] Update code formatting
[geet.kumar75] Remove custom implementations of KafkaHeader, KafkaHeaders, etc.
[geet.kumar75] Changes based on review comments
[daniel.o.programmer] [BEAM-3513] Removing PrimitiveCombineGroupedValues override w/ FnAPI.
[echauchot] Introduce MetricsPusher in runner core to regularly push aggregated
[echauchot] Instanciate MetricsPusher in runner-specific code because we need
[echauchot] Improve MetricsPusher: do not aggregate metrics when not needed, improve
[echauchot] Create JsonMetricsSerializer
[echauchot] Stop MetricsPusher thread by observing pipeline state and improve the
[echauchot] Make metrics sink configurable through PipelineOptions, pass
[echauchot] Add MetricsPusher tests specific to Spark (because Spark streaming tests
[echauchot] Add a MetricksPusher test to runner-core (batch and streaming are done
[echauchot] Push metrics at the end of a batch pipeline in flink runner
[echauchot] improve MetricsPusher lifecycle and thread safety
[echauchot] Make MetricsPusher merge a list a MetricsContainerStepMaps because there
[echauchot] Fix thread synchronisation and replace usages of instance variable by
[echauchot] Clear dummyMetricsSink before test
[echauchot] Push metrics at the end of a batch pipeline in spark runner
[echauchot] Improve MetricsPusher teardown to enable multiple pipelines in a single
[echauchot] Manually generate json and remove jackson
[echauchot] Replace use of http client by use of java.net.HttpUrlConnection and deal
[echauchot] Remove DEFAULT_PERIOD constant in favor of already existing
[echauchot] Remove unneeded null check, format
[echauchot] convert MetricsSink to an interface with a single writeMetrics method
[echauchot] Remove MetricsSerializer base class and inline serialization in
[echauchot] Dynamically create the sinks by reflection
[echauchot] Split DummyMetricsSink into NoOpMetricsSink (default sink) and
[echauchot] Reduce overhead when no metricsSink is provided, do not start polling
[echauchot] Make MetricsPusher a regular object instead of a singleton to allow
[echauchot] Explicitely start MetricsPusher from the runners
[echauchot] Separate MetricsHttpSink POC to a new runners-extensions artifact and
[echauchot] Fix cycle bug between teardown() and pushmetrics()
[echauchot] Update MetricsPusher and TestMetricsSink to new serializable
[echauchot] Use regular jackson object mapper to serialize metrics now that they are
[echauchot] Give MetricsPusher a bit of time to push before assert in test
[echauchot] Make MetricsPusher thread a daemon
[echauchot] Fix build and clean: dependencies, rat, checkstyle, findbugs, remove
[geet.kumar75] Support kafka versions 0.10.1.0 and above
[echauchot] Move build to gradle
[echauchot] MetricsSink no more needs to be generic
[echauchot] SparkRunnerDebugger does not need to export metrics as it does not run
[tgroh] Use Existing Matchers in WatermarkManagerTest
[echauchot] Move MetricsHttpSink and related classes to a new sub-module
[kirpichov] Consistently handle EmptyMatchTreatment
[rangadi] Add 10 millis sleep when there are no elements left in a partition.
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam3 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 0f2ba71e1b6db88ed79744e363586a8ff16dcb08 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 0f2ba71e1b6db88ed79744e363586a8ff16dcb08
Commit message: "Merge pull request #5195: Use Existing Matchers in WatermarkManagerTest"
> git rev-list --no-walk 247a62ff1d4368f1e7c2ade6bed5dec71d8d2bcc # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5809426347260041593.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3285690391011211747.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6101718920793672879.sh
+ virtualenv .env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/bin/python>
Installing setuptools, pip, wheel...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5530998206000920874.sh
+ .env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.env/lib/python2.7/site-packages (39.0.1)
Requirement already up-to-date: pip in ./.env/lib/python2.7/site-packages (10.0.1)
cheetah 2.4.4 requires Markdown>=2.0.1, which is not installed.
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins845102992366333830.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4490238239090703186.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14)) (0.1.7)
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15)) (2.10)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16)) (39.0.1)
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17)) (2.6.0)
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18)) (1.4)
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19)) (3.2.0)
Requirement already satisfied: PyYAML==3.12 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 20)) (3.12)
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21)) (0.8.1)
Requirement already satisfied: numpy==1.13.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22)) (1.13.3)
Requirement already satisfied: functools32 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 23)) (3.2.3.post2)
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24)) (0.5.5)
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25)) (0.3.0)
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17)) (0.2.5)
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.11.0)
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.0)
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.6)
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.1.4)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.1.18)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.2)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.19)
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
cheetah 2.4.4 requires Markdown>=2.0.1, which is not installed.
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3190979353997182633.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (1.8.2)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (0.2.6)
Requirement already satisfied: grpcio<2,>=1.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (1.8.3)
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/urllib3/util/ssl_.py>:339: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
SNIMissingWarning
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/urllib3/util/ssl_.py>:137: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecurePlatformWarning
Could not find a version that satisfies the requirement hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0) (from versions: )
No matching distribution found for hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user geet.kumar75@gmail.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user shoyer@google.com
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user boyuanz@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1184
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1184/display/redirect>
------------------------------------------
[...truncated 250.32 KB...]
[INFO] Compiling 3 source files to <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/9cecf502/beam/sdks/java/io/elasticsearch-tests/elasticsearch-tests-5/target/test-classes>
[INFO]
[INFO] --- maven-checkstyle-plugin:3.0.0:check (default) @ beam-sdks-java-io-elasticsearch-tests-5 ---
[INFO]
[INFO] --- maven-surefire-plugin:2.21.0:test (default-test) @ beam-sdks-java-io-elasticsearch-tests-5 ---
[INFO] Tests are skipped.
[INFO]
[INFO] --- build-helper-maven-plugin:3.0.0:regex-properties (render-artifact-id) @ beam-sdks-java-io-elasticsearch-tests-5 ---
[INFO]
[INFO] --- maven-jar-plugin:3.0.2:jar (default-jar) @ beam-sdks-java-io-elasticsearch-tests-5 ---
[INFO] Building jar: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/9cecf502/beam/sdks/java/io/elasticsearch-tests/elasticsearch-tests-5/target/beam-sdks-java-io-elasticsearch-tests-5-2.5.0-SNAPSHOT.jar>
[INFO]
[INFO] --- maven-site-plugin:3.7:attach-descriptor (attach-descriptor) @ beam-sdks-java-io-elasticsearch-tests-5 ---
[INFO] Skipping because packaging 'jar' is not pom.
[INFO]
[INFO] --- maven-jar-plugin:3.0.2:test-jar (default-test-jar) @ beam-sdks-java-io-elasticsearch-tests-5 ---
[INFO] Building jar: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/9cecf502/beam/sdks/java/io/elasticsearch-tests/elasticsearch-tests-5/target/beam-sdks-java-io-elasticsearch-tests-5-2.5.0-SNAPSHOT-tests.jar>
[INFO]
[INFO] --- maven-shade-plugin:3.1.0:shade (bundle-and-repackage) @ beam-sdks-java-io-elasticsearch-tests-5 ---
[INFO] Replacing original artifact with shaded artifact.
[INFO] Replacing <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/9cecf502/beam/sdks/java/io/elasticsearch-tests/elasticsearch-tests-5/target/beam-sdks-java-io-elasticsearch-tests-5-2.5.0-SNAPSHOT.jar> with <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/9cecf502/beam/sdks/java/io/elasticsearch-tests/elasticsearch-tests-5/target/beam-sdks-java-io-elasticsearch-tests-5-2.5.0-SNAPSHOT-shaded.jar>
[INFO] Replacing original test artifact with shaded test artifact.
[INFO] Replacing <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/9cecf502/beam/sdks/java/io/elasticsearch-tests/elasticsearch-tests-5/target/beam-sdks-java-io-elasticsearch-tests-5-2.5.0-SNAPSHOT-tests.jar> with <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/9cecf502/beam/sdks/java/io/elasticsearch-tests/elasticsearch-tests-5/target/beam-sdks-java-io-elasticsearch-tests-5-2.5.0-SNAPSHOT-shaded-tests.jar>
[INFO]
[INFO] --- maven-dependency-plugin:3.0.2:analyze-only (default) @ beam-sdks-java-io-elasticsearch-tests-5 ---
[INFO] No dependency problems found
[INFO]
[INFO] --- maven-install-plugin:2.5.2:install (default-install) @ beam-sdks-java-io-elasticsearch-tests-5 ---
[INFO] Installing <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/9cecf502/beam/sdks/java/io/elasticsearch-tests/elasticsearch-tests-5/target/beam-sdks-java-io-elasticsearch-tests-5-2.5.0-SNAPSHOT.jar> to /home/jenkins/.m2/repository/org/apache/beam/beam-sdks-java-io-elasticsearch-tests-5/2.5.0-SNAPSHOT/beam-sdks-java-io-elasticsearch-tests-5-2.5.0-SNAPSHOT.jar
[INFO] Installing <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/9cecf502/beam/sdks/java/io/elasticsearch-tests/elasticsearch-tests-5/pom.xml> to /home/jenkins/.m2/repository/org/apache/beam/beam-sdks-java-io-elasticsearch-tests-5/2.5.0-SNAPSHOT/beam-sdks-java-io-elasticsearch-tests-5-2.5.0-SNAPSHOT.pom
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Beam :: Parent .............................. SUCCESS [ 6.360 s]
[INFO] Apache Beam :: SDKs :: Java :: Build Tools ......... SUCCESS [ 3.547 s]
[INFO] Apache Beam :: Model ............................... SUCCESS [ 0.097 s]
[INFO] Apache Beam :: Model :: Pipeline ................... SUCCESS [ 8.483 s]
[INFO] Apache Beam :: Model :: Job Management ............. SUCCESS [ 4.438 s]
[INFO] Apache Beam :: Model :: Fn Execution ............... SUCCESS [ 5.093 s]
[INFO] Apache Beam :: SDKs ................................ SUCCESS [ 0.114 s]
[INFO] Apache Beam :: SDKs :: Go .......................... SUCCESS [01:02 min]
[INFO] Apache Beam :: SDKs :: Go :: Container ............. SUCCESS [ 33.009 s]
[INFO] Apache Beam :: SDKs :: Java ........................ SUCCESS [ 0.268 s]
[INFO] Apache Beam :: SDKs :: Java :: Core ................ SUCCESS [ 48.645 s]
[INFO] Apache Beam :: SDKs :: Java :: Fn Execution ........ SUCCESS [ 4.956 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions .......... SUCCESS [ 0.097 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Google Cloud Platform Core SUCCESS [ 4.187 s]
[INFO] Apache Beam :: Runners ............................. SUCCESS [ 0.090 s]
[INFO] Apache Beam :: Runners :: Core Construction Java ... SUCCESS [ 5.751 s]
[INFO] Apache Beam :: Runners :: Core Java ................ SUCCESS [ 5.421 s]
[INFO] Apache Beam :: SDKs :: Java :: Harness ............. SUCCESS [ 11.237 s]
[INFO] Apache Beam :: SDKs :: Java :: Container ........... SUCCESS [ 9.480 s]
[INFO] Apache Beam :: SDKs :: Java :: IO .................. SUCCESS [ 0.048 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Amazon Web Services SUCCESS [ 3.264 s]
[INFO] Apache Beam :: Runners :: Local Java Core .......... SUCCESS [ 1.812 s]
[INFO] Apache Beam :: Runners :: Direct Java .............. SUCCESS [ 15.346 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: AMQP .......... SUCCESS [ 3.449 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Common ........ SUCCESS [ 2.834 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Cassandra ..... SUCCESS [ 4.102 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch . SUCCESS [ 3.311 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests SUCCESS [ 0.753 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: Common SUCCESS [ 0.936 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: 2.x SUCCESS [ 2.949 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: 5.x FAILURE [ 3.562 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: XML ........... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Protobuf SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Google Cloud Platform SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Dataflow .... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: File-based-io-tests SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop Common . SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop File System SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: JDBC .......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop Input Format SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: HBase ......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: HCatalog ...... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: JMS ........... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Kafka ......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Kinesis ....... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: MongoDB ....... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: MQTT .......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Redis ......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Solr .......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Tika .......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes .... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Starter SKIPPED
[INFO] Apache Beam :: Examples ............................ SKIPPED
[INFO] Apache Beam :: Examples :: Java .................... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Examples SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Jackson SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Join library SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Sketching SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Sorter SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: SQL ... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Nexmark ............. SKIPPED
[INFO] Apache Beam :: SDKs :: Python ...................... SKIPPED
[INFO] Apache Beam :: SDKs :: Python :: Container ......... SKIPPED
[INFO] Apache Beam :: Runners :: Java Fn Execution ........ SKIPPED
[INFO] Apache Beam :: Runners :: Java Local Artifact Service SKIPPED
[INFO] Apache Beam :: Runners :: Reference ................ SKIPPED
[INFO] Apache Beam :: Runners :: Reference :: Java ........ SKIPPED
[INFO] Apache Beam :: Runners :: Reference :: Job Orchestrator SKIPPED
[INFO] Apache Beam :: Runners :: Flink .................... SKIPPED
[INFO] Apache Beam :: Runners :: Gearpump ................. SKIPPED
[INFO] Apache Beam :: Runners :: Spark .................... SKIPPED
[INFO] Apache Beam :: Runners :: Apex ..................... SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform .... SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform :: GCE metadata provisioning SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform :: GCS artifact proxy SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Aggregated Javadoc .. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 04:20 min
[INFO] Finished at: 2018-04-23T18:05:40Z
[INFO] Final Memory: 250M/1387M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-install-plugin:2.5.2:install (default-install) on project beam-sdks-java-io-elasticsearch-tests-5: Failed to install metadata org.apache.beam:beam-sdks-java-io-elasticsearch-tests-5/maven-metadata.xml: Could not parse metadata /home/jenkins/.m2/repository/org/apache/beam/beam-sdks-java-io-elasticsearch-tests-5/maven-metadata-local.xml: in epilog non whitespace content is not allowed but got s (position: END_TAG seen ...</metadata>\ns... @12:2) -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :beam-sdks-java-io-elasticsearch-tests-5
STDERR:
2018-04-23 18:05:41,447 9cecf502 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-04-23 18:05:41,453 9cecf502 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 175, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 265, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 290, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/9cecf502/beam/sdks/python/target>
2018-04-23 18:05:41,454 9cecf502 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-04-23 18:05:41,455 9cecf502 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 780, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 175, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 265, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 290, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/9cecf502/beam/sdks/python/target>
2018-04-23 18:05:41,456 9cecf502 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-04-23 18:05:41,486 9cecf502 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-04-23 18:05:41,486 9cecf502 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/9cecf502/pkb.log>
2018-04-23 18:05:41,486 9cecf502 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/9cecf502/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user shoyer@google.com
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user boyuanz@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1183
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1183/display/redirect?page=changes>
Changes:
[alex] Improve CassandraIO write throughput by performing async queries
------------------------------------------
[...truncated 5.07 KB...]
Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17)) (0.2.5)
Collecting xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/42/a9/7e99652c6bc619d19d58cdd8c47560730eb5825d43a7e25db2e1d776ceb7/xmltodict-0.11.0-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/69/bc/230987c0dc22c763529330b2e669dbdba374d6a10c1f61232274184731be/ntlm_auth-1.1.0-py2.py3-none-any.whl
Collecting cryptography>=1.3 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/dd/c2/3a5bfefb25690725824ade71e6b65449f0a9f4b29702cce10560f786ebf6/cryptography-2.2.2-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Collecting cffi>=1.7; platform_python_implementation != "PyPy" (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/14/dd/3e7a1e1280e7d767bd3fa15791759c91ec19058ebe31217fe66f3e9a8c49/cffi-1.11.5-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Collecting asn1crypto>=0.21.0 (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Using cached https://files.pythonhosted.org/packages/ea/cd/35485615f45f30a510576f1a56d1e0a7ad7bd8ab5ed7cdc600ef7cd06222/asn1crypto-0.24.0-py2.py3-none-any.whl
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Collecting pycparser (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
cheetah 2.4.4 requires Markdown>=2.0.1, which is not installed.
Installing collected packages: absl-py, colorlog, blinker, futures, pint, numpy, contextlib2, xmltodict, ntlm-auth, pycparser, cffi, asn1crypto, cryptography, requests-ntlm, pywinrm
Successfully installed absl-py-0.2.0 asn1crypto-0.24.0 blinker-1.4 cffi-1.11.5 colorlog-2.6.0 contextlib2-0.5.5 cryptography-2.2.2 futures-3.2.0 ntlm-auth-1.1.0 numpy-1.13.3 pint-0.8.1 pycparser-2.18 pywinrm-0.3.0 requests-ntlm-1.1.0 xmltodict-0.11.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8862380498167712045.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.7)
Collecting dill==0.2.6 (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.5.0.dev0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/82/d8/3eab58811282ac7271a081ba5c0d4b875ce786ca68ce43e2a62ade32e9a8/oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (3.12)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0) (1.11.0)
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/0d/4d/4e5985d075d241d686a1663fa1f88b61d544658d08c1375c7c6aac32afc3/typing-3.6.4-py2-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.1.1 in ./.env/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0) (3.2.0)
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/1d/0c/64f84f91643f775fdb64c6c10f4a4f0d827f8b0d98a2ba2b4bb9dc2f8646/google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.5.0.dev0)
Collecting googledatastore==7.0.1 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/37/92/c74a643126d58505daec9addf872dfaffea3305981b90cc435f4b9213cdd/google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting proto-google-cloud-pubsub-v1==0.15.4 (from apache-beam==2.5.0.dev0)
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/76/67/6165c516ff6ceaa62eb61f11d8451e1b0acc4d3775e181630aba9652babb/google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Collecting pyhamcrest<2.0,>=1.9 (from apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.5.0.dev0) (1.1.6)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.18.4)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (0.6.2)
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Collecting pbr>=0.11 (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ba/f95e3ec83f93919b1437028e989cf3fa5ff4f5cae4a1f62255f71deddb5b/pbr-4.0.2-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e9/51/bcd96bf6231d4b2cc5e023c511bee86637ba375c44a6f9d1b4b7ad1ce4b9/pyasn1_modules-0.2.1-py2.py3-none-any.whl
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) (0.4.2)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) (39.0.1)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/14/3a/096c7ad18e102d4f219f5dd15951f9728ca5092a3385d2e8f79a7c1e1017/fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ef/dd/00e90bd1f6788f06ca5ea83a0ec8dd76350b38303bb8f09d2bf692eb1294/google_cloud_core-0.25.0-py2.py3-none-any.whl
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (1.22)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2.6)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (3.0.4)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) (2018.4.16)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/9f/ff/68860dec8b90424028ca54df8b9fc8cd1c47ef58710bd5e591ec8c619243/monotonic-1.4-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/e0/18/e282ecb28c7eb0ef5ea9a69fb3290658a9ab37bb69647f6ba7f0f78987c4/google_auth-1.4.1-py2.py3-none-any.whl
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/4f/b4/ff312fa42f91535c67567c1d08e972db0e7c548e9a63c6f3bcc5213b32fc/google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Using cached https://files.pythonhosted.org/packages/ac/e8/5492fd5ada0b05a1bc485bcb634b559acdec59383eef5c4203b5e22be296/cachetools-2.0.1-py2.py3-none-any.whl
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
cheetah 2.4.4 requires Markdown>=2.0.1, which is not installed.
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, pbr, mock, rsa, pyasn1-modules, oauth2client, pytz, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, cachetools, google-auth, google-auth-httplib2, google-cloud-core, future, ply, google-gax, grpc-google-iam-v1, proto-google-cloud-pubsub-v1, gapic-google-cloud-pubsub-v1, google-cloud-pubsub, google-cloud-bigquery, nose, pyhamcrest, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.11.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.4 nose-1.3.7 oauth2client-4.1.2 pbr-4.0.2 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2.post1 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pytz-2018.4 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7717623542518876600.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-04-23 12:01:05,212 9d798f4a MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/9d798f4a/pkb.log>
2018-04-23 12:01:05,213 9d798f4a MainThread INFO PerfKitBenchmarker version: v1.12.0-508-gd036cc1
2018-04-23 12:01:05,213 9d798f4a MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-04-23 12:01:05,545 9d798f4a MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-04-23 12:01:05,546 9d798f4a MainThread INFO Initializing the edw service decoder
2018-04-23 12:01:05,681 9d798f4a MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-04-23 12:01:05,684 9d798f4a MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-04-23 12:01:05,684 9d798f4a MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-04-23 12:01:18,838 9d798f4a MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-04-23 12:11:57,509 9d798f4a MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-04-23 12:11:57,526 9d798f4a MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/9d798f4a/beam/sdks/python/target/apache-beam-2.5.0.dev0.tar.gz>
2018-04-23 12:11:57,783 9d798f4a MainThread beam_integration_benchmark(1/1) INFO Ran: {python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/9d798f4a/beam/sdks/python/target/apache-beam-2.5.0.dev0.tar.gz}> ReturnCode:1
STDOUT:
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:397: UserWarning: Normalizing '2.5.0.dev' to '2.5.0.dev0'
normalized_version,
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: invalid command 'nosetests'
2018-04-23 12:11:57,784 9d798f4a MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-04-23 12:11:57,784 9d798f4a MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-04-23 12:11:57,786 9d798f4a MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 780, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-04-23 12:11:57,787 9d798f4a MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-04-23 12:11:57,797 9d798f4a MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-04-23 12:11:57,798 9d798f4a MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/9d798f4a/pkb.log>
2018-04-23 12:11:57,798 9d798f4a MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/9d798f4a/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user shoyer@google.com
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user boyuanz@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1182
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1182/display/redirect>
------------------------------------------
[...truncated 31.55 KB...]
[INFO] Compiling 1 source file to <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/7d1c2545/beam/sdks/java/build-tools/target/test-classes>
[INFO]
[INFO] --- maven-checkstyle-plugin:3.0.0:check (default) @ beam-sdks-java-build-tools ---
[INFO]
[INFO] --- maven-surefire-plugin:2.21.0:test (default-test) @ beam-sdks-java-build-tools ---
[INFO] Tests are skipped.
[INFO]
[INFO] --- build-helper-maven-plugin:3.0.0:regex-properties (render-artifact-id) @ beam-sdks-java-build-tools ---
[INFO]
[INFO] --- maven-jar-plugin:3.0.2:jar (default-jar) @ beam-sdks-java-build-tools ---
[INFO] Building jar: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/7d1c2545/beam/sdks/java/build-tools/target/beam-sdks-java-build-tools-2.5.0-SNAPSHOT.jar>
[INFO]
[INFO] --- maven-site-plugin:3.7:attach-descriptor (attach-descriptor) @ beam-sdks-java-build-tools ---
[INFO] Skipping because packaging 'jar' is not pom.
[INFO]
[INFO] --- maven-jar-plugin:3.0.2:test-jar (default-test-jar) @ beam-sdks-java-build-tools ---
[INFO] Building jar: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/7d1c2545/beam/sdks/java/build-tools/target/beam-sdks-java-build-tools-2.5.0-SNAPSHOT-tests.jar>
[INFO]
[INFO] --- maven-shade-plugin:3.1.0:shade (bundle-and-repackage) @ beam-sdks-java-build-tools ---
[INFO] Replacing original artifact with shaded artifact.
[INFO] Replacing <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/7d1c2545/beam/sdks/java/build-tools/target/beam-sdks-java-build-tools-2.5.0-SNAPSHOT.jar> with <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/7d1c2545/beam/sdks/java/build-tools/target/beam-sdks-java-build-tools-2.5.0-SNAPSHOT-shaded.jar>
[INFO] Replacing original test artifact with shaded test artifact.
[INFO] Replacing <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/7d1c2545/beam/sdks/java/build-tools/target/beam-sdks-java-build-tools-2.5.0-SNAPSHOT-tests.jar> with <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/7d1c2545/beam/sdks/java/build-tools/target/beam-sdks-java-build-tools-2.5.0-SNAPSHOT-shaded-tests.jar>
[INFO]
[INFO] --- maven-dependency-plugin:3.0.2:analyze-only (default) @ beam-sdks-java-build-tools ---
[INFO] No dependency problems found
[INFO]
[INFO] --- maven-install-plugin:2.5.2:install (default-install) @ beam-sdks-java-build-tools ---
[INFO] Installing <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/7d1c2545/beam/sdks/java/build-tools/target/beam-sdks-java-build-tools-2.5.0-SNAPSHOT.jar> to /home/jenkins/.m2/repository/org/apache/beam/beam-sdks-java-build-tools/2.5.0-SNAPSHOT/beam-sdks-java-build-tools-2.5.0-SNAPSHOT.jar
[INFO] Installing <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/7d1c2545/beam/sdks/java/build-tools/pom.xml> to /home/jenkins/.m2/repository/org/apache/beam/beam-sdks-java-build-tools/2.5.0-SNAPSHOT/beam-sdks-java-build-tools-2.5.0-SNAPSHOT.pom
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Beam :: Parent .............................. SUCCESS [ 3.165 s]
[INFO] Apache Beam :: SDKs :: Java :: Build Tools ......... FAILURE [ 2.650 s]
[INFO] Apache Beam :: Model ............................... SKIPPED
[INFO] Apache Beam :: Model :: Pipeline ................... SKIPPED
[INFO] Apache Beam :: Model :: Job Management ............. SKIPPED
[INFO] Apache Beam :: Model :: Fn Execution ............... SKIPPED
[INFO] Apache Beam :: SDKs ................................ SKIPPED
[INFO] Apache Beam :: SDKs :: Go .......................... SKIPPED
[INFO] Apache Beam :: SDKs :: Go :: Container ............. SKIPPED
[INFO] Apache Beam :: SDKs :: Java ........................ SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Core ................ SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Fn Execution ........ SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions .......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Google Cloud Platform Core SKIPPED
[INFO] Apache Beam :: Runners ............................. SKIPPED
[INFO] Apache Beam :: Runners :: Core Construction Java ... SKIPPED
[INFO] Apache Beam :: Runners :: Core Java ................ SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Harness ............. SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Container ........... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO .................. SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Amazon Web Services SKIPPED
[INFO] Apache Beam :: Runners :: Local Java Core .......... SKIPPED
[INFO] Apache Beam :: Runners :: Direct Java .............. SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: AMQP .......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Common ........ SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Cassandra ..... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch . SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: Common SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: 2.x SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: 5.x SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: XML ........... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Protobuf SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Google Cloud Platform SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Dataflow .... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: File-based-io-tests SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop Common . SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop File System SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: JDBC .......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop Input Format SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: HBase ......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: HCatalog ...... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: JMS ........... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Kafka ......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Kinesis ....... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: MongoDB ....... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: MQTT .......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Redis ......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Solr .......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Tika .......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes .... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Starter SKIPPED
[INFO] Apache Beam :: Examples ............................ SKIPPED
[INFO] Apache Beam :: Examples :: Java .................... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Examples SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Jackson SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Join library SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Sketching SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Sorter SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: SQL ... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Nexmark ............. SKIPPED
[INFO] Apache Beam :: SDKs :: Python ...................... SKIPPED
[INFO] Apache Beam :: SDKs :: Python :: Container ......... SKIPPED
[INFO] Apache Beam :: Runners :: Java Fn Execution ........ SKIPPED
[INFO] Apache Beam :: Runners :: Java Local Artifact Service SKIPPED
[INFO] Apache Beam :: Runners :: Reference ................ SKIPPED
[INFO] Apache Beam :: Runners :: Reference :: Java ........ SKIPPED
[INFO] Apache Beam :: Runners :: Reference :: Job Orchestrator SKIPPED
[INFO] Apache Beam :: Runners :: Flink .................... SKIPPED
[INFO] Apache Beam :: Runners :: Gearpump ................. SKIPPED
[INFO] Apache Beam :: Runners :: Spark .................... SKIPPED
[INFO] Apache Beam :: Runners :: Apex ..................... SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform .... SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform :: GCE metadata provisioning SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform :: GCS artifact proxy SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Aggregated Javadoc .. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 7.721 s
[INFO] Finished at: 2018-04-23T06:00:52Z
[INFO] Final Memory: 68M/703M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-install-plugin:2.5.2:install (default-install) on project beam-sdks-java-build-tools: Failed to install metadata org.apache.beam:beam-sdks-java-build-tools:2.5.0-SNAPSHOT/maven-metadata.xml: Could not parse metadata /home/jenkins/.m2/repository/org/apache/beam/beam-sdks-java-build-tools/2.5.0-SNAPSHOT/maven-metadata-local.xml: in epilog non whitespace content is not allowed but got n (position: END_TAG seen ...</metadata>\nn... @21:2) -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :beam-sdks-java-build-tools
STDERR:
2018-04-23 06:00:53,435 7d1c2545 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-04-23 06:00:53,440 7d1c2545 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 175, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 265, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 290, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/7d1c2545/beam/sdks/python/target>
2018-04-23 06:00:53,441 7d1c2545 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-04-23 06:00:53,443 7d1c2545 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 780, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 175, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 265, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 290, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/7d1c2545/beam/sdks/python/target>
2018-04-23 06:00:53,443 7d1c2545 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-04-23 06:00:53,451 7d1c2545 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-04-23 06:00:53,451 7d1c2545 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/7d1c2545/pkb.log>
2018-04-23 06:00:53,451 7d1c2545 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/7d1c2545/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user shoyer@google.com
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user boyuanz@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1181
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1181/display/redirect>
------------------------------------------
[...truncated 31.55 KB...]
[INFO] Compiling 1 source file to <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/167ba521/beam/sdks/java/build-tools/target/test-classes>
[INFO]
[INFO] --- maven-checkstyle-plugin:3.0.0:check (default) @ beam-sdks-java-build-tools ---
[INFO]
[INFO] --- maven-surefire-plugin:2.21.0:test (default-test) @ beam-sdks-java-build-tools ---
[INFO] Tests are skipped.
[INFO]
[INFO] --- build-helper-maven-plugin:3.0.0:regex-properties (render-artifact-id) @ beam-sdks-java-build-tools ---
[INFO]
[INFO] --- maven-jar-plugin:3.0.2:jar (default-jar) @ beam-sdks-java-build-tools ---
[INFO] Building jar: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/167ba521/beam/sdks/java/build-tools/target/beam-sdks-java-build-tools-2.5.0-SNAPSHOT.jar>
[INFO]
[INFO] --- maven-site-plugin:3.7:attach-descriptor (attach-descriptor) @ beam-sdks-java-build-tools ---
[INFO] Skipping because packaging 'jar' is not pom.
[INFO]
[INFO] --- maven-jar-plugin:3.0.2:test-jar (default-test-jar) @ beam-sdks-java-build-tools ---
[INFO] Building jar: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/167ba521/beam/sdks/java/build-tools/target/beam-sdks-java-build-tools-2.5.0-SNAPSHOT-tests.jar>
[INFO]
[INFO] --- maven-shade-plugin:3.1.0:shade (bundle-and-repackage) @ beam-sdks-java-build-tools ---
[INFO] Replacing original artifact with shaded artifact.
[INFO] Replacing <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/167ba521/beam/sdks/java/build-tools/target/beam-sdks-java-build-tools-2.5.0-SNAPSHOT.jar> with <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/167ba521/beam/sdks/java/build-tools/target/beam-sdks-java-build-tools-2.5.0-SNAPSHOT-shaded.jar>
[INFO] Replacing original test artifact with shaded test artifact.
[INFO] Replacing <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/167ba521/beam/sdks/java/build-tools/target/beam-sdks-java-build-tools-2.5.0-SNAPSHOT-tests.jar> with <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/167ba521/beam/sdks/java/build-tools/target/beam-sdks-java-build-tools-2.5.0-SNAPSHOT-shaded-tests.jar>
[INFO]
[INFO] --- maven-dependency-plugin:3.0.2:analyze-only (default) @ beam-sdks-java-build-tools ---
[INFO] No dependency problems found
[INFO]
[INFO] --- maven-install-plugin:2.5.2:install (default-install) @ beam-sdks-java-build-tools ---
[INFO] Installing <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/167ba521/beam/sdks/java/build-tools/target/beam-sdks-java-build-tools-2.5.0-SNAPSHOT.jar> to /home/jenkins/.m2/repository/org/apache/beam/beam-sdks-java-build-tools/2.5.0-SNAPSHOT/beam-sdks-java-build-tools-2.5.0-SNAPSHOT.jar
[INFO] Installing <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/167ba521/beam/sdks/java/build-tools/pom.xml> to /home/jenkins/.m2/repository/org/apache/beam/beam-sdks-java-build-tools/2.5.0-SNAPSHOT/beam-sdks-java-build-tools-2.5.0-SNAPSHOT.pom
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Beam :: Parent .............................. SUCCESS [ 2.984 s]
[INFO] Apache Beam :: SDKs :: Java :: Build Tools ......... FAILURE [ 2.697 s]
[INFO] Apache Beam :: Model ............................... SKIPPED
[INFO] Apache Beam :: Model :: Pipeline ................... SKIPPED
[INFO] Apache Beam :: Model :: Job Management ............. SKIPPED
[INFO] Apache Beam :: Model :: Fn Execution ............... SKIPPED
[INFO] Apache Beam :: SDKs ................................ SKIPPED
[INFO] Apache Beam :: SDKs :: Go .......................... SKIPPED
[INFO] Apache Beam :: SDKs :: Go :: Container ............. SKIPPED
[INFO] Apache Beam :: SDKs :: Java ........................ SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Core ................ SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Fn Execution ........ SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions .......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Google Cloud Platform Core SKIPPED
[INFO] Apache Beam :: Runners ............................. SKIPPED
[INFO] Apache Beam :: Runners :: Core Construction Java ... SKIPPED
[INFO] Apache Beam :: Runners :: Core Java ................ SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Harness ............. SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Container ........... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO .................. SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Amazon Web Services SKIPPED
[INFO] Apache Beam :: Runners :: Local Java Core .......... SKIPPED
[INFO] Apache Beam :: Runners :: Direct Java .............. SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: AMQP .......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Common ........ SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Cassandra ..... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch . SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: Common SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: 2.x SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: 5.x SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: XML ........... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Protobuf SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Google Cloud Platform SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Dataflow .... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: File-based-io-tests SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop Common . SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop File System SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: JDBC .......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop Input Format SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: HBase ......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: HCatalog ...... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: JMS ........... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Kafka ......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Kinesis ....... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: MongoDB ....... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: MQTT .......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Redis ......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Solr .......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Tika .......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes .... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Starter SKIPPED
[INFO] Apache Beam :: Examples ............................ SKIPPED
[INFO] Apache Beam :: Examples :: Java .................... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Examples SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Jackson SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Join library SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Sketching SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Sorter SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: SQL ... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Nexmark ............. SKIPPED
[INFO] Apache Beam :: SDKs :: Python ...................... SKIPPED
[INFO] Apache Beam :: SDKs :: Python :: Container ......... SKIPPED
[INFO] Apache Beam :: Runners :: Java Fn Execution ........ SKIPPED
[INFO] Apache Beam :: Runners :: Java Local Artifact Service SKIPPED
[INFO] Apache Beam :: Runners :: Reference ................ SKIPPED
[INFO] Apache Beam :: Runners :: Reference :: Java ........ SKIPPED
[INFO] Apache Beam :: Runners :: Reference :: Job Orchestrator SKIPPED
[INFO] Apache Beam :: Runners :: Flink .................... SKIPPED
[INFO] Apache Beam :: Runners :: Gearpump ................. SKIPPED
[INFO] Apache Beam :: Runners :: Spark .................... SKIPPED
[INFO] Apache Beam :: Runners :: Apex ..................... SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform .... SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform :: GCE metadata provisioning SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform :: GCS artifact proxy SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Aggregated Javadoc .. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 7.874 s
[INFO] Finished at: 2018-04-23T00:00:52Z
[INFO] Final Memory: 68M/693M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-install-plugin:2.5.2:install (default-install) on project beam-sdks-java-build-tools: Failed to install metadata org.apache.beam:beam-sdks-java-build-tools:2.5.0-SNAPSHOT/maven-metadata.xml: Could not parse metadata /home/jenkins/.m2/repository/org/apache/beam/beam-sdks-java-build-tools/2.5.0-SNAPSHOT/maven-metadata-local.xml: in epilog non whitespace content is not allowed but got n (position: END_TAG seen ...</metadata>\nn... @21:2) -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :beam-sdks-java-build-tools
STDERR:
2018-04-23 00:00:52,948 167ba521 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-04-23 00:00:52,954 167ba521 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 175, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 265, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 290, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/167ba521/beam/sdks/python/target>
2018-04-23 00:00:52,956 167ba521 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-04-23 00:00:52,958 167ba521 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 780, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 175, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 265, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 290, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/167ba521/beam/sdks/python/target>
2018-04-23 00:00:52,958 167ba521 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-04-23 00:00:52,992 167ba521 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-04-23 00:00:52,992 167ba521 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/167ba521/pkb.log>
2018-04-23 00:00:52,993 167ba521 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/167ba521/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user shoyer@google.com
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user boyuanz@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1180
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1180/display/redirect>
------------------------------------------
[...truncated 31.55 KB...]
[INFO] Compiling 1 source file to <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/5a05526b/beam/sdks/java/build-tools/target/test-classes>
[INFO]
[INFO] --- maven-checkstyle-plugin:3.0.0:check (default) @ beam-sdks-java-build-tools ---
[INFO]
[INFO] --- maven-surefire-plugin:2.21.0:test (default-test) @ beam-sdks-java-build-tools ---
[INFO] Tests are skipped.
[INFO]
[INFO] --- build-helper-maven-plugin:3.0.0:regex-properties (render-artifact-id) @ beam-sdks-java-build-tools ---
[INFO]
[INFO] --- maven-jar-plugin:3.0.2:jar (default-jar) @ beam-sdks-java-build-tools ---
[INFO] Building jar: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/5a05526b/beam/sdks/java/build-tools/target/beam-sdks-java-build-tools-2.5.0-SNAPSHOT.jar>
[INFO]
[INFO] --- maven-site-plugin:3.7:attach-descriptor (attach-descriptor) @ beam-sdks-java-build-tools ---
[INFO] Skipping because packaging 'jar' is not pom.
[INFO]
[INFO] --- maven-jar-plugin:3.0.2:test-jar (default-test-jar) @ beam-sdks-java-build-tools ---
[INFO] Building jar: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/5a05526b/beam/sdks/java/build-tools/target/beam-sdks-java-build-tools-2.5.0-SNAPSHOT-tests.jar>
[INFO]
[INFO] --- maven-shade-plugin:3.1.0:shade (bundle-and-repackage) @ beam-sdks-java-build-tools ---
[INFO] Replacing original artifact with shaded artifact.
[INFO] Replacing <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/5a05526b/beam/sdks/java/build-tools/target/beam-sdks-java-build-tools-2.5.0-SNAPSHOT.jar> with <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/5a05526b/beam/sdks/java/build-tools/target/beam-sdks-java-build-tools-2.5.0-SNAPSHOT-shaded.jar>
[INFO] Replacing original test artifact with shaded test artifact.
[INFO] Replacing <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/5a05526b/beam/sdks/java/build-tools/target/beam-sdks-java-build-tools-2.5.0-SNAPSHOT-tests.jar> with <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/5a05526b/beam/sdks/java/build-tools/target/beam-sdks-java-build-tools-2.5.0-SNAPSHOT-shaded-tests.jar>
[INFO]
[INFO] --- maven-dependency-plugin:3.0.2:analyze-only (default) @ beam-sdks-java-build-tools ---
[INFO] No dependency problems found
[INFO]
[INFO] --- maven-install-plugin:2.5.2:install (default-install) @ beam-sdks-java-build-tools ---
[INFO] Installing <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/5a05526b/beam/sdks/java/build-tools/target/beam-sdks-java-build-tools-2.5.0-SNAPSHOT.jar> to /home/jenkins/.m2/repository/org/apache/beam/beam-sdks-java-build-tools/2.5.0-SNAPSHOT/beam-sdks-java-build-tools-2.5.0-SNAPSHOT.jar
[INFO] Installing <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/5a05526b/beam/sdks/java/build-tools/pom.xml> to /home/jenkins/.m2/repository/org/apache/beam/beam-sdks-java-build-tools/2.5.0-SNAPSHOT/beam-sdks-java-build-tools-2.5.0-SNAPSHOT.pom
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Beam :: Parent .............................. SUCCESS [ 7.356 s]
[INFO] Apache Beam :: SDKs :: Java :: Build Tools ......... FAILURE [ 5.704 s]
[INFO] Apache Beam :: Model ............................... SKIPPED
[INFO] Apache Beam :: Model :: Pipeline ................... SKIPPED
[INFO] Apache Beam :: Model :: Job Management ............. SKIPPED
[INFO] Apache Beam :: Model :: Fn Execution ............... SKIPPED
[INFO] Apache Beam :: SDKs ................................ SKIPPED
[INFO] Apache Beam :: SDKs :: Go .......................... SKIPPED
[INFO] Apache Beam :: SDKs :: Go :: Container ............. SKIPPED
[INFO] Apache Beam :: SDKs :: Java ........................ SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Core ................ SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Fn Execution ........ SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions .......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Google Cloud Platform Core SKIPPED
[INFO] Apache Beam :: Runners ............................. SKIPPED
[INFO] Apache Beam :: Runners :: Core Construction Java ... SKIPPED
[INFO] Apache Beam :: Runners :: Core Java ................ SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Harness ............. SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Container ........... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO .................. SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Amazon Web Services SKIPPED
[INFO] Apache Beam :: Runners :: Local Java Core .......... SKIPPED
[INFO] Apache Beam :: Runners :: Direct Java .............. SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: AMQP .......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Common ........ SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Cassandra ..... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch . SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: Common SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: 2.x SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: 5.x SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: XML ........... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Protobuf SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Google Cloud Platform SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Dataflow .... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: File-based-io-tests SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop Common . SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop File System SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: JDBC .......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop Input Format SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: HBase ......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: HCatalog ...... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: JMS ........... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Kafka ......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Kinesis ....... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: MongoDB ....... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: MQTT .......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Redis ......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Solr .......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Tika .......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes .... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Starter SKIPPED
[INFO] Apache Beam :: Examples ............................ SKIPPED
[INFO] Apache Beam :: Examples :: Java .................... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Examples SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Jackson SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Join library SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Sketching SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Sorter SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: SQL ... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Nexmark ............. SKIPPED
[INFO] Apache Beam :: SDKs :: Python ...................... SKIPPED
[INFO] Apache Beam :: SDKs :: Python :: Container ......... SKIPPED
[INFO] Apache Beam :: Runners :: Java Fn Execution ........ SKIPPED
[INFO] Apache Beam :: Runners :: Java Local Artifact Service SKIPPED
[INFO] Apache Beam :: Runners :: Reference ................ SKIPPED
[INFO] Apache Beam :: Runners :: Reference :: Java ........ SKIPPED
[INFO] Apache Beam :: Runners :: Reference :: Job Orchestrator SKIPPED
[INFO] Apache Beam :: Runners :: Flink .................... SKIPPED
[INFO] Apache Beam :: Runners :: Gearpump ................. SKIPPED
[INFO] Apache Beam :: Runners :: Spark .................... SKIPPED
[INFO] Apache Beam :: Runners :: Apex ..................... SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform .... SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform :: GCE metadata provisioning SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform :: GCS artifact proxy SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Aggregated Javadoc .. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 17.699 s
[INFO] Finished at: 2018-04-22T18:01:26Z
[INFO] Final Memory: 68M/700M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-install-plugin:2.5.2:install (default-install) on project beam-sdks-java-build-tools: Failed to install metadata org.apache.beam:beam-sdks-java-build-tools:2.5.0-SNAPSHOT/maven-metadata.xml: Could not parse metadata /home/jenkins/.m2/repository/org/apache/beam/beam-sdks-java-build-tools/2.5.0-SNAPSHOT/maven-metadata-local.xml: in epilog non whitespace content is not allowed but got n (position: END_TAG seen ...</metadata>\nn... @21:2) -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :beam-sdks-java-build-tools
STDERR:
2018-04-22 18:01:27,039 5a05526b MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-04-22 18:01:27,047 5a05526b MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 175, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 265, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 290, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/5a05526b/beam/sdks/python/target>
2018-04-22 18:01:27,049 5a05526b MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-04-22 18:01:27,051 5a05526b MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 780, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 175, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 265, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 290, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/5a05526b/beam/sdks/python/target>
2018-04-22 18:01:27,051 5a05526b MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-04-22 18:01:27,076 5a05526b MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-04-22 18:01:27,076 5a05526b MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/5a05526b/pkb.log>
2018-04-22 18:01:27,076 5a05526b MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/5a05526b/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user shoyer@google.com
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user boyuanz@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1179
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1179/display/redirect>
------------------------------------------
[...truncated 31.55 KB...]
[INFO] Compiling 1 source file to <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/b00fb09a/beam/sdks/java/build-tools/target/test-classes>
[INFO]
[INFO] --- maven-checkstyle-plugin:3.0.0:check (default) @ beam-sdks-java-build-tools ---
[INFO]
[INFO] --- maven-surefire-plugin:2.21.0:test (default-test) @ beam-sdks-java-build-tools ---
[INFO] Tests are skipped.
[INFO]
[INFO] --- build-helper-maven-plugin:3.0.0:regex-properties (render-artifact-id) @ beam-sdks-java-build-tools ---
[INFO]
[INFO] --- maven-jar-plugin:3.0.2:jar (default-jar) @ beam-sdks-java-build-tools ---
[INFO] Building jar: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/b00fb09a/beam/sdks/java/build-tools/target/beam-sdks-java-build-tools-2.5.0-SNAPSHOT.jar>
[INFO]
[INFO] --- maven-site-plugin:3.7:attach-descriptor (attach-descriptor) @ beam-sdks-java-build-tools ---
[INFO] Skipping because packaging 'jar' is not pom.
[INFO]
[INFO] --- maven-jar-plugin:3.0.2:test-jar (default-test-jar) @ beam-sdks-java-build-tools ---
[INFO] Building jar: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/b00fb09a/beam/sdks/java/build-tools/target/beam-sdks-java-build-tools-2.5.0-SNAPSHOT-tests.jar>
[INFO]
[INFO] --- maven-shade-plugin:3.1.0:shade (bundle-and-repackage) @ beam-sdks-java-build-tools ---
[INFO] Replacing original artifact with shaded artifact.
[INFO] Replacing <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/b00fb09a/beam/sdks/java/build-tools/target/beam-sdks-java-build-tools-2.5.0-SNAPSHOT.jar> with <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/b00fb09a/beam/sdks/java/build-tools/target/beam-sdks-java-build-tools-2.5.0-SNAPSHOT-shaded.jar>
[INFO] Replacing original test artifact with shaded test artifact.
[INFO] Replacing <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/b00fb09a/beam/sdks/java/build-tools/target/beam-sdks-java-build-tools-2.5.0-SNAPSHOT-tests.jar> with <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/b00fb09a/beam/sdks/java/build-tools/target/beam-sdks-java-build-tools-2.5.0-SNAPSHOT-shaded-tests.jar>
[INFO]
[INFO] --- maven-dependency-plugin:3.0.2:analyze-only (default) @ beam-sdks-java-build-tools ---
[INFO] No dependency problems found
[INFO]
[INFO] --- maven-install-plugin:2.5.2:install (default-install) @ beam-sdks-java-build-tools ---
[INFO] Installing <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/b00fb09a/beam/sdks/java/build-tools/target/beam-sdks-java-build-tools-2.5.0-SNAPSHOT.jar> to /home/jenkins/.m2/repository/org/apache/beam/beam-sdks-java-build-tools/2.5.0-SNAPSHOT/beam-sdks-java-build-tools-2.5.0-SNAPSHOT.jar
[INFO] Installing <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/b00fb09a/beam/sdks/java/build-tools/pom.xml> to /home/jenkins/.m2/repository/org/apache/beam/beam-sdks-java-build-tools/2.5.0-SNAPSHOT/beam-sdks-java-build-tools-2.5.0-SNAPSHOT.pom
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Beam :: Parent .............................. SUCCESS [ 3.220 s]
[INFO] Apache Beam :: SDKs :: Java :: Build Tools ......... FAILURE [ 2.564 s]
[INFO] Apache Beam :: Model ............................... SKIPPED
[INFO] Apache Beam :: Model :: Pipeline ................... SKIPPED
[INFO] Apache Beam :: Model :: Job Management ............. SKIPPED
[INFO] Apache Beam :: Model :: Fn Execution ............... SKIPPED
[INFO] Apache Beam :: SDKs ................................ SKIPPED
[INFO] Apache Beam :: SDKs :: Go .......................... SKIPPED
[INFO] Apache Beam :: SDKs :: Go :: Container ............. SKIPPED
[INFO] Apache Beam :: SDKs :: Java ........................ SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Core ................ SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Fn Execution ........ SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions .......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Google Cloud Platform Core SKIPPED
[INFO] Apache Beam :: Runners ............................. SKIPPED
[INFO] Apache Beam :: Runners :: Core Construction Java ... SKIPPED
[INFO] Apache Beam :: Runners :: Core Java ................ SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Harness ............. SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Container ........... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO .................. SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Amazon Web Services SKIPPED
[INFO] Apache Beam :: Runners :: Local Java Core .......... SKIPPED
[INFO] Apache Beam :: Runners :: Direct Java .............. SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: AMQP .......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Common ........ SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Cassandra ..... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch . SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: Common SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: 2.x SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: 5.x SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: XML ........... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Protobuf SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Google Cloud Platform SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Dataflow .... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: File-based-io-tests SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop Common . SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop File System SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: JDBC .......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop Input Format SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: HBase ......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: HCatalog ...... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: JMS ........... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Kafka ......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Kinesis ....... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: MongoDB ....... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: MQTT .......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Redis ......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Solr .......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Tika .......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes .... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Starter SKIPPED
[INFO] Apache Beam :: Examples ............................ SKIPPED
[INFO] Apache Beam :: Examples :: Java .................... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Examples SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Jackson SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Join library SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Sketching SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Sorter SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: SQL ... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Nexmark ............. SKIPPED
[INFO] Apache Beam :: SDKs :: Python ...................... SKIPPED
[INFO] Apache Beam :: SDKs :: Python :: Container ......... SKIPPED
[INFO] Apache Beam :: Runners :: Java Fn Execution ........ SKIPPED
[INFO] Apache Beam :: Runners :: Java Local Artifact Service SKIPPED
[INFO] Apache Beam :: Runners :: Reference ................ SKIPPED
[INFO] Apache Beam :: Runners :: Reference :: Java ........ SKIPPED
[INFO] Apache Beam :: Runners :: Reference :: Job Orchestrator SKIPPED
[INFO] Apache Beam :: Runners :: Flink .................... SKIPPED
[INFO] Apache Beam :: Runners :: Gearpump ................. SKIPPED
[INFO] Apache Beam :: Runners :: Spark .................... SKIPPED
[INFO] Apache Beam :: Runners :: Apex ..................... SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform .... SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform :: GCE metadata provisioning SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform :: GCS artifact proxy SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Aggregated Javadoc .. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 8.023 s
[INFO] Finished at: 2018-04-22T12:00:53Z
[INFO] Final Memory: 67M/674M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-install-plugin:2.5.2:install (default-install) on project beam-sdks-java-build-tools: Failed to install metadata org.apache.beam:beam-sdks-java-build-tools:2.5.0-SNAPSHOT/maven-metadata.xml: Could not parse metadata /home/jenkins/.m2/repository/org/apache/beam/beam-sdks-java-build-tools/2.5.0-SNAPSHOT/maven-metadata-local.xml: in epilog non whitespace content is not allowed but got n (position: END_TAG seen ...</metadata>\nn... @21:2) -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :beam-sdks-java-build-tools
STDERR:
2018-04-22 12:00:53,755 b00fb09a MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-04-22 12:00:53,761 b00fb09a MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 175, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 265, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 290, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/b00fb09a/beam/sdks/python/target>
2018-04-22 12:00:53,763 b00fb09a MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-04-22 12:00:53,765 b00fb09a MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 780, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 175, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 265, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 290, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/b00fb09a/beam/sdks/python/target>
2018-04-22 12:00:53,765 b00fb09a MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-04-22 12:00:53,776 b00fb09a MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-04-22 12:00:53,777 b00fb09a MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/b00fb09a/pkb.log>
2018-04-22 12:00:53,777 b00fb09a MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/b00fb09a/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user shoyer@google.com
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user boyuanz@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1178
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1178/display/redirect>
------------------------------------------
[...truncated 31.55 KB...]
[INFO] Compiling 1 source file to <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/2af71720/beam/sdks/java/build-tools/target/test-classes>
[INFO]
[INFO] --- maven-checkstyle-plugin:3.0.0:check (default) @ beam-sdks-java-build-tools ---
[INFO]
[INFO] --- maven-surefire-plugin:2.21.0:test (default-test) @ beam-sdks-java-build-tools ---
[INFO] Tests are skipped.
[INFO]
[INFO] --- build-helper-maven-plugin:3.0.0:regex-properties (render-artifact-id) @ beam-sdks-java-build-tools ---
[INFO]
[INFO] --- maven-jar-plugin:3.0.2:jar (default-jar) @ beam-sdks-java-build-tools ---
[INFO] Building jar: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/2af71720/beam/sdks/java/build-tools/target/beam-sdks-java-build-tools-2.5.0-SNAPSHOT.jar>
[INFO]
[INFO] --- maven-site-plugin:3.7:attach-descriptor (attach-descriptor) @ beam-sdks-java-build-tools ---
[INFO] Skipping because packaging 'jar' is not pom.
[INFO]
[INFO] --- maven-jar-plugin:3.0.2:test-jar (default-test-jar) @ beam-sdks-java-build-tools ---
[INFO] Building jar: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/2af71720/beam/sdks/java/build-tools/target/beam-sdks-java-build-tools-2.5.0-SNAPSHOT-tests.jar>
[INFO]
[INFO] --- maven-shade-plugin:3.1.0:shade (bundle-and-repackage) @ beam-sdks-java-build-tools ---
[INFO] Replacing original artifact with shaded artifact.
[INFO] Replacing <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/2af71720/beam/sdks/java/build-tools/target/beam-sdks-java-build-tools-2.5.0-SNAPSHOT.jar> with <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/2af71720/beam/sdks/java/build-tools/target/beam-sdks-java-build-tools-2.5.0-SNAPSHOT-shaded.jar>
[INFO] Replacing original test artifact with shaded test artifact.
[INFO] Replacing <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/2af71720/beam/sdks/java/build-tools/target/beam-sdks-java-build-tools-2.5.0-SNAPSHOT-tests.jar> with <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/2af71720/beam/sdks/java/build-tools/target/beam-sdks-java-build-tools-2.5.0-SNAPSHOT-shaded-tests.jar>
[INFO]
[INFO] --- maven-dependency-plugin:3.0.2:analyze-only (default) @ beam-sdks-java-build-tools ---
[INFO] No dependency problems found
[INFO]
[INFO] --- maven-install-plugin:2.5.2:install (default-install) @ beam-sdks-java-build-tools ---
[INFO] Installing <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/2af71720/beam/sdks/java/build-tools/target/beam-sdks-java-build-tools-2.5.0-SNAPSHOT.jar> to /home/jenkins/.m2/repository/org/apache/beam/beam-sdks-java-build-tools/2.5.0-SNAPSHOT/beam-sdks-java-build-tools-2.5.0-SNAPSHOT.jar
[INFO] Installing <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/2af71720/beam/sdks/java/build-tools/pom.xml> to /home/jenkins/.m2/repository/org/apache/beam/beam-sdks-java-build-tools/2.5.0-SNAPSHOT/beam-sdks-java-build-tools-2.5.0-SNAPSHOT.pom
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Beam :: Parent .............................. SUCCESS [ 3.089 s]
[INFO] Apache Beam :: SDKs :: Java :: Build Tools ......... FAILURE [ 2.819 s]
[INFO] Apache Beam :: Model ............................... SKIPPED
[INFO] Apache Beam :: Model :: Pipeline ................... SKIPPED
[INFO] Apache Beam :: Model :: Job Management ............. SKIPPED
[INFO] Apache Beam :: Model :: Fn Execution ............... SKIPPED
[INFO] Apache Beam :: SDKs ................................ SKIPPED
[INFO] Apache Beam :: SDKs :: Go .......................... SKIPPED
[INFO] Apache Beam :: SDKs :: Go :: Container ............. SKIPPED
[INFO] Apache Beam :: SDKs :: Java ........................ SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Core ................ SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Fn Execution ........ SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions .......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Google Cloud Platform Core SKIPPED
[INFO] Apache Beam :: Runners ............................. SKIPPED
[INFO] Apache Beam :: Runners :: Core Construction Java ... SKIPPED
[INFO] Apache Beam :: Runners :: Core Java ................ SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Harness ............. SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Container ........... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO .................. SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Amazon Web Services SKIPPED
[INFO] Apache Beam :: Runners :: Local Java Core .......... SKIPPED
[INFO] Apache Beam :: Runners :: Direct Java .............. SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: AMQP .......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Common ........ SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Cassandra ..... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch . SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: Common SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: 2.x SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: 5.x SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: XML ........... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Protobuf SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Google Cloud Platform SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Dataflow .... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: File-based-io-tests SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop Common . SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop File System SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: JDBC .......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop Input Format SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: HBase ......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: HCatalog ...... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: JMS ........... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Kafka ......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Kinesis ....... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: MongoDB ....... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: MQTT .......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Redis ......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Solr .......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Tika .......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes .... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Starter SKIPPED
[INFO] Apache Beam :: Examples ............................ SKIPPED
[INFO] Apache Beam :: Examples :: Java .................... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Examples SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Jackson SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Join library SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Sketching SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Sorter SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: SQL ... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Nexmark ............. SKIPPED
[INFO] Apache Beam :: SDKs :: Python ...................... SKIPPED
[INFO] Apache Beam :: SDKs :: Python :: Container ......... SKIPPED
[INFO] Apache Beam :: Runners :: Java Fn Execution ........ SKIPPED
[INFO] Apache Beam :: Runners :: Java Local Artifact Service SKIPPED
[INFO] Apache Beam :: Runners :: Reference ................ SKIPPED
[INFO] Apache Beam :: Runners :: Reference :: Java ........ SKIPPED
[INFO] Apache Beam :: Runners :: Reference :: Job Orchestrator SKIPPED
[INFO] Apache Beam :: Runners :: Flink .................... SKIPPED
[INFO] Apache Beam :: Runners :: Gearpump ................. SKIPPED
[INFO] Apache Beam :: Runners :: Spark .................... SKIPPED
[INFO] Apache Beam :: Runners :: Apex ..................... SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform .... SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform :: GCE metadata provisioning SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform :: GCS artifact proxy SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Aggregated Javadoc .. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 8.246 s
[INFO] Finished at: 2018-04-22T06:00:50Z
[INFO] Final Memory: 67M/673M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-install-plugin:2.5.2:install (default-install) on project beam-sdks-java-build-tools: Failed to install metadata org.apache.beam:beam-sdks-java-build-tools:2.5.0-SNAPSHOT/maven-metadata.xml: Could not parse metadata /home/jenkins/.m2/repository/org/apache/beam/beam-sdks-java-build-tools/2.5.0-SNAPSHOT/maven-metadata-local.xml: in epilog non whitespace content is not allowed but got n (position: END_TAG seen ...</metadata>\nn... @21:2) -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :beam-sdks-java-build-tools
STDERR:
2018-04-22 06:00:51,124 2af71720 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-04-22 06:00:51,130 2af71720 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 175, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 265, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 290, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/2af71720/beam/sdks/python/target>
2018-04-22 06:00:51,131 2af71720 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-04-22 06:00:51,132 2af71720 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 780, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 175, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 265, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 290, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/2af71720/beam/sdks/python/target>
2018-04-22 06:00:51,132 2af71720 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-04-22 06:00:51,141 2af71720 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-04-22 06:00:51,142 2af71720 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/2af71720/pkb.log>
2018-04-22 06:00:51,142 2af71720 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/2af71720/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user shoyer@google.com
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user boyuanz@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1177
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1177/display/redirect>
------------------------------------------
[...truncated 31.55 KB...]
[INFO] Compiling 1 source file to <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/8b4bb4c7/beam/sdks/java/build-tools/target/test-classes>
[INFO]
[INFO] --- maven-checkstyle-plugin:3.0.0:check (default) @ beam-sdks-java-build-tools ---
[INFO]
[INFO] --- maven-surefire-plugin:2.21.0:test (default-test) @ beam-sdks-java-build-tools ---
[INFO] Tests are skipped.
[INFO]
[INFO] --- build-helper-maven-plugin:3.0.0:regex-properties (render-artifact-id) @ beam-sdks-java-build-tools ---
[INFO]
[INFO] --- maven-jar-plugin:3.0.2:jar (default-jar) @ beam-sdks-java-build-tools ---
[INFO] Building jar: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/8b4bb4c7/beam/sdks/java/build-tools/target/beam-sdks-java-build-tools-2.5.0-SNAPSHOT.jar>
[INFO]
[INFO] --- maven-site-plugin:3.7:attach-descriptor (attach-descriptor) @ beam-sdks-java-build-tools ---
[INFO] Skipping because packaging 'jar' is not pom.
[INFO]
[INFO] --- maven-jar-plugin:3.0.2:test-jar (default-test-jar) @ beam-sdks-java-build-tools ---
[INFO] Building jar: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/8b4bb4c7/beam/sdks/java/build-tools/target/beam-sdks-java-build-tools-2.5.0-SNAPSHOT-tests.jar>
[INFO]
[INFO] --- maven-shade-plugin:3.1.0:shade (bundle-and-repackage) @ beam-sdks-java-build-tools ---
[INFO] Replacing original artifact with shaded artifact.
[INFO] Replacing <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/8b4bb4c7/beam/sdks/java/build-tools/target/beam-sdks-java-build-tools-2.5.0-SNAPSHOT.jar> with <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/8b4bb4c7/beam/sdks/java/build-tools/target/beam-sdks-java-build-tools-2.5.0-SNAPSHOT-shaded.jar>
[INFO] Replacing original test artifact with shaded test artifact.
[INFO] Replacing <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/8b4bb4c7/beam/sdks/java/build-tools/target/beam-sdks-java-build-tools-2.5.0-SNAPSHOT-tests.jar> with <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/8b4bb4c7/beam/sdks/java/build-tools/target/beam-sdks-java-build-tools-2.5.0-SNAPSHOT-shaded-tests.jar>
[INFO]
[INFO] --- maven-dependency-plugin:3.0.2:analyze-only (default) @ beam-sdks-java-build-tools ---
[INFO] No dependency problems found
[INFO]
[INFO] --- maven-install-plugin:2.5.2:install (default-install) @ beam-sdks-java-build-tools ---
[INFO] Installing <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/8b4bb4c7/beam/sdks/java/build-tools/target/beam-sdks-java-build-tools-2.5.0-SNAPSHOT.jar> to /home/jenkins/.m2/repository/org/apache/beam/beam-sdks-java-build-tools/2.5.0-SNAPSHOT/beam-sdks-java-build-tools-2.5.0-SNAPSHOT.jar
[INFO] Installing <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/8b4bb4c7/beam/sdks/java/build-tools/pom.xml> to /home/jenkins/.m2/repository/org/apache/beam/beam-sdks-java-build-tools/2.5.0-SNAPSHOT/beam-sdks-java-build-tools-2.5.0-SNAPSHOT.pom
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Beam :: Parent .............................. SUCCESS [ 6.123 s]
[INFO] Apache Beam :: SDKs :: Java :: Build Tools ......... FAILURE [ 4.996 s]
[INFO] Apache Beam :: Model ............................... SKIPPED
[INFO] Apache Beam :: Model :: Pipeline ................... SKIPPED
[INFO] Apache Beam :: Model :: Job Management ............. SKIPPED
[INFO] Apache Beam :: Model :: Fn Execution ............... SKIPPED
[INFO] Apache Beam :: SDKs ................................ SKIPPED
[INFO] Apache Beam :: SDKs :: Go .......................... SKIPPED
[INFO] Apache Beam :: SDKs :: Go :: Container ............. SKIPPED
[INFO] Apache Beam :: SDKs :: Java ........................ SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Core ................ SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Fn Execution ........ SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions .......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Google Cloud Platform Core SKIPPED
[INFO] Apache Beam :: Runners ............................. SKIPPED
[INFO] Apache Beam :: Runners :: Core Construction Java ... SKIPPED
[INFO] Apache Beam :: Runners :: Core Java ................ SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Harness ............. SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Container ........... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO .................. SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Amazon Web Services SKIPPED
[INFO] Apache Beam :: Runners :: Local Java Core .......... SKIPPED
[INFO] Apache Beam :: Runners :: Direct Java .............. SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: AMQP .......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Common ........ SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Cassandra ..... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch . SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: Common SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: 2.x SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: 5.x SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: XML ........... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Protobuf SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Google Cloud Platform SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Dataflow .... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: File-based-io-tests SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop Common . SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop File System SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: JDBC .......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop Input Format SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: HBase ......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: HCatalog ...... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: JMS ........... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Kafka ......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Kinesis ....... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: MongoDB ....... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: MQTT .......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Redis ......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Solr .......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Tika .......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes .... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Starter SKIPPED
[INFO] Apache Beam :: Examples ............................ SKIPPED
[INFO] Apache Beam :: Examples :: Java .................... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Examples SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Jackson SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Join library SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Sketching SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Sorter SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: SQL ... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Nexmark ............. SKIPPED
[INFO] Apache Beam :: SDKs :: Python ...................... SKIPPED
[INFO] Apache Beam :: SDKs :: Python :: Container ......... SKIPPED
[INFO] Apache Beam :: Runners :: Java Fn Execution ........ SKIPPED
[INFO] Apache Beam :: Runners :: Java Local Artifact Service SKIPPED
[INFO] Apache Beam :: Runners :: Reference ................ SKIPPED
[INFO] Apache Beam :: Runners :: Reference :: Java ........ SKIPPED
[INFO] Apache Beam :: Runners :: Reference :: Job Orchestrator SKIPPED
[INFO] Apache Beam :: Runners :: Flink .................... SKIPPED
[INFO] Apache Beam :: Runners :: Gearpump ................. SKIPPED
[INFO] Apache Beam :: Runners :: Spark .................... SKIPPED
[INFO] Apache Beam :: Runners :: Apex ..................... SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform .... SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform :: GCE metadata provisioning SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform :: GCS artifact proxy SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Aggregated Javadoc .. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 15.396 s
[INFO] Finished at: 2018-04-22T00:01:03Z
[INFO] Final Memory: 68M/731M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-install-plugin:2.5.2:install (default-install) on project beam-sdks-java-build-tools: Failed to install metadata org.apache.beam:beam-sdks-java-build-tools:2.5.0-SNAPSHOT/maven-metadata.xml: Could not parse metadata /home/jenkins/.m2/repository/org/apache/beam/beam-sdks-java-build-tools/2.5.0-SNAPSHOT/maven-metadata-local.xml: in epilog non whitespace content is not allowed but got n (position: END_TAG seen ...</metadata>\nn... @21:2) -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :beam-sdks-java-build-tools
STDERR:
2018-04-22 00:01:04,082 8b4bb4c7 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-04-22 00:01:04,087 8b4bb4c7 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 175, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 265, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 290, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/8b4bb4c7/beam/sdks/python/target>
2018-04-22 00:01:04,088 8b4bb4c7 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-04-22 00:01:04,090 8b4bb4c7 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 780, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 175, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 265, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 290, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/8b4bb4c7/beam/sdks/python/target>
2018-04-22 00:01:04,090 8b4bb4c7 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-04-22 00:01:04,120 8b4bb4c7 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-04-22 00:01:04,120 8b4bb4c7 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/8b4bb4c7/pkb.log>
2018-04-22 00:01:04,120 8b4bb4c7 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/8b4bb4c7/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user shoyer@google.com
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user boyuanz@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #1176
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1176/display/redirect>
------------------------------------------
[...truncated 31.55 KB...]
[INFO] Compiling 1 source file to <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/97e2e943/beam/sdks/java/build-tools/target/test-classes>
[INFO]
[INFO] --- maven-checkstyle-plugin:3.0.0:check (default) @ beam-sdks-java-build-tools ---
[INFO]
[INFO] --- maven-surefire-plugin:2.21.0:test (default-test) @ beam-sdks-java-build-tools ---
[INFO] Tests are skipped.
[INFO]
[INFO] --- build-helper-maven-plugin:3.0.0:regex-properties (render-artifact-id) @ beam-sdks-java-build-tools ---
[INFO]
[INFO] --- maven-jar-plugin:3.0.2:jar (default-jar) @ beam-sdks-java-build-tools ---
[INFO] Building jar: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/97e2e943/beam/sdks/java/build-tools/target/beam-sdks-java-build-tools-2.5.0-SNAPSHOT.jar>
[INFO]
[INFO] --- maven-site-plugin:3.7:attach-descriptor (attach-descriptor) @ beam-sdks-java-build-tools ---
[INFO] Skipping because packaging 'jar' is not pom.
[INFO]
[INFO] --- maven-jar-plugin:3.0.2:test-jar (default-test-jar) @ beam-sdks-java-build-tools ---
[INFO] Building jar: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/97e2e943/beam/sdks/java/build-tools/target/beam-sdks-java-build-tools-2.5.0-SNAPSHOT-tests.jar>
[INFO]
[INFO] --- maven-shade-plugin:3.1.0:shade (bundle-and-repackage) @ beam-sdks-java-build-tools ---
[INFO] Replacing original artifact with shaded artifact.
[INFO] Replacing <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/97e2e943/beam/sdks/java/build-tools/target/beam-sdks-java-build-tools-2.5.0-SNAPSHOT.jar> with <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/97e2e943/beam/sdks/java/build-tools/target/beam-sdks-java-build-tools-2.5.0-SNAPSHOT-shaded.jar>
[INFO] Replacing original test artifact with shaded test artifact.
[INFO] Replacing <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/97e2e943/beam/sdks/java/build-tools/target/beam-sdks-java-build-tools-2.5.0-SNAPSHOT-tests.jar> with <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/97e2e943/beam/sdks/java/build-tools/target/beam-sdks-java-build-tools-2.5.0-SNAPSHOT-shaded-tests.jar>
[INFO]
[INFO] --- maven-dependency-plugin:3.0.2:analyze-only (default) @ beam-sdks-java-build-tools ---
[INFO] No dependency problems found
[INFO]
[INFO] --- maven-install-plugin:2.5.2:install (default-install) @ beam-sdks-java-build-tools ---
[INFO] Installing <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/97e2e943/beam/sdks/java/build-tools/target/beam-sdks-java-build-tools-2.5.0-SNAPSHOT.jar> to /home/jenkins/.m2/repository/org/apache/beam/beam-sdks-java-build-tools/2.5.0-SNAPSHOT/beam-sdks-java-build-tools-2.5.0-SNAPSHOT.jar
[INFO] Installing <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/97e2e943/beam/sdks/java/build-tools/pom.xml> to /home/jenkins/.m2/repository/org/apache/beam/beam-sdks-java-build-tools/2.5.0-SNAPSHOT/beam-sdks-java-build-tools-2.5.0-SNAPSHOT.pom
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Beam :: Parent .............................. SUCCESS [ 2.933 s]
[INFO] Apache Beam :: SDKs :: Java :: Build Tools ......... FAILURE [ 2.716 s]
[INFO] Apache Beam :: Model ............................... SKIPPED
[INFO] Apache Beam :: Model :: Pipeline ................... SKIPPED
[INFO] Apache Beam :: Model :: Job Management ............. SKIPPED
[INFO] Apache Beam :: Model :: Fn Execution ............... SKIPPED
[INFO] Apache Beam :: SDKs ................................ SKIPPED
[INFO] Apache Beam :: SDKs :: Go .......................... SKIPPED
[INFO] Apache Beam :: SDKs :: Go :: Container ............. SKIPPED
[INFO] Apache Beam :: SDKs :: Java ........................ SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Core ................ SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Fn Execution ........ SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions .......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Google Cloud Platform Core SKIPPED
[INFO] Apache Beam :: Runners ............................. SKIPPED
[INFO] Apache Beam :: Runners :: Core Construction Java ... SKIPPED
[INFO] Apache Beam :: Runners :: Core Java ................ SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Harness ............. SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Container ........... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO .................. SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Amazon Web Services SKIPPED
[INFO] Apache Beam :: Runners :: Local Java Core .......... SKIPPED
[INFO] Apache Beam :: Runners :: Direct Java .............. SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: AMQP .......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Common ........ SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Cassandra ..... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch . SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: Common SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: 2.x SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: 5.x SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: XML ........... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Protobuf SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Google Cloud Platform SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Dataflow .... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: File-based-io-tests SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop Common . SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop File System SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: JDBC .......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop Input Format SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: HBase ......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: HCatalog ...... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: JMS ........... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Kafka ......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Kinesis ....... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: MongoDB ....... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: MQTT .......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Redis ......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Solr .......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Tika .......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes .... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Starter SKIPPED
[INFO] Apache Beam :: Examples ............................ SKIPPED
[INFO] Apache Beam :: Examples :: Java .................... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Examples SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Jackson SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Join library SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Sketching SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Sorter SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: SQL ... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Nexmark ............. SKIPPED
[INFO] Apache Beam :: SDKs :: Python ...................... SKIPPED
[INFO] Apache Beam :: SDKs :: Python :: Container ......... SKIPPED
[INFO] Apache Beam :: Runners :: Java Fn Execution ........ SKIPPED
[INFO] Apache Beam :: Runners :: Java Local Artifact Service SKIPPED
[INFO] Apache Beam :: Runners :: Reference ................ SKIPPED
[INFO] Apache Beam :: Runners :: Reference :: Java ........ SKIPPED
[INFO] Apache Beam :: Runners :: Reference :: Job Orchestrator SKIPPED
[INFO] Apache Beam :: Runners :: Flink .................... SKIPPED
[INFO] Apache Beam :: Runners :: Gearpump ................. SKIPPED
[INFO] Apache Beam :: Runners :: Spark .................... SKIPPED
[INFO] Apache Beam :: Runners :: Apex ..................... SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform .... SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform :: GCE metadata provisioning SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform :: GCS artifact proxy SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Aggregated Javadoc .. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 7.583 s
[INFO] Finished at: 2018-04-21T18:01:05Z
[INFO] Final Memory: 67M/702M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-install-plugin:2.5.2:install (default-install) on project beam-sdks-java-build-tools: Failed to install metadata org.apache.beam:beam-sdks-java-build-tools:2.5.0-SNAPSHOT/maven-metadata.xml: Could not parse metadata /home/jenkins/.m2/repository/org/apache/beam/beam-sdks-java-build-tools/2.5.0-SNAPSHOT/maven-metadata-local.xml: in epilog non whitespace content is not allowed but got n (position: END_TAG seen ...</metadata>\nn... @21:2) -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :beam-sdks-java-build-tools
STDERR:
2018-04-21 18:01:05,837 97e2e943 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-04-21 18:01:05,842 97e2e943 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 175, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 265, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 290, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/97e2e943/beam/sdks/python/target>
2018-04-21 18:01:05,843 97e2e943 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-04-21 18:01:05,844 97e2e943 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 780, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 646, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 175, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 265, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 290, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/97e2e943/beam/sdks/python/target>
2018-04-21 18:01:05,845 97e2e943 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-04-21 18:01:05,886 97e2e943 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-04-21 18:01:05,887 97e2e943 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/97e2e943/pkb.log>
2018-04-21 18:01:05,887 97e2e943 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/97e2e943/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user dawid@getindata.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user samuel.waggoner@healthsparq.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user grzegorz.kolakowski@getindata.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user mariand@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user arnaudfournier921@gmail.com
Not sending mail to unregistered user bbassingthwaite@vendasta.com
Not sending mail to unregistered user 1147435+chuanyu@users.noreply.github.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user rezarokni@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user luke_zhu@brown.edu
Not sending mail to unregistered user swegner2@gmail.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user dcavazos@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user willy@bounceexchange.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user djiofack007@gmail.com
Not sending mail to unregistered user alan.myrvold@comcast.net
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user shoyer@google.com
Not sending mail to unregistered user 7542791+rezarokni@users.noreply.github.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user andreas.ehrencrona@velik.it
Not sending mail to unregistered user abhyankar@gmail.com
Not sending mail to unregistered user aaltay@gmail.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user 1028332163@qq.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user arnaudfournier021@gmail.com
Not sending mail to unregistered user szewinho@gmail.com
Not sending mail to unregistered user yifanzou@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user wtanaka@yahoo.com
Not sending mail to unregistered user boyuanz@google.com