You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2019/07/29 17:12:48 UTC
Build failed in Jenkins: beam_PostCommit_Python35 #78
See <https://builds.apache.org/job/beam_PostCommit_Python35/78/display/redirect?page=changes>
Changes:
[mxm] [BEAM-7366] Restore Spotless check for Flink Runner
------------------------------------------
[...truncated 537.52 KB...]
createTime: '2019-07-29T17:03:31.274314Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2019-07-29_10_03_30-8577663469496961280'
location: 'us-central1'
name: 'beamapp-jenkins-0729162353-279962'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2019-07-29T17:03:31.274314Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
root: INFO: Created job with id: [2019-07-29_10_03_30-8577663469496961280]
root: INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-29_10_03_30-8577663469496961280?project=apache-beam-testing
root: INFO: Job 2019-07-29_10_03_30-8577663469496961280 is in state JOB_STATE_RUNNING
root: INFO: 2019-07-29T17:03:30.189Z: JOB_MESSAGE_DETAILED: Autoscaling is enabled for job 2019-07-29_10_03_30-8577663469496961280. The number of workers will be between 1 and 1000.
root: INFO: 2019-07-29T17:03:30.256Z: JOB_MESSAGE_DETAILED: Autoscaling was automatically enabled for job 2019-07-29_10_03_30-8577663469496961280.
root: INFO: 2019-07-29T17:03:33.344Z: JOB_MESSAGE_DETAILED: Checking permissions granted to controller Service Account.
root: INFO: 2019-07-29T17:03:34.112Z: JOB_MESSAGE_BASIC: Worker configuration: n1-standard-1 in us-central1-b.
root: INFO: 2019-07-29T17:03:34.755Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
root: INFO: 2019-07-29T17:03:34.806Z: JOB_MESSAGE_DEBUG: Combiner lifting skipped for step assert_that/Group/GroupByKey: GroupByKey not followed by a combiner.
root: INFO: 2019-07-29T17:03:34.841Z: JOB_MESSAGE_DEBUG: Combiner lifting skipped for step read from datastore/Reshuffle/ReshufflePerKey/GroupByKey: GroupByKey not followed by a combiner.
root: INFO: 2019-07-29T17:03:34.888Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
root: INFO: 2019-07-29T17:03:34.933Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
root: INFO: 2019-07-29T17:03:35.109Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
root: INFO: 2019-07-29T17:03:35.179Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
root: INFO: 2019-07-29T17:03:35.217Z: JOB_MESSAGE_DETAILED: Fusing consumer read from datastore/SplitQuery into read from datastore/UserQuery/Read
root: INFO: 2019-07-29T17:03:35.264Z: JOB_MESSAGE_DETAILED: Fusing consumer read from datastore/Reshuffle/ReshufflePerKey/Map(reify_timestamps) into read from datastore/Reshuffle/AddRandomKeys
root: INFO: 2019-07-29T17:03:35.310Z: JOB_MESSAGE_DETAILED: Fusing consumer read from datastore/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps) into read from datastore/Reshuffle/ReshufflePerKey/GroupByKey/GroupByWindow
root: INFO: 2019-07-29T17:03:35.361Z: JOB_MESSAGE_DETAILED: Fusing consumer Globally/CombineGlobally(CountCombineFn)/CombinePerKey/Combine/Extract into Globally/CombineGlobally(CountCombineFn)/CombinePerKey/Combine
root: INFO: 2019-07-29T17:03:35.406Z: JOB_MESSAGE_DETAILED: Fusing consumer Globally/CombineGlobally(CountCombineFn)/CombinePerKey/Combine into Globally/CombineGlobally(CountCombineFn)/CombinePerKey/GroupByKey/Read
root: INFO: 2019-07-29T17:03:35.456Z: JOB_MESSAGE_DETAILED: Fusing consumer read from datastore/Reshuffle/ReshufflePerKey/GroupByKey/GroupByWindow into read from datastore/Reshuffle/ReshufflePerKey/GroupByKey/Read
root: INFO: 2019-07-29T17:03:35.504Z: JOB_MESSAGE_DETAILED: Fusing consumer Globally/CombineGlobally(CountCombineFn)/CombinePerKey/GroupByKey/Write into Globally/CombineGlobally(CountCombineFn)/CombinePerKey/GroupByKey/Reify
root: INFO: 2019-07-29T17:03:35.552Z: JOB_MESSAGE_DETAILED: Fusing consumer Globally/CombineGlobally(CountCombineFn)/CombinePerKey/GroupByKey/Reify into Globally/CombineGlobally(CountCombineFn)/CombinePerKey/GroupByKey+Globally/CombineGlobally(CountCombineFn)/CombinePerKey/Combine/Partial
root: INFO: 2019-07-29T17:03:35.599Z: JOB_MESSAGE_DETAILED: Fusing consumer read from datastore/Reshuffle/AddRandomKeys into read from datastore/SplitQuery
root: INFO: 2019-07-29T17:03:35.647Z: JOB_MESSAGE_DETAILED: Fusing consumer read from datastore/Reshuffle/ReshufflePerKey/GroupByKey/Write into read from datastore/Reshuffle/ReshufflePerKey/GroupByKey/Reify
root: INFO: 2019-07-29T17:03:35.694Z: JOB_MESSAGE_DETAILED: Fusing consumer Globally/CombineGlobally(CountCombineFn)/CombinePerKey/GroupByKey+Globally/CombineGlobally(CountCombineFn)/CombinePerKey/Combine/Partial into Globally/CombineGlobally(CountCombineFn)/KeyWithVoid
root: INFO: 2019-07-29T17:03:35.743Z: JOB_MESSAGE_DETAILED: Fusing consumer Globally/CombineGlobally(CountCombineFn)/KeyWithVoid into read from datastore/Read
root: INFO: 2019-07-29T17:03:35.783Z: JOB_MESSAGE_DETAILED: Fusing consumer read from datastore/Reshuffle/ReshufflePerKey/GroupByKey/Reify into read from datastore/Reshuffle/ReshufflePerKey/Map(reify_timestamps)
root: INFO: 2019-07-29T17:03:35.831Z: JOB_MESSAGE_DETAILED: Fusing consumer read from datastore/Read into read from datastore/Reshuffle/RemoveRandomKeys
root: INFO: 2019-07-29T17:03:35.876Z: JOB_MESSAGE_DETAILED: Fusing consumer read from datastore/Reshuffle/RemoveRandomKeys into read from datastore/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)
root: INFO: 2019-07-29T17:03:35.918Z: JOB_MESSAGE_DETAILED: Fusing consumer Globally/CombineGlobally(CountCombineFn)/UnKey into Globally/CombineGlobally(CountCombineFn)/CombinePerKey/Combine/Extract
root: INFO: 2019-07-29T17:03:35.959Z: JOB_MESSAGE_DETAILED: Fusing consumer assert_that/Match into assert_that/Unkey
root: INFO: 2019-07-29T17:03:36.002Z: JOB_MESSAGE_DETAILED: Fusing consumer assert_that/Unkey into assert_that/Group/Map(_merge_tagged_vals_under_key)
root: INFO: 2019-07-29T17:03:36.048Z: JOB_MESSAGE_DETAILED: Fusing consumer assert_that/Group/Map(_merge_tagged_vals_under_key) into assert_that/Group/GroupByKey/GroupByWindow
root: INFO: 2019-07-29T17:03:36.099Z: JOB_MESSAGE_DETAILED: Fusing consumer assert_that/Group/GroupByKey/GroupByWindow into assert_that/Group/GroupByKey/Read
root: INFO: 2019-07-29T17:03:36.144Z: JOB_MESSAGE_DETAILED: Unzipping flatten s21 for input s19.out
root: INFO: 2019-07-29T17:03:36.190Z: JOB_MESSAGE_DETAILED: Fusing unzipped copy of assert_that/Group/GroupByKey/Reify, through flatten assert_that/Group/Flatten, into producer assert_that/Group/pair_with_0
root: INFO: 2019-07-29T17:03:36.229Z: JOB_MESSAGE_DETAILED: Unzipping flatten s21-u40 for input s22-reify-value18-c38
root: INFO: 2019-07-29T17:03:36.276Z: JOB_MESSAGE_DETAILED: Fusing unzipped copy of assert_that/Group/GroupByKey/Write, through flatten assert_that/Group/Flatten/Unzipped-1, into producer assert_that/Group/GroupByKey/Reify
root: INFO: 2019-07-29T17:03:36.326Z: JOB_MESSAGE_DETAILED: Fusing consumer assert_that/Group/pair_with_0 into assert_that/Create/Read
root: INFO: 2019-07-29T17:03:36.376Z: JOB_MESSAGE_DETAILED: Fusing consumer assert_that/Group/GroupByKey/Reify into assert_that/Group/pair_with_1
root: INFO: 2019-07-29T17:03:36.421Z: JOB_MESSAGE_DETAILED: Fusing consumer assert_that/Group/GroupByKey/Write into assert_that/Group/GroupByKey/Reify
root: INFO: 2019-07-29T17:03:36.468Z: JOB_MESSAGE_DETAILED: Fusing consumer assert_that/ToVoidKey into assert_that/WindowInto(WindowIntoFn)
root: INFO: 2019-07-29T17:03:36.514Z: JOB_MESSAGE_DETAILED: Fusing consumer Globally/CombineGlobally(CountCombineFn)/InjectDefault/InjectDefault into Globally/CombineGlobally(CountCombineFn)/DoOnce/Read
root: INFO: 2019-07-29T17:03:36.562Z: JOB_MESSAGE_DETAILED: Fusing consumer assert_that/Group/pair_with_1 into assert_that/ToVoidKey
root: INFO: 2019-07-29T17:03:36.601Z: JOB_MESSAGE_DETAILED: Fusing consumer assert_that/WindowInto(WindowIntoFn) into Globally/CombineGlobally(CountCombineFn)/InjectDefault/InjectDefault
root: INFO: 2019-07-29T17:03:36.640Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
root: INFO: 2019-07-29T17:03:36.689Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
root: INFO: 2019-07-29T17:03:36.734Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
root: INFO: 2019-07-29T17:03:36.781Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
root: INFO: 2019-07-29T17:03:37.002Z: JOB_MESSAGE_DEBUG: Executing wait step start51
root: INFO: 2019-07-29T17:03:37.101Z: JOB_MESSAGE_BASIC: Executing operation Globally/CombineGlobally(CountCombineFn)/CombinePerKey/GroupByKey/Create
root: INFO: 2019-07-29T17:03:37.152Z: JOB_MESSAGE_BASIC: Executing operation assert_that/Group/GroupByKey/Create
root: INFO: 2019-07-29T17:03:37.166Z: JOB_MESSAGE_DEBUG: Starting worker pool setup.
root: INFO: 2019-07-29T17:03:37.185Z: JOB_MESSAGE_BASIC: Executing operation read from datastore/Reshuffle/ReshufflePerKey/GroupByKey/Create
root: INFO: 2019-07-29T17:03:37.207Z: JOB_MESSAGE_BASIC: Starting 1 workers in us-central1-b...
root: INFO: 2019-07-29T17:03:37.247Z: JOB_MESSAGE_BASIC: Finished operation Globally/CombineGlobally(CountCombineFn)/CombinePerKey/GroupByKey/Create
root: INFO: 2019-07-29T17:03:37.262Z: JOB_MESSAGE_BASIC: Finished operation read from datastore/Reshuffle/ReshufflePerKey/GroupByKey/Create
root: INFO: 2019-07-29T17:03:37.262Z: JOB_MESSAGE_BASIC: Finished operation assert_that/Group/GroupByKey/Create
root: INFO: 2019-07-29T17:03:37.335Z: JOB_MESSAGE_DEBUG: Value "Globally/CombineGlobally(CountCombineFn)/CombinePerKey/GroupByKey/Session" materialized.
root: INFO: 2019-07-29T17:03:37.370Z: JOB_MESSAGE_DEBUG: Value "assert_that/Group/GroupByKey/Session" materialized.
root: INFO: 2019-07-29T17:03:37.397Z: JOB_MESSAGE_DEBUG: Value "read from datastore/Reshuffle/ReshufflePerKey/GroupByKey/Session" materialized.
root: INFO: 2019-07-29T17:03:37.461Z: JOB_MESSAGE_BASIC: Executing operation assert_that/Create/Read+assert_that/Group/pair_with_0+assert_that/Group/GroupByKey/Reify+assert_that/Group/GroupByKey/Write
root: INFO: 2019-07-29T17:03:37.509Z: JOB_MESSAGE_BASIC: Executing operation read from datastore/UserQuery/Read+read from datastore/SplitQuery+read from datastore/Reshuffle/AddRandomKeys+read from datastore/Reshuffle/ReshufflePerKey/Map(reify_timestamps)+read from datastore/Reshuffle/ReshufflePerKey/GroupByKey/Reify+read from datastore/Reshuffle/ReshufflePerKey/GroupByKey/Write
root: INFO: 2019-07-29T17:03:37.710Z: JOB_MESSAGE_BASIC: Worker configuration: n1-standard-1 in us-central1-b.
root: DEBUG: Response returned status 503, retrying
root: DEBUG: Retrying request to url https://dataflow.googleapis.com/v1b3/projects/apache-beam-testing/locations/us-central1/jobs/2019-07-29_10_03_30-8577663469496961280?alt=json after exception HttpError accessing <https://dataflow.googleapis.com/v1b3/projects/apache-beam-testing/locations/us-central1/jobs/2019-07-29_10_03_30-8577663469496961280?alt=json>: response: <{'x-frame-options': 'SAMEORIGIN', 'transfer-encoding': 'chunked', 'date': 'Mon, 29 Jul 2019 17:04:21 GMT', 'server': 'ESF', 'vary': 'Origin, X-Origin, Referer', 'content-length': '102', 'content-type': 'application/json; charset=UTF-8', 'cache-control': 'private', 'x-xss-protection': '0', '-content-encoding': 'gzip', 'x-content-type-options': 'nosniff', 'status': '503'}>, content <{
"error": {
"code": 503,
"message": "Deadline exceeded",
"status": "UNAVAILABLE"
}
}
>
root: INFO: 2019-07-29T17:04:54.883Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running step(s).
root: INFO: 2019-07-29T17:06:07.728Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
root: INFO: 2019-07-29T17:06:07.760Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
root: INFO: 2019-07-29T17:06:37.721Z: JOB_MESSAGE_BASIC: Worker configuration: n1-standard-1 in us-central1-b.
root: INFO: 2019-07-29T17:09:21.183Z: JOB_MESSAGE_BASIC: Finished operation assert_that/Create/Read+assert_that/Group/pair_with_0+assert_that/Group/GroupByKey/Reify+assert_that/Group/GroupByKey/Write
root: INFO: 2019-07-29T17:09:27.332Z: JOB_MESSAGE_BASIC: Finished operation read from datastore/UserQuery/Read+read from datastore/SplitQuery+read from datastore/Reshuffle/AddRandomKeys+read from datastore/Reshuffle/ReshufflePerKey/Map(reify_timestamps)+read from datastore/Reshuffle/ReshufflePerKey/GroupByKey/Reify+read from datastore/Reshuffle/ReshufflePerKey/GroupByKey/Write
root: INFO: 2019-07-29T17:09:27.418Z: JOB_MESSAGE_BASIC: Executing operation read from datastore/Reshuffle/ReshufflePerKey/GroupByKey/Close
root: INFO: 2019-07-29T17:09:27.470Z: JOB_MESSAGE_BASIC: Finished operation read from datastore/Reshuffle/ReshufflePerKey/GroupByKey/Close
root: INFO: 2019-07-29T17:09:27.551Z: JOB_MESSAGE_BASIC: Executing operation read from datastore/Reshuffle/ReshufflePerKey/GroupByKey/Read+read from datastore/Reshuffle/ReshufflePerKey/GroupByKey/GroupByWindow+read from datastore/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)+read from datastore/Reshuffle/RemoveRandomKeys+read from datastore/Read+Globally/CombineGlobally(CountCombineFn)/KeyWithVoid+Globally/CombineGlobally(CountCombineFn)/CombinePerKey/GroupByKey+Globally/CombineGlobally(CountCombineFn)/CombinePerKey/Combine/Partial+Globally/CombineGlobally(CountCombineFn)/CombinePerKey/GroupByKey/Reify+Globally/CombineGlobally(CountCombineFn)/CombinePerKey/GroupByKey/Write
root: INFO: 2019-07-29T17:09:36.969Z: JOB_MESSAGE_DETAILED: Checking permissions granted to controller Service Account.
root: INFO: 2019-07-29T17:09:37.043Z: JOB_MESSAGE_BASIC: Finished operation read from datastore/Reshuffle/ReshufflePerKey/GroupByKey/Read+read from datastore/Reshuffle/ReshufflePerKey/GroupByKey/GroupByWindow+read from datastore/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)+read from datastore/Reshuffle/RemoveRandomKeys+read from datastore/Read+Globally/CombineGlobally(CountCombineFn)/KeyWithVoid+Globally/CombineGlobally(CountCombineFn)/CombinePerKey/GroupByKey+Globally/CombineGlobally(CountCombineFn)/CombinePerKey/Combine/Partial+Globally/CombineGlobally(CountCombineFn)/CombinePerKey/GroupByKey/Reify+Globally/CombineGlobally(CountCombineFn)/CombinePerKey/GroupByKey/Write
root: INFO: 2019-07-29T17:09:37.138Z: JOB_MESSAGE_BASIC: Executing operation Globally/CombineGlobally(CountCombineFn)/CombinePerKey/GroupByKey/Close
root: INFO: 2019-07-29T17:09:37.200Z: JOB_MESSAGE_BASIC: Finished operation Globally/CombineGlobally(CountCombineFn)/CombinePerKey/GroupByKey/Close
root: INFO: 2019-07-29T17:09:37.295Z: JOB_MESSAGE_BASIC: Executing operation Globally/CombineGlobally(CountCombineFn)/CombinePerKey/GroupByKey/Read+Globally/CombineGlobally(CountCombineFn)/CombinePerKey/Combine+Globally/CombineGlobally(CountCombineFn)/CombinePerKey/Combine/Extract+Globally/CombineGlobally(CountCombineFn)/UnKey
root: INFO: 2019-07-29T17:09:37.701Z: JOB_MESSAGE_BASIC: Worker configuration: n1-standard-1 in us-central1-b.
root: INFO: 2019-07-29T17:09:43.037Z: JOB_MESSAGE_BASIC: Finished operation Globally/CombineGlobally(CountCombineFn)/CombinePerKey/GroupByKey/Read+Globally/CombineGlobally(CountCombineFn)/CombinePerKey/Combine+Globally/CombineGlobally(CountCombineFn)/CombinePerKey/Combine/Extract+Globally/CombineGlobally(CountCombineFn)/UnKey
root: INFO: 2019-07-29T17:09:43.122Z: JOB_MESSAGE_DEBUG: Value "Globally/CombineGlobally(CountCombineFn)/UnKey.out" materialized.
root: INFO: 2019-07-29T17:09:43.226Z: JOB_MESSAGE_BASIC: Executing operation Globally/CombineGlobally(CountCombineFn)/InjectDefault/_UnpickledSideInput(UnKey.out.0)
root: INFO: 2019-07-29T17:09:43.281Z: JOB_MESSAGE_BASIC: Finished operation Globally/CombineGlobally(CountCombineFn)/InjectDefault/_UnpickledSideInput(UnKey.out.0)
root: INFO: 2019-07-29T17:09:43.371Z: JOB_MESSAGE_DEBUG: Value "Globally/CombineGlobally(CountCombineFn)/InjectDefault/_UnpickledSideInput(UnKey.out.0).output" materialized.
root: INFO: 2019-07-29T17:09:43.475Z: JOB_MESSAGE_BASIC: Executing operation Globally/CombineGlobally(CountCombineFn)/DoOnce/Read+Globally/CombineGlobally(CountCombineFn)/InjectDefault/InjectDefault+assert_that/WindowInto(WindowIntoFn)+assert_that/ToVoidKey+assert_that/Group/pair_with_1+assert_that/Group/GroupByKey/Reify+assert_that/Group/GroupByKey/Write
root: INFO: 2019-07-29T17:09:48.946Z: JOB_MESSAGE_BASIC: Finished operation Globally/CombineGlobally(CountCombineFn)/DoOnce/Read+Globally/CombineGlobally(CountCombineFn)/InjectDefault/InjectDefault+assert_that/WindowInto(WindowIntoFn)+assert_that/ToVoidKey+assert_that/Group/pair_with_1+assert_that/Group/GroupByKey/Reify+assert_that/Group/GroupByKey/Write
root: INFO: 2019-07-29T17:09:49.040Z: JOB_MESSAGE_BASIC: Executing operation assert_that/Group/GroupByKey/Close
root: INFO: 2019-07-29T17:09:49.098Z: JOB_MESSAGE_BASIC: Finished operation assert_that/Group/GroupByKey/Close
root: INFO: 2019-07-29T17:09:49.186Z: JOB_MESSAGE_BASIC: Executing operation assert_that/Group/GroupByKey/Read+assert_that/Group/GroupByKey/GroupByWindow+assert_that/Group/Map(_merge_tagged_vals_under_key)+assert_that/Unkey+assert_that/Match
root: INFO: 2019-07-29T17:09:56.870Z: JOB_MESSAGE_BASIC: Finished operation assert_that/Group/GroupByKey/Read+assert_that/Group/GroupByKey/GroupByWindow+assert_that/Group/Map(_merge_tagged_vals_under_key)+assert_that/Unkey+assert_that/Match
root: INFO: 2019-07-29T17:09:56.960Z: JOB_MESSAGE_DEBUG: Executing success step success49
root: INFO: 2019-07-29T17:09:57.102Z: JOB_MESSAGE_DETAILED: Cleaning up.
root: INFO: 2019-07-29T17:09:57.174Z: JOB_MESSAGE_DEBUG: Starting worker pool teardown.
root: INFO: 2019-07-29T17:09:57.222Z: JOB_MESSAGE_BASIC: Stopping worker pool...
--------------------- >> end captured logging << ---------------------
<https://builds.apache.org/job/beam_PostCommit_Python35/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1150: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-29_09_14_20-16554626987630401447?project=apache-beam-testing.
experiments = p.options.view_as(DebugOptions).experiments or []
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-29_09_29_55-1781273861391683266?project=apache-beam-testing.
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-29_09_39_26-4380675135403761168?project=apache-beam-testing.
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-29_09_47_58-12546024895394856940?project=apache-beam-testing.
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-29_09_14_17-16872718917027727628?project=apache-beam-testing.
<https://builds.apache.org/job/beam_PostCommit_Python35/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py>:686: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-29_09_39_43-6611921833867009894?project=apache-beam-testing.
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-29_09_48_55-5726842228423571644?project=apache-beam-testing.
kms_key=transform.kms_key))
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-29_09_14_17-4329247305013312919?project=apache-beam-testing.
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-29_09_36_23-8817564965612675548?project=apache-beam-testing.
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-29_09_46_03-15767421516849907671?project=apache-beam-testing.
<https://builds.apache.org/job/beam_PostCommit_Python35/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1150: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
<https://builds.apache.org/job/beam_PostCommit_Python35/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:638: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-29_09_14_18-12470095459566569642?project=apache-beam-testing.
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-29_09_29_09-7216937159026432051?project=apache-beam-testing.
<https://builds.apache.org/job/beam_PostCommit_Python35/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1150: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-29_09_38_33-8482127783097293291?project=apache-beam-testing.
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-29_09_48_16-14093974718501818233?project=apache-beam-testing.
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-29_09_57_49-15481298757806687401?project=apache-beam-testing.
<https://builds.apache.org/job/beam_PostCommit_Python35/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py>:686: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=transform.kms_key))
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-29_09_14_16-14999101986688534145?project=apache-beam-testing.
<https://builds.apache.org/job/beam_PostCommit_Python35/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py>:686: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=transform.kms_key))
<https://builds.apache.org/job/beam_PostCommit_Python35/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:572: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
<https://builds.apache.org/job/beam_PostCommit_Python35/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1150: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
<https://builds.apache.org/job/beam_PostCommit_Python35/ws/src/sdks/python/apache_beam/io/fileio_test.py>:232: FutureWarning: MatchAll is experimental.
| 'GetPath' >> beam.Map(lambda metadata: metadata.path))
<https://builds.apache.org/job/beam_PostCommit_Python35/ws/src/sdks/python/apache_beam/io/fileio_test.py>:243: FutureWarning: MatchAll is experimental.
| 'Checksums' >> beam.Map(compute_hash))
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-29_09_23_15-13237398010837671234?project=apache-beam-testing.
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-29_09_33_01-7045251661021664475?project=apache-beam-testing.
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-29_09_42_31-5738131477163088266?project=apache-beam-testing.
<https://builds.apache.org/job/beam_PostCommit_Python35/ws/src/sdks/python/apache_beam/io/fileio_test.py>:243: FutureWarning: ReadMatches is experimental.
| 'Checksums' >> beam.Map(compute_hash))
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-29_09_14_16-16264651409388807458?project=apache-beam-testing.
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-29_09_24_53-10010703935941892890?project=apache-beam-testing.
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-29_09_37_26-9733252687878411867?project=apache-beam-testing.
<https://builds.apache.org/job/beam_PostCommit_Python35/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1150: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
<https://builds.apache.org/job/beam_PostCommit_Python35/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:638: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-29_09_47_16-12256420093469972924?project=apache-beam-testing.
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-29_09_56_27-735232717687081112?project=apache-beam-testing.
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-29_09_14_21-4601012039830775987?project=apache-beam-testing.
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-29_09_24_17-2837499384743381733?project=apache-beam-testing.
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-29_09_32_58-15476535426693801445?project=apache-beam-testing.
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-29_09_43_10-12223229886322267843?project=apache-beam-testing.
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-29_09_53_52-668165926708515553?project=apache-beam-testing.
Exception in thread Thread-48:
Traceback (most recent call last):
File "/usr/lib/python3.5/threading.py", line 914, in _bootstrap_inner
self.run()
File "/usr/lib/python3.5/threading.py", line 862, in run
self._target(*self._args, **self._kwargs)
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-29_10_03_30-8577663469496961280?project=apache-beam-testing.
File "<https://builds.apache.org/job/beam_PostCommit_Python35/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 190, in poll_for_job_completion
job_id, page_token=page_token, start_time=last_message_time)
File "<https://builds.apache.org/job/beam_PostCommit_Python35/ws/src/sdks/python/apache_beam/utils/retry.py",> line 197, in wrapper
return fun(*args, **kwargs)
File "<https://builds.apache.org/job/beam_PostCommit_Python35/ws/src/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py",> line 755, in list_messages
response = self._client.projects_locations_jobs_messages.List(request)
File "<https://builds.apache.org/job/beam_PostCommit_Python35/ws/src/sdks/python/apache_beam/runners/dataflow/internal/clients/dataflow/dataflow_v1b3_client.py",> line 553, in List
config, request, global_params=global_params)
File "<https://builds.apache.org/job/beam_PostCommit_Python35/ws/src/build/gradleenv/-1734967054/lib/python3.5/site-packages/apitools/base/py/base_api.py",> line 731, in _RunMethod
return self.ProcessHttpResponse(method_config, http_response, request)
File "<https://builds.apache.org/job/beam_PostCommit_Python35/ws/src/build/gradleenv/-1734967054/lib/python3.5/site-packages/apitools/base/py/base_api.py",> line 737, in ProcessHttpResponse
self.__ProcessHttpResponse(method_config, http_response, request))
File "<https://builds.apache.org/job/beam_PostCommit_Python35/ws/src/build/gradleenv/-1734967054/lib/python3.5/site-packages/apitools/base/py/base_api.py",> line 604, in __ProcessHttpResponse
http_response, method_config=method_config, request=request)
apitools.base.py.exceptions.HttpNotFoundError: HttpError accessing <https://dataflow.googleapis.com/v1b3/projects/apache-beam-testing/locations/us-central1/jobs/2019-07-29_10_03_30-8577663469496961280/messages?alt=json&startTime=2019-07-29T17%3A09%3A57.222Z>: response: <{'x-frame-options': 'SAMEORIGIN', 'transfer-encoding': 'chunked', 'date': 'Mon, 29 Jul 2019 17:10:44 GMT', 'server': 'ESF', 'vary': 'Origin, X-Origin, Referer', 'content-length': '279', 'content-type': 'application/json; charset=UTF-8', 'cache-control': 'private', 'x-xss-protection': '0', '-content-encoding': 'gzip', 'x-content-type-options': 'nosniff', 'status': '404'}>, content <{
"error": {
"code": 404,
"message": "(35d9c984e28269ca): Information about job 2019-07-29_10_03_30-8577663469496961280 could not be found in our system. Please double check the id is correct. If it is please contact customer support.",
"status": "NOT_FOUND"
}
}
>
<https://builds.apache.org/job/beam_PostCommit_Python35/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py>:686: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=transform.kms_key))
<https://builds.apache.org/job/beam_PostCommit_Python35/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:73: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=kms_key))
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-29_09_14_18-2231155599897928094?project=apache-beam-testing.
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-29_09_24_43-2446914317168478353?project=apache-beam-testing.
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-29_09_32_56-4270316422688565508?project=apache-beam-testing.
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-29_09_42_13-14858510199775532259?project=apache-beam-testing.
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-29_09_51_06-12936717678592015247?project=apache-beam-testing.
----------------------------------------------------------------------
XML: nosetests-postCommitIT-df-py35.xml
----------------------------------------------------------------------
XML: <https://builds.apache.org/job/beam_PostCommit_Python35/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 42 tests in 3534.826s
FAILED (SKIP=5, failures=1)
> Task :sdks:python:test-suites:dataflow:py35:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://builds.apache.org/job/beam_PostCommit_Python35/ws/src/sdks/python/test-suites/dataflow/py35/build.gradle'> line: 49
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py35:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 59m 49s
63 actionable tasks: 46 executed, 17 from cache
Publishing build scan...
https://gradle.com/s/d53osftofm53q
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Jenkins build is back to normal : beam_PostCommit_Python35 #80
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Python35/80/display/redirect?page=changes>
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python35 #79
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Python35/79/display/redirect?page=changes>
Changes:
[iemejia] [BEAM-7807] Forbid TestNG imports on Beam classes
[iemejia] [BEAM-7807] Update Beam tests to not import TestNG related classes
[alireza4263] [BEAM-7783] BeamIOSourceRel using custom BeamStatistics
[pabloem] [BEAM-7814] Make BigqueryMatcher wait for results. (#9154)
------------------------------------------
[...truncated 186.94 KB...]
{
"encoding": {
"@type": "kind:windowed_value",
"component_encodings": [
{
"@type": "FastPrimitivesCoder$eNprYE5OLEhMzkiNT0pNzNVLzk9JLSqGUlxuicUlAUWZuZklmWWpxc4gQa5CBs3GQsbaQqYIfgYGhvi0xJycpMTk7HiwlkJ8pgVkJmfnpEJNYQGawlpbyJZUnKQHACYlLgM=",
"component_encodings": [
{
"@type": "FastPrimitivesCoder$eNprYE5OLEhMzkiNT0pNzNVLzk9JLSqGUlxuicUlAUWZuZklmWWpxc4gQa5CBs3GQsbaQqYIfgYGhvi0xJycpMTk7HiwlkJ8pgVkJmfnpEJNYQGawlpbyJZUnKQHACYlLgM=",
"component_encodings": []
},
{
"@type": "FastPrimitivesCoder$eNprYE5OLEhMzkiNT0pNzNVLzk9JLSqGUlxuicUlAUWZuZklmWWpxc4gQa5CBs3GQsbaQqYIfgYGhvi0xJycpMTk7HiwlkJ8pgVkJmfnpEJNYQGawlpbyJZUnKQHACYlLgM=",
"component_encodings": []
}
],
"is_pair_like": true
},
{
"@type": "kind:global_window"
}
],
"is_wrapper": true
},
"output_name": "out",
"user_name": "Assert Checksums/Match.out"
}
],
"parallel_input": {
"@type": "OutputReference",
"output_name": "out",
"step_name": "s27"
},
"serialized_fn": "eNq1VVt320QQXtlO2qoJbRJoUlrALRQUoFYudZOWptA4l6amblDTegsEsZLWlhLdZrVqkkN8DpTjnPwO/goPvPCjGMlOQ4D0jeOjlWbmm29n57L+uajZLGa2y02Ls6AiBQuTViSCpGJHgqs15vvM8nlTsDjmYilaCVUgk7+A0oGCRs8QQmIR2TxJoGg7nu9XzGxVTVtwJrnZSkNbehE6lbQTdj9ijin3Yq7CAD2LNLXI4Rsow2AXzhhwVqsrdYJPoT5WGz4kZJ+QXxXSVshTONfogjpJFfTahfNdGKIBfupuFHB9i4fbXpgcvW8mPnvJ9Z1IbCd4TK5npzTXo0TWoiDwpLm+J90onK3qibD1xNlO9DjX6H/Lin6cFT3LSiXeg+E86Hs+CyyH3Ye3Hl8cqBG4QAuobYVwsQsjkxJGDRg7cew2lyaTUqjwdk5gpZ4vMU54J88lmjMrXDqAcQMmTrh6QRwJaQaRk/qYtcv0Cjq8oXbwbheuGHA138dEEluaJrx3AO8b8AEdzJQcUuZDufFflbM5CnDNLWluvxYD9VGsxZ+SkMO8Fh2F7N0nUjkSC9l3r0ydItkvkP0i2S4SsUZkgThKX9MqkEuIeKWQZvgTKUnEqET8QRRF2X2UuS9tLpJOieyNkH2FbJXIfiljVJrAED2Qo3/L0D3SXmcck75AGMWnic7id3IKKFQIdQi20vUGHcdMrDDP506ZJQkX8m75higvLOAKHx7ARxotIcL3Egk38rQlWAbuwMd0DIVFzPyD3G151+Zx1uvwCT2HlqyZl4WIBGi5m+BB9JLDJFVReM78tG/9VMJnPQSzZVaPz+kwCnw35jbuY+Y736QXXu9sHpmgkiP72r63njcS93nAQwlTEqbp1v88HTzBFm7rqfT8bDRm3HJ9p3adKEMDRWUo/xWV8cKwMozvkXy9qgziCrN5b74+zq0uVHFobhsw5064l+nEPxu8t1El2wjmu3DHgLsuNvQXBtxzyw332iYsUC0bwFvT89ZMtTVdnZprVdmdqemZ6tzUzG1rarbK7Jn56hybY9b8HNxnXfjSgK+68KADi/R8NhTZpWS6XigTqJ28G9GQ6ysOxwFjMhKJuvYkq/PDTK3CEl6My40OrGg5lRfGqcz5Elht0CFURak81j1spAewZmF5HxlQ78LXBjzuAvo/0dyam5Gto/CN5q423BxrWL0QmWgnmDMTL5qnbj2VsGHAs39F/zynaCIFPaZ4YdFihkPXb91nqbUJ33Xg+03YfOPfQNMLnWgHc6/CD8hpduBHjY7kLWunQeqzrO+zq4kDqyu9SLzEdHiLpb4E65COokp6AVaQBbFpR4HlhVyAjehsWKTw2m0uMCrntED6EHWpR7rRF4FjQK28kXbyKJGjfRpHD6Gu+pHF/N6hsGwuMnipJWGr8hcAYCuy",
"user_name": "Assert Checksums/Match"
}
}
],
"type": "JOB_TYPE_BATCH"
}
root: INFO: Create job: <Job
createTime: '2019-07-29T18:01:22.256628Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2019-07-29_11_01_20-1919703129159653506'
location: 'us-central1'
name: 'beamapp-jenkins-0729180055-769686'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2019-07-29T18:01:22.256628Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
root: INFO: Created job with id: [2019-07-29_11_01_20-1919703129159653506]
root: INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-29_11_01_20-1919703129159653506?project=apache-beam-testing
root: INFO: Job 2019-07-29_11_01_20-1919703129159653506 is in state JOB_STATE_RUNNING
root: INFO: 2019-07-29T18:01:20.397Z: JOB_MESSAGE_DETAILED: Autoscaling is enabled for job 2019-07-29_11_01_20-1919703129159653506. The number of workers will be between 1 and 1000.
root: INFO: 2019-07-29T18:01:20.611Z: JOB_MESSAGE_DETAILED: Autoscaling was automatically enabled for job 2019-07-29_11_01_20-1919703129159653506.
root: INFO: 2019-07-29T18:01:24.324Z: JOB_MESSAGE_DETAILED: Checking permissions granted to controller Service Account.
root: INFO: 2019-07-29T18:01:25.078Z: JOB_MESSAGE_BASIC: Worker configuration: n1-standard-1 in us-central1-a.
root: INFO: 2019-07-29T18:01:25.766Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
root: INFO: 2019-07-29T18:01:25.810Z: JOB_MESSAGE_DEBUG: Combiner lifting skipped for step Assert Checksums/Group/GroupByKey: GroupByKey not followed by a combiner.
root: INFO: 2019-07-29T18:01:25.852Z: JOB_MESSAGE_DEBUG: Combiner lifting skipped for step Matched Files/Group/GroupByKey: GroupByKey not followed by a combiner.
root: INFO: 2019-07-29T18:01:25.900Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
root: INFO: 2019-07-29T18:01:25.942Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
root: INFO: 2019-07-29T18:01:26.077Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
root: INFO: 2019-07-29T18:01:26.146Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
root: INFO: 2019-07-29T18:01:26.185Z: JOB_MESSAGE_DETAILED: Unzipping flatten s9 for input s7.out
root: INFO: 2019-07-29T18:01:26.222Z: JOB_MESSAGE_DETAILED: Fusing unzipped copy of Matched Files/Group/GroupByKey/Reify, through flatten Matched Files/Group/Flatten, into producer Matched Files/Group/pair_with_0
root: INFO: 2019-07-29T18:01:26.258Z: JOB_MESSAGE_DETAILED: Fusing consumer Matched Files/Group/GroupByKey/Reify into Matched Files/Group/pair_with_1
root: INFO: 2019-07-29T18:01:26.306Z: JOB_MESSAGE_DETAILED: Fusing consumer Matched Files/Match into Matched Files/Unkey
root: INFO: 2019-07-29T18:01:26.356Z: JOB_MESSAGE_DETAILED: Fusing consumer Matched Files/Unkey into Matched Files/Group/Map(_merge_tagged_vals_under_key)
root: INFO: 2019-07-29T18:01:26.402Z: JOB_MESSAGE_DETAILED: Fusing consumer Matched Files/Group/Map(_merge_tagged_vals_under_key) into Matched Files/Group/GroupByKey/GroupByWindow
root: INFO: 2019-07-29T18:01:26.442Z: JOB_MESSAGE_DETAILED: Fusing consumer Matched Files/Group/GroupByKey/GroupByWindow into Matched Files/Group/GroupByKey/Read
root: INFO: 2019-07-29T18:01:26.488Z: JOB_MESSAGE_DETAILED: Unzipping flatten s9-u22 for input s10-reify-value0-c20
root: INFO: 2019-07-29T18:01:26.535Z: JOB_MESSAGE_DETAILED: Fusing unzipped copy of Matched Files/Group/GroupByKey/Write, through flatten Matched Files/Group/Flatten/Unzipped-1, into producer Matched Files/Group/GroupByKey/Reify
root: INFO: 2019-07-29T18:01:26.565Z: JOB_MESSAGE_DETAILED: Fusing consumer Matched Files/Group/GroupByKey/Write into Matched Files/Group/GroupByKey/Reify
root: INFO: 2019-07-29T18:01:26.595Z: JOB_MESSAGE_DETAILED: Fusing consumer Matched Files/Group/pair_with_1 into Matched Files/ToVoidKey
root: INFO: 2019-07-29T18:01:26.636Z: JOB_MESSAGE_DETAILED: Fusing consumer Matched Files/ToVoidKey into Matched Files/WindowInto(WindowIntoFn)
root: INFO: 2019-07-29T18:01:26.684Z: JOB_MESSAGE_DETAILED: Fusing consumer MatchAll/ParDo(_MatchAllFn) into Create/Read
root: INFO: 2019-07-29T18:01:26.721Z: JOB_MESSAGE_DETAILED: Fusing consumer Matched Files/WindowInto(WindowIntoFn) into GetPath
root: INFO: 2019-07-29T18:01:26.762Z: JOB_MESSAGE_DETAILED: Fusing consumer GetPath into MatchAll/ParDo(_MatchAllFn)
root: INFO: 2019-07-29T18:01:26.807Z: JOB_MESSAGE_DETAILED: Fusing consumer Assert Checksums/Match into Assert Checksums/Unkey
root: INFO: 2019-07-29T18:01:26.854Z: JOB_MESSAGE_DETAILED: Fusing consumer Assert Checksums/Group/GroupByKey/GroupByWindow into Assert Checksums/Group/GroupByKey/Read
root: INFO: 2019-07-29T18:01:26.913Z: JOB_MESSAGE_DETAILED: Fusing consumer Assert Checksums/Group/GroupByKey/Write into Assert Checksums/Group/GroupByKey/Reify
root: INFO: 2019-07-29T18:01:26.962Z: JOB_MESSAGE_DETAILED: Unzipping flatten s24 for input s22.out
root: INFO: 2019-07-29T18:01:27.007Z: JOB_MESSAGE_DETAILED: Fusing unzipped copy of Assert Checksums/Group/GroupByKey/Reify, through flatten Assert Checksums/Group/Flatten, into producer Assert Checksums/Group/pair_with_0
root: INFO: 2019-07-29T18:01:27.064Z: JOB_MESSAGE_DETAILED: Fusing consumer Assert Checksums/Group/GroupByKey/Reify into Assert Checksums/Group/pair_with_1
root: INFO: 2019-07-29T18:01:27.101Z: JOB_MESSAGE_DETAILED: Fusing consumer Assert Checksums/Unkey into Assert Checksums/Group/Map(_merge_tagged_vals_under_key)
root: INFO: 2019-07-29T18:01:27.143Z: JOB_MESSAGE_DETAILED: Fusing consumer Assert Checksums/Group/Map(_merge_tagged_vals_under_key) into Assert Checksums/Group/GroupByKey/GroupByWindow
root: INFO: 2019-07-29T18:01:27.182Z: JOB_MESSAGE_DETAILED: Fusing consumer Assert Checksums/ToVoidKey into Assert Checksums/WindowInto(WindowIntoFn)
root: INFO: 2019-07-29T18:01:27.228Z: JOB_MESSAGE_DETAILED: Fusing consumer ReadIn into ReadMatches/ParDo(_ReadMatchesFn)
root: INFO: 2019-07-29T18:01:27.273Z: JOB_MESSAGE_DETAILED: Fusing consumer Assert Checksums/Group/pair_with_1 into Assert Checksums/ToVoidKey
root: INFO: 2019-07-29T18:01:27.348Z: JOB_MESSAGE_DETAILED: Fusing consumer Assert Checksums/WindowInto(WindowIntoFn) into Checksums
root: INFO: 2019-07-29T18:01:27.401Z: JOB_MESSAGE_DETAILED: Fusing consumer ReadMatches/ParDo(_ReadMatchesFn) into MatchOneAll/ParDo(_MatchAllFn)
root: INFO: 2019-07-29T18:01:27.446Z: JOB_MESSAGE_DETAILED: Fusing consumer MatchOneAll/ParDo(_MatchAllFn) into SingleFile/Read
root: INFO: 2019-07-29T18:01:27.489Z: JOB_MESSAGE_DETAILED: Fusing consumer Checksums into ReadIn
root: INFO: 2019-07-29T18:01:27.537Z: JOB_MESSAGE_DETAILED: Fusing consumer Assert Checksums/Group/pair_with_0 into Assert Checksums/Create/Read
root: INFO: 2019-07-29T18:01:27.580Z: JOB_MESSAGE_DETAILED: Fusing consumer Matched Files/Group/pair_with_0 into Matched Files/Create/Read
root: INFO: 2019-07-29T18:01:27.632Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
root: INFO: 2019-07-29T18:01:27.680Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
root: INFO: 2019-07-29T18:01:27.718Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
root: INFO: 2019-07-29T18:01:27.751Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
root: INFO: 2019-07-29T18:01:27.910Z: JOB_MESSAGE_DEBUG: Executing wait step start37
root: INFO: 2019-07-29T18:01:28.019Z: JOB_MESSAGE_BASIC: Executing operation Matched Files/Group/GroupByKey/Create
root: INFO: 2019-07-29T18:01:28.067Z: JOB_MESSAGE_BASIC: Executing operation Assert Checksums/Group/GroupByKey/Create
root: INFO: 2019-07-29T18:01:28.067Z: JOB_MESSAGE_DEBUG: Starting worker pool setup.
root: INFO: 2019-07-29T18:01:28.111Z: JOB_MESSAGE_BASIC: Starting 1 workers in us-central1-a...
root: INFO: 2019-07-29T18:01:28.168Z: JOB_MESSAGE_BASIC: Finished operation Matched Files/Group/GroupByKey/Create
root: INFO: 2019-07-29T18:01:28.183Z: JOB_MESSAGE_BASIC: Finished operation Assert Checksums/Group/GroupByKey/Create
root: INFO: 2019-07-29T18:01:28.269Z: JOB_MESSAGE_DEBUG: Value "Matched Files/Group/GroupByKey/Session" materialized.
root: INFO: 2019-07-29T18:01:28.321Z: JOB_MESSAGE_DEBUG: Value "Assert Checksums/Group/GroupByKey/Session" materialized.
root: INFO: 2019-07-29T18:01:28.364Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+MatchAll/ParDo(_MatchAllFn)+GetPath+Matched Files/WindowInto(WindowIntoFn)+Matched Files/ToVoidKey+Matched Files/Group/pair_with_1+Matched Files/Group/GroupByKey/Reify+Matched Files/Group/GroupByKey/Write
root: INFO: 2019-07-29T18:01:28.403Z: JOB_MESSAGE_BASIC: Executing operation Matched Files/Create/Read+Matched Files/Group/pair_with_0+Matched Files/Group/GroupByKey/Reify+Matched Files/Group/GroupByKey/Write
root: INFO: 2019-07-29T18:01:28.424Z: JOB_MESSAGE_BASIC: Worker configuration: n1-standard-1 in us-central1-a.
root: INFO: 2019-07-29T18:01:28.448Z: JOB_MESSAGE_BASIC: Executing operation SingleFile/Read+MatchOneAll/ParDo(_MatchAllFn)+ReadMatches/ParDo(_ReadMatchesFn)+ReadIn+Checksums+Assert Checksums/WindowInto(WindowIntoFn)+Assert Checksums/ToVoidKey+Assert Checksums/Group/pair_with_1+Assert Checksums/Group/GroupByKey/Reify+Assert Checksums/Group/GroupByKey/Write
root: INFO: 2019-07-29T18:01:28.485Z: JOB_MESSAGE_BASIC: Executing operation Assert Checksums/Create/Read+Assert Checksums/Group/pair_with_0+Assert Checksums/Group/GroupByKey/Reify+Assert Checksums/Group/GroupByKey/Write
--------------------- >> end captured logging << ---------------------
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-29_10_33_13-8621547323229702864?project=apache-beam-testing.
<https://builds.apache.org/job/beam_PostCommit_Python35/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1150: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-29_10_49_46-11152823909643435410?project=apache-beam-testing.
experiments = p.options.view_as(DebugOptions).experiments or []
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-29_11_00_08-10516507137718543536?project=apache-beam-testing.
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-29_10_33_10-10976540868266161522?project=apache-beam-testing.
<https://builds.apache.org/job/beam_PostCommit_Python35/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py>:686: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=transform.kms_key))
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-29_10_56_49-5657981805749534674?project=apache-beam-testing.
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-29_11_06_42-14936148133958891224?project=apache-beam-testing.
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-29_11_16_44-15870549640413822395?project=apache-beam-testing.
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-29_10_33_14-1298605329916714841?project=apache-beam-testing.
<https://builds.apache.org/job/beam_PostCommit_Python35/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1150: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-29_10_47_10-11332141571616163022?project=apache-beam-testing.
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-29_10_57_36-4971724100815511672?project=apache-beam-testing.
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-29_11_07_29-14840288146329432450?project=apache-beam-testing.
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-29_10_33_10-8598705492963194078?project=apache-beam-testing.
<https://builds.apache.org/job/beam_PostCommit_Python35/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1150: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-29_10_55_34-10505438015041720780?project=apache-beam-testing.
experiments = p.options.view_as(DebugOptions).experiments or []
<https://builds.apache.org/job/beam_PostCommit_Python35/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:638: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-29_11_06_08-17020635805596634056?project=apache-beam-testing.
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-29_11_16_40-12221638675963538311?project=apache-beam-testing.
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-29_10_33_10-12135021353562721826?project=apache-beam-testing.
<https://builds.apache.org/job/beam_PostCommit_Python35/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py>:686: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-29_10_43_27-14973447856550496488?project=apache-beam-testing.
kms_key=transform.kms_key))
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-29_10_53_15-12329679915844108092?project=apache-beam-testing.
<https://builds.apache.org/job/beam_PostCommit_Python35/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:73: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=kms_key))
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-29_11_02_33-16664300598768416817?project=apache-beam-testing.
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-29_11_17_47-2485898633742394663?project=apache-beam-testing.
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-29_10_33_09-7513928440588264310?project=apache-beam-testing.
<https://builds.apache.org/job/beam_PostCommit_Python35/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py>:686: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=transform.kms_key))
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-29_10_42_23-11310748289808289502?project=apache-beam-testing.
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-29_10_52_21-13050680584289665696?project=apache-beam-testing.
<https://builds.apache.org/job/beam_PostCommit_Python35/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:572: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-29_11_01_20-1919703129159653506?project=apache-beam-testing.
<https://builds.apache.org/job/beam_PostCommit_Python35/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1150: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-29_11_06_44-14553750767099328517?project=apache-beam-testing.
experiments = p.options.view_as(DebugOptions).experiments or []
<https://builds.apache.org/job/beam_PostCommit_Python35/ws/src/sdks/python/apache_beam/io/fileio_test.py>:232: FutureWarning: MatchAll is experimental.
| 'GetPath' >> beam.Map(lambda metadata: metadata.path))
<https://builds.apache.org/job/beam_PostCommit_Python35/ws/src/sdks/python/apache_beam/io/fileio_test.py>:243: FutureWarning: MatchAll is experimental.
| 'Checksums' >> beam.Map(compute_hash))
<https://builds.apache.org/job/beam_PostCommit_Python35/ws/src/sdks/python/apache_beam/io/fileio_test.py>:243: FutureWarning: ReadMatches is experimental.
| 'Checksums' >> beam.Map(compute_hash))
Exception in thread Thread-4:
Traceback (most recent call last):
File "/usr/lib/python3.5/threading.py", line 914, in _bootstrap_inner
self.run()
File "/usr/lib/python3.5/threading.py", line 862, in run
self._target(*self._args, **self._kwargs)
File "<https://builds.apache.org/job/beam_PostCommit_Python35/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 157, in poll_for_job_completion
response = runner.dataflow_client.get_job(job_id)
File "<https://builds.apache.org/job/beam_PostCommit_Python35/ws/src/sdks/python/apache_beam/utils/retry.py",> line 197, in wrapper
return fun(*args, **kwargs)
File "<https://builds.apache.org/job/beam_PostCommit_Python35/ws/src/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py",> line 670, in get_job
response = self._client.projects_locations_jobs.Get(request)
File "<https://builds.apache.org/job/beam_PostCommit_Python35/ws/src/sdks/python/apache_beam/runners/dataflow/internal/clients/dataflow/dataflow_v1b3_client.py",> line 689, in Get
config, request, global_params=global_params)
File "<https://builds.apache.org/job/beam_PostCommit_Python35/ws/src/build/gradleenv/-1734967054/lib/python3.5/site-packages/apitools/base/py/base_api.py",> line 731, in _RunMethod
return self.ProcessHttpResponse(method_config, http_response, request)
File "<https://builds.apache.org/job/beam_PostCommit_Python35/ws/src/build/gradleenv/-1734967054/lib/python3.5/site-packages/apitools/base/py/base_api.py",> line 737, in ProcessHttpResponse
self.__ProcessHttpResponse(method_config, http_response, request))
File "<https://builds.apache.org/job/beam_PostCommit_Python35/ws/src/build/gradleenv/-1734967054/lib/python3.5/site-packages/apitools/base/py/base_api.py",> line 604, in __ProcessHttpResponse
http_response, method_config=method_config, request=request)
apitools.base.py.exceptions.HttpNotFoundError: HttpError accessing <https://dataflow.googleapis.com/v1b3/projects/apache-beam-testing/locations/us-central1/jobs/2019-07-29_11_01_20-1919703129159653506?alt=json>: response: <{'date': 'Mon, 29 Jul 2019 18:03:03 GMT', 'status': '404', 'vary': 'Origin, X-Origin, Referer', 'x-frame-options': 'SAMEORIGIN', 'content-type': 'application/json; charset=UTF-8', '-content-encoding': 'gzip', 'x-xss-protection': '0', 'x-content-type-options': 'nosniff', 'content-length': '279', 'cache-control': 'private', 'server': 'ESF', 'transfer-encoding': 'chunked'}>, content <{
"error": {
"code": 404,
"message": "(80da9803cf5ada7a): Information about job 2019-07-29_11_01_20-1919703129159653506 could not be found in our system. Please double check the id is correct. If it is please contact customer support.",
"status": "NOT_FOUND"
}
}
>
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-29_10_33_16-4133702961387473119?project=apache-beam-testing.
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-29_10_43_23-1491068208893852088?project=apache-beam-testing.
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-29_10_53_15-16951228614865433416?project=apache-beam-testing.
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-29_11_03_12-8097370271356912039?project=apache-beam-testing.
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-29_11_14_20-752848600474614315?project=apache-beam-testing.
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-29_11_23_46-15325132052343334191?project=apache-beam-testing.
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-29_10_33_11-12766600169402754362?project=apache-beam-testing.
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-29_10_43_47-9046934534248456716?project=apache-beam-testing.
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-29_10_54_36-1559723213419036609?project=apache-beam-testing.
<https://builds.apache.org/job/beam_PostCommit_Python35/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1150: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-29_11_04_31-942841750339672512?project=apache-beam-testing.
<https://builds.apache.org/job/beam_PostCommit_Python35/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:638: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
----------------------------------------------------------------------
XML: nosetests-postCommitIT-df-py35.xml
----------------------------------------------------------------------
XML: <https://builds.apache.org/job/beam_PostCommit_Python35/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 42 tests in 3627.176s
FAILED (SKIP=5, failures=1)
> Task :sdks:python:test-suites:dataflow:py35:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://builds.apache.org/job/beam_PostCommit_Python35/ws/src/sdks/python/test-suites/dataflow/py35/build.gradle'> line: 49
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py35:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 1m 21s
63 actionable tasks: 46 executed, 17 from cache
Publishing build scan...
https://gradle.com/s/wbn7edooygkmu
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org