You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@beam.apache.org by Ahmet Altay <al...@google.com.INVALID> on 2017/01/21 03:31:56 UTC
Re: Build failed in Jenkins: beam_PostCommit_Python_Verify #1058
Opened: https://issues.apache.org/jira/browse/BEAM-1293
I am looking at it, it is caused by:
https://github.com/apache/beam/pull/1802
On Fri, Jan 20, 2017 at 6:56 PM, Apache Jenkins Server <
jenkins@builds.apache.org> wrote:
> See <https://builds.apache.org/job/beam_PostCommit_Python_
> Verify/1058/changes>
>
> Changes:
>
> [robertwb] Make TestPipeline.run fail when the underlying execution fails.
>
> [robertwb] DataflowRunner will raise an exception on failures.
>
> ------------------------------------------
> [...truncated 18920 lines...]
> }
> },
> {
> "kind": "GroupByKey",
> "name": "s8",
> "properties": {
> "display_data": [],
> "output_info": [
> {
> "encoding": {
> "@type": "kind:windowed_value",
> "component_encodings": [
> {
> "@type": "kind:pair",
> "component_encodings": [
> {
> "@type": "FastPrimitivesCoder$
> eNprYEpOLEhMzkiNT0pNzNVLzk9JLSqGUlxuicUlAUWZuZklmWWpxc4gQa5C
> Bs3GQsbaQqZQ/vi0xJycpMTk7Hiw+kJmPEYFZCZn56RCjWABGsFaW8iWVJykBwDlGS3/",
> "component_encodings": [
> {
> "@type": "FastPrimitivesCoder$
> eNprYEpOLEhMzkiNT0pNzNVLzk9JLSqGUlxuicUlAUWZuZklmWWpxc4gQa5C
> Bs3GQsbaQqZQ/vi0xJycpMTk7Hiw+kJmPEYFZCZn56RCjWABGsFaW8iWVJykBwDlGS3/",
> "component_encodings": []
> },
> {
> "@type": "FastPrimitivesCoder$
> eNprYEpOLEhMzkiNT0pNzNVLzk9JLSqGUlxuicUlAUWZuZklmWWpxc4gQa5C
> Bs3GQsbaQqZQ/vi0xJycpMTk7Hiw+kJmPEYFZCZn56RCjWABGsFaW8iWVJykBwDlGS3/",
> "component_encodings": []
> }
> ],
> "is_pair_like": true
> },
> {
> "@type": "FastPrimitivesCoder$
> eNprYEpOLEhMzkiNT0pNzNVLzk9JLSqGUlxuicUlAUWZuZklmWWpxc4gQa5C
> Bs3GQsbaQqZQ/vi0xJycpMTk7Hiw+kJmPEYFZCZn56RCjWABGsFaW8iWVJykBwDlGS3/",
> "component_encodings": [
> {
> "@type": "FastPrimitivesCoder$
> eNprYEpOLEhMzkiNT0pNzNVLzk9JLSqGUlxuicUlAUWZuZklmWWpxc4gQa5C
> Bs3GQsbaQqZQ/vi0xJycpMTk7Hiw+kJmPEYFZCZn56RCjWABGsFaW8iWVJykBwDlGS3/",
> "component_encodings": []
> },
> {
> "@type": "FastPrimitivesCoder$
> eNprYEpOLEhMzkiNT0pNzNVLzk9JLSqGUlxuicUlAUWZuZklmWWpxc4gQa5C
> Bs3GQsbaQqZQ/vi0xJycpMTk7Hiw+kJmPEYFZCZn56RCjWABGsFaW8iWVJykBwDlGS3/",
> "component_encodings": []
> }
> ],
> "is_pair_like": true
> }
> ],
> "is_pair_like": true
> },
> {
> "@type": "kind:global_window"
> }
> ],
> "is_wrapper": true
> },
> "output_name": "out",
> "user_name": "assert_that/Group.out"
> }
> ],
> "parallel_input": {
> "@type": "OutputReference",
> "output_name": "out",
> "step_name": "s7"
> },
> "serialized_fn": "<string of 252 bytes>",
> "user_name": "assert_that/Group"
> }
> },
> {
> "kind": "ParallelDo",
> "name": "s9",
> "properties": {
> "display_data": [
> {
> "key": "fn",
> "label": "Transform Function",
> "namespace": "apache_beam.transforms.core.
> CallableWrapperDoFn",
> "type": "STRING",
> "value": "<lambda>"
> },
> {
> "key": "fn",
> "label": "Transform Function",
> "namespace": "apache_beam.transforms.core.ParDo",
> "shortValue": "CallableWrapperDoFn",
> "type": "STRING",
> "value": "apache_beam.transforms.core.CallableWrapperDoFn"
> }
> ],
> "non_parallel_inputs": {},
> "output_info": [
> {
> "encoding": {
> "@type": "kind:windowed_value",
> "component_encodings": [
> {
> "@type": "FastPrimitivesCoder$
> eNprYEpOLEhMzkiNT0pNzNVLzk9JLSqGUlxuicUlAUWZuZklmWWpxc4gQa5C
> Bs3GQsbaQqZQ/vi0xJycpMTk7Hiw+kJmPEYFZCZn56RCjWABGsFaW8iWVJykBwDlGS3/",
> "component_encodings": [
> {
> "@type": "FastPrimitivesCoder$
> eNprYEpOLEhMzkiNT0pNzNVLzk9JLSqGUlxuicUlAUWZuZklmWWpxc4gQa5C
> Bs3GQsbaQqZQ/vi0xJycpMTk7Hiw+kJmPEYFZCZn56RCjWABGsFaW8iWVJykBwDlGS3/",
> "component_encodings": []
> },
> {
> "@type": "FastPrimitivesCoder$
> eNprYEpOLEhMzkiNT0pNzNVLzk9JLSqGUlxuicUlAUWZuZklmWWpxc4gQa5C
> Bs3GQsbaQqZQ/vi0xJycpMTk7Hiw+kJmPEYFZCZn56RCjWABGsFaW8iWVJykBwDlGS3/",
> "component_encodings": []
> }
> ],
> "is_pair_like": true
> },
> {
> "@type": "kind:global_window"
> }
> ],
> "is_wrapper": true
> },
> "output_name": "out",
> "user_name": "assert_that/UnKey.out"
> }
> ],
> "parallel_input": {
> "@type": "OutputReference",
> "output_name": "out",
> "step_name": "s8"
> },
> "serialized_fn": "<string of 1060 bytes>",
> "user_name": "assert_that/UnKey"
> }
> },
> {
> "kind": "ParallelDo",
> "name": "s10",
> "properties": {
> "display_data": [
> {
> "key": "fn",
> "label": "Transform Function",
> "namespace": "apache_beam.transforms.core.
> CallableWrapperDoFn",
> "type": "STRING",
> "value": "_equal"
> },
> {
> "key": "fn",
> "label": "Transform Function",
> "namespace": "apache_beam.transforms.core.ParDo",
> "shortValue": "CallableWrapperDoFn",
> "type": "STRING",
> "value": "apache_beam.transforms.core.CallableWrapperDoFn"
> }
> ],
> "non_parallel_inputs": {},
> "output_info": [
> {
> "encoding": {
> "@type": "kind:windowed_value",
> "component_encodings": [
> {
> "@type": "FastPrimitivesCoder$
> eNprYEpOLEhMzkiNT0pNzNVLzk9JLSqGUlxuicUlAUWZuZklmWWpxc4gQa5C
> Bs3GQsbaQqZQ/vi0xJycpMTk7Hiw+kJmPEYFZCZn56RCjWABGsFaW8iWVJykBwDlGS3/",
> "component_encodings": [
> {
> "@type": "FastPrimitivesCoder$
> eNprYEpOLEhMzkiNT0pNzNVLzk9JLSqGUlxuicUlAUWZuZklmWWpxc4gQa5C
> Bs3GQsbaQqZQ/vi0xJycpMTk7Hiw+kJmPEYFZCZn56RCjWABGsFaW8iWVJykBwDlGS3/",
> "component_encodings": []
> },
> {
> "@type": "FastPrimitivesCoder$
> eNprYEpOLEhMzkiNT0pNzNVLzk9JLSqGUlxuicUlAUWZuZklmWWpxc4gQa5C
> Bs3GQsbaQqZQ/vi0xJycpMTk7Hiw+kJmPEYFZCZn56RCjWABGsFaW8iWVJykBwDlGS3/",
> "component_encodings": []
> }
> ],
> "is_pair_like": true
> },
> {
> "@type": "kind:global_window"
> }
> ],
> "is_wrapper": true
> },
> "output_name": "out",
> "user_name": "assert_that/Match.out"
> }
> ],
> "parallel_input": {
> "@type": "OutputReference",
> "output_name": "out",
> "step_name": "s9"
> },
> "serialized_fn": "<string of 1248 bytes>",
> "user_name": "assert_that/Match"
> }
> }
> ],
> "type": "JOB_TYPE_BATCH"
> }
> root: INFO: Create job: <Job
> id: u'2017-01-20_18_52_25-2593466258290409747'
> projectId: u'apache-beam-testing'
> steps: []
> tempFiles: []
> type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
> root: INFO: Created job with id: [2017-01-20_18_52_25-2593466258290409747]
> root: INFO: To access the Dataflow monitoring console, please navigate to
> https://console.developers.google.com/project/apache-
> beam-testing/dataflow/job/2017-01-20_18_52_25-2593466258290409747
> root: INFO: Job 2017-01-20_18_52_25-2593466258290409747 is in state
> JOB_STATE_RUNNING
> root: INFO: 2017-01-20_18_52_25-2593466258290409747_00000159bef0745a:
> 2017-01-21T02:52:26.586Z: JOB_MESSAGE_DETAILED: (dd4fb6e4e691a70): Checking
> required Cloud APIs are enabled.
> root: INFO: 2017-01-20_18_52_25-2593466258290409747_00000159bef076ed:
> 2017-01-21T02:52:27.245Z: JOB_MESSAGE_DEBUG: (dd4fb6e4e691389): Combiner
> lifting skipped for step assert_that/Group: GroupByKey not followed by a
> combiner.
> root: INFO: 2017-01-20_18_52_25-2593466258290409747_00000159bef076f0:
> 2017-01-21T02:52:27.248Z: JOB_MESSAGE_DEBUG: (dd4fb6e4e691cab): Combiner
> lifting skipped for step CountWords/GroupCounts: GroupByKey not followed by
> a combiner.
> root: INFO: 2017-01-20_18_52_25-2593466258290409747_00000159bef076f2:
> 2017-01-21T02:52:27.250Z: JOB_MESSAGE_DETAILED: (dd4fb6e4e6915cd):
> Expanding GroupByKey operations into optimizable parts.
> root: INFO: 2017-01-20_18_52_25-2593466258290409747_00000159bef076f4:
> 2017-01-21T02:52:27.252Z: JOB_MESSAGE_DETAILED: (dd4fb6e4e691eef): Lifting
> ValueCombiningMappingFns into MergeBucketsMappingFns
> root: INFO: 2017-01-20_18_52_25-2593466258290409747_00000159bef076fa:
> 2017-01-21T02:52:27.258Z: JOB_MESSAGE_DETAILED: (dd4fb6e4e691a55):
> Annotating graph with Autotuner information.
> root: INFO: 2017-01-20_18_52_25-2593466258290409747_00000159bef07703:
> 2017-01-21T02:52:27.267Z: JOB_MESSAGE_DETAILED: (dd4fb6e4e69112a): Fusing
> adjacent ParDo, Read, Write, and Flatten operations
> root: INFO: 2017-01-20_18_52_25-2593466258290409747_00000159bef07707:
> 2017-01-21T02:52:27.271Z: JOB_MESSAGE_DETAILED: (dd4fb6e4e69136e): Fusing
> consumer CountWords/GroupCounts/Reify into CountWords/AddCount
> root: INFO: 2017-01-20_18_52_25-2593466258290409747_00000159bef0770a:
> 2017-01-21T02:52:27.274Z: JOB_MESSAGE_DETAILED: (dd4fb6e4e691c90): Fusing
> consumer assert_that/WindowInto into CountWords/AddCounts
> root: INFO: 2017-01-20_18_52_25-2593466258290409747_00000159bef0770c:
> 2017-01-21T02:52:27.276Z: JOB_MESSAGE_DETAILED: (dd4fb6e4e6915b2): Fusing
> consumer CountWords/AddCounts into CountWords/GroupCounts/GroupByWindow
> root: INFO: 2017-01-20_18_52_25-2593466258290409747_00000159bef0770e:
> 2017-01-21T02:52:27.278Z: JOB_MESSAGE_DETAILED: (dd4fb6e4e691ed4): Fusing
> consumer assert_that/UnKey into assert_that/Group/GroupByWindow
> root: INFO: 2017-01-20_18_52_25-2593466258290409747_00000159bef07710:
> 2017-01-21T02:52:27.280Z: JOB_MESSAGE_DETAILED: (dd4fb6e4e6917f6): Fusing
> consumer assert_that/Match into assert_that/UnKey
> root: INFO: 2017-01-20_18_52_25-2593466258290409747_00000159bef07712:
> 2017-01-21T02:52:27.282Z: JOB_MESSAGE_DETAILED: (dd4fb6e4e691118): Fusing
> consumer assert_that/Group/GroupByWindow into assert_that/Group/Read
> root: INFO: 2017-01-20_18_52_25-2593466258290409747_00000159bef07714:
> 2017-01-21T02:52:27.284Z: JOB_MESSAGE_DETAILED: (dd4fb6e4e691a3a): Fusing
> consumer assert_that/Group/Write into assert_that/Group/Reify
> root: INFO: 2017-01-20_18_52_25-2593466258290409747_00000159bef07717:
> 2017-01-21T02:52:27.287Z: JOB_MESSAGE_DETAILED: (dd4fb6e4e69135c): Fusing
> consumer assert_that/Group/Reify into assert_that/ToVoidKey
> root: INFO: 2017-01-20_18_52_25-2593466258290409747_00000159bef07719:
> 2017-01-21T02:52:27.289Z: JOB_MESSAGE_DETAILED: (dd4fb6e4e691c7e): Fusing
> consumer assert_that/ToVoidKey into assert_that/WindowInto
> root: INFO: 2017-01-20_18_52_25-2593466258290409747_00000159bef0771b:
> 2017-01-21T02:52:27.291Z: JOB_MESSAGE_DETAILED: (dd4fb6e4e6915a0): Fusing
> consumer CountWords/AddCount into GetWords
> root: INFO: 2017-01-20_18_52_25-2593466258290409747_00000159bef0771d:
> 2017-01-21T02:52:27.293Z: JOB_MESSAGE_DETAILED: (dd4fb6e4e691ec2): Fusing
> consumer CountWords/GroupCounts/Write into CountWords/GroupCounts/Reify
> root: INFO: 2017-01-20_18_52_25-2593466258290409747_00000159bef0771f:
> 2017-01-21T02:52:27.295Z: JOB_MESSAGE_DETAILED: (dd4fb6e4e6917e4): Fusing
> consumer CountWords/GroupCounts/GroupByWindow into
> CountWords/GroupCounts/Read
> root: INFO: 2017-01-20_18_52_25-2593466258290409747_00000159bef0774c:
> 2017-01-21T02:52:27.340Z: JOB_MESSAGE_DEBUG: (dd4fb6e4e6919f2): Workflow
> config is missing a default resource spec.
> root: INFO: 2017-01-20_18_52_25-2593466258290409747_00000159bef0774e:
> 2017-01-21T02:52:27.342Z: JOB_MESSAGE_DETAILED: (dd4fb6e4e691314): Adding
> StepResource setup and teardown to workflow graph.
> root: INFO: 2017-01-20_18_52_25-2593466258290409747_00000159bef0775b:
> 2017-01-21T02:52:27.355Z: JOB_MESSAGE_DEBUG: (21352c0bda4bef0c): Adding
> workflow start and stop steps.
> root: INFO: 2017-01-20_18_52_25-2593466258290409747_00000159bef07764:
> 2017-01-21T02:52:27.364Z: JOB_MESSAGE_DEBUG: (164d31e485c40469): Assigning
> stage ids.
> root: INFO: 2017-01-20_18_52_25-2593466258290409747_00000159bef07802:
> 2017-01-21T02:52:27.522Z: JOB_MESSAGE_DEBUG: (dd4fb6e4e691558): Executing
> wait step start2
> root: INFO: 2017-01-20_18_52_25-2593466258290409747_00000159bef0780a:
> 2017-01-21T02:52:27.530Z: JOB_MESSAGE_DEBUG: (d3b021bc4e6fe578): Executing
> operation SomeWords
> root: INFO: 2017-01-20_18_52_25-2593466258290409747_00000159bef07815:
> 2017-01-21T02:52:27.541Z: JOB_MESSAGE_DEBUG: (67fe3466d6ec64e5): Value
> "SomeWords.out" materialized.
> root: INFO: 2017-01-20_18_52_25-2593466258290409747_00000159bef0781f:
> 2017-01-21T02:52:27.551Z: JOB_MESSAGE_BASIC: S01: (6b5dd00f8d89d3c0):
> Executing operation CountWords/GroupCounts/Create
> root: INFO: 2017-01-20_18_52_25-2593466258290409747_00000159bef078ea:
> 2017-01-21T02:52:27.754Z: JOB_MESSAGE_DEBUG: (1a4e68d4be5c4255): Starting
> worker pool setup.
> root: INFO: 2017-01-20_18_52_25-2593466258290409747_00000159bef078ec:
> 2017-01-21T02:52:27.756Z: JOB_MESSAGE_BASIC: (1a4e68d4be5c433f): Starting 1
> workers...
> root: INFO: 2017-01-20_18_52_25-2593466258290409747_00000159bef078fa:
> 2017-01-21T02:52:27.770Z: JOB_MESSAGE_DEBUG: (cf0fca63dd5368f5): Value
> "CountWords/GroupCounts/Session" materialized.
> root: INFO: 2017-01-20_18_52_25-2593466258290409747_00000159bef07904:
> 2017-01-21T02:52:27.780Z: JOB_MESSAGE_BASIC: S02: (9b825cc75d0cf044):
> Executing operation GetWords+CountWords/AddCount+
> CountWords/GroupCounts/Reify+CountWords/GroupCounts/Write
> root: INFO: 2017-01-20_18_52_25-2593466258290409747_00000159bef1456c:
> 2017-01-21T02:53:20.108Z: JOB_MESSAGE_DETAILED: (400b30140836b17e): Workers
> have started successfully.
> root: INFO: 2017-01-20_18_52_25-2593466258290409747_00000159bef33807:
> 2017-01-21T02:55:27.751Z: JOB_MESSAGE_BASIC: S03: (cdee436503947f25):
> Executing operation CountWords/GroupCounts/Close
> root: INFO: 2017-01-20_18_52_25-2593466258290409747_00000159bef33820:
> 2017-01-21T02:55:27.776Z: JOB_MESSAGE_BASIC: S04: (275f2f1044a83018):
> Executing operation assert_that/Group/Create
> root: INFO: 2017-01-20_18_52_25-2593466258290409747_00000159bef338d1:
> 2017-01-21T02:55:27.953Z: JOB_MESSAGE_DEBUG: (cf0fca63dd536874): Value
> "assert_that/Group/Session" materialized.
> root: INFO: 2017-01-20_18_52_25-2593466258290409747_00000159bef338dd:
> 2017-01-21T02:55:27.965Z: JOB_MESSAGE_BASIC: S05: (cdee436503947546):
> Executing operation CountWords/GroupCounts/Read+CountWords/GroupCounts/
> GroupByWindow+CountWords/AddCounts+assert_that/WindowInto+assert_that/
> ToVoidKey+assert_that/Group/Reify+assert_that/Group/Write
> root: INFO: 2017-01-20_18_52_25-2593466258290409747_00000159bef33bae:
> 2017-01-21T02:55:28.686Z: JOB_MESSAGE_BASIC: S06: (3494be37f5547273):
> Executing operation assert_that/Group/Close
> root: INFO: 2017-01-20_18_52_25-2593466258290409747_00000159bef33bc3:
> 2017-01-21T02:55:28.707Z: JOB_MESSAGE_BASIC: S07: (3dc0ebbed1ad784e):
> Executing operation assert_that/Group/Read+assert_
> that/Group/GroupByWindow+assert_that/UnKey+assert_that/Match
> root: INFO: 2017-01-20_18_52_25-2593466258290409747_00000159bef33d94:
> 2017-01-21T02:55:29.172Z: JOB_MESSAGE_DEBUG: (67fe3466d6ec63b6): Executing
> success step success0
> root: INFO: 2017-01-20_18_52_25-2593466258290409747_00000159bef33dde:
> 2017-01-21T02:55:29.246Z: JOB_MESSAGE_DETAILED: (d3b021bc4e6fe6c5):
> Cleaning up.
> root: INFO: 2017-01-20_18_52_25-2593466258290409747_00000159bef33e52:
> 2017-01-21T02:55:29.362Z: JOB_MESSAGE_DEBUG: (d3b021bc4e6fe4db): Starting
> worker pool teardown.
> root: INFO: 2017-01-20_18_52_25-2593466258290409747_00000159bef33e54:
> 2017-01-21T02:55:29.364Z: JOB_MESSAGE_BASIC: (d3b021bc4e6fe2f1): Stopping
> worker pool...
> root: INFO: 2017-01-20_18_52_25-2593466258290409747_00000159bef43c43:
> 2017-01-21T02:56:34.371Z: JOB_MESSAGE_BASIC: (d3b021bc4e6fe812): Worker
> pool stopped.
> root: INFO: 2017-01-20_18_52_25-2593466258290409747_00000159bef43c59:
> 2017-01-21T02:56:34.393Z: JOB_MESSAGE_DEBUG: (d3b021bc4e6fe254): Tearing
> down pending resources...
> root: INFO: Job 2017-01-20_18_52_25-2593466258290409747 is in state
> JOB_STATE_DONE
> --------------------- >> end captured logging << ---------------------
>
> ----------------------------------------------------------------------
> Ran 20 tests in 5343.428s
>
> FAILED (errors=19)
> Build step 'Execute shell' marked build as failure
>