You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2019/08/03 00:50:03 UTC
Build failed in Jenkins: beam_PostCommit_Python36 #125
See <https://builds.apache.org/job/beam_PostCommit_Python36/125/display/redirect?page=changes>
Changes:
[heejong] [BEAM-7877] Change the log level when deleting unknown temoprary files
------------------------------------------
[...truncated 243.92 KB...]
"@type": "OutputReference",
"output_name": "out",
"step_name": "s21"
},
"serialized_fn": "ref_AppliedPTransform_FilterOutSpammers/MapToVoidKey0_61",
"user_name": "FilterOutSpammers/MapToVoidKey0"
}
}
],
"type": "JOB_TYPE_STREAMING"
}
root: INFO: Create job: <Job
createTime: '2019-08-02T23:59:17.233547Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2019-08-02_16_59_15-5237574317351094133'
location: 'us-central1'
name: 'beamapp-jenkins-0802235906-762093'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2019-08-02T23:59:17.233547Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
root: INFO: Created job with id: [2019-08-02_16_59_15-5237574317351094133]
root: INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-08-02_16_59_15-5237574317351094133?project=apache-beam-testing
root: INFO: Job 2019-08-02_16_59_15-5237574317351094133 is in state JOB_STATE_RUNNING
root: INFO: 2019-08-02T23:59:19.936Z: JOB_MESSAGE_DETAILED: Checking permissions granted to controller Service Account.
root: INFO: 2019-08-02T23:59:20.576Z: JOB_MESSAGE_BASIC: Worker configuration: n1-standard-4 in us-central1-a.
root: INFO: 2019-08-02T23:59:21.091Z: JOB_MESSAGE_DETAILED: Expanding SplittableParDo operations into optimizable parts.
root: INFO: 2019-08-02T23:59:21.096Z: JOB_MESSAGE_DETAILED: Expanding CollectionToSingleton operations into optimizable parts.
root: INFO: 2019-08-02T23:59:21.108Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
root: INFO: 2019-08-02T23:59:21.125Z: JOB_MESSAGE_DEBUG: Combiner lifting skipped for step FilterOutSpammers/_UnpickledSideInput(MapToVoidKey0.out.0)/GroupByKey: GroupByKey not followed by a combiner.
root: INFO: 2019-08-02T23:59:21.131Z: JOB_MESSAGE_DEBUG: Combiner lifting skipped for step CalculateSpammyUsers/ProcessAndFilter/_UnpickledSideInput(MapToVoidKey0.out.0)/GroupByKey: GroupByKey not followed by a combiner.
root: INFO: 2019-08-02T23:59:21.149Z: JOB_MESSAGE_DETAILED: Expanding SplittableProcessKeyed operations into optimizable parts.
root: INFO: 2019-08-02T23:59:21.160Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into streaming Read/Write steps
root: INFO: 2019-08-02T23:59:21.213Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
root: INFO: 2019-08-02T23:59:21.264Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
root: INFO: 2019-08-02T23:59:21.270Z: JOB_MESSAGE_DETAILED: Fusing consumer WindowIntoFixedWindows into AddEventTimestamps
root: INFO: 2019-08-02T23:59:21.273Z: JOB_MESSAGE_DETAILED: Fusing consumer ExtractUserScores into AddEventTimestamps
root: INFO: 2019-08-02T23:59:21.278Z: JOB_MESSAGE_DETAILED: Fusing consumer CalculateSpammyUsers/ProcessAndFilter/MapToVoidKey0 into CalculateSpammyUsers/CombineGlobally(MeanCombineFn)/UnKey
root: INFO: 2019-08-02T23:59:21.281Z: JOB_MESSAGE_DETAILED: Fusing consumer CalculateSpammyUsers/ProcessAndFilter/MapToVoidKey0 into CalculateSpammyUsers/CombineGlobally(MeanCombineFn)/UnKey
root: INFO: 2019-08-02T23:59:21.287Z: JOB_MESSAGE_DETAILED: Fusing consumer FilterOutSpammers/MapToVoidKey0 into CreateSpammersView/UnKey
root: INFO: 2019-08-02T23:59:21.289Z: JOB_MESSAGE_DETAILED: Fusing consumer FilterOutSpammers/MapToVoidKey0 into CreateSpammersView/UnKey
root: INFO: 2019-08-02T23:59:21.292Z: JOB_MESSAGE_DETAILED: Fusing consumer WindowIntoSessions into ExtractUserScores
root: INFO: 2019-08-02T23:59:21.296Z: JOB_MESSAGE_DETAILED: Fusing consumer UserFixedWindows into ExtractUserScores
root: INFO: 2019-08-02T23:59:21.300Z: JOB_MESSAGE_DETAILED: Fusing consumer CalculateSpammyUsers/Values into CalculateSpammyUsers/SumUsersScores/Combine
root: INFO: 2019-08-02T23:59:21.307Z: JOB_MESSAGE_DETAILED: Fusing consumer AddEventTimestamps into ParseGameEventFn
root: INFO: 2019-08-02T23:59:21.314Z: JOB_MESSAGE_DETAILED: Fusing consumer CalculateSpammyUsers/SumUsersScores/GroupByKey/MergeBuckets into CalculateSpammyUsers/SumUsersScores/GroupByKey/ReadStream
root: INFO: 2019-08-02T23:59:21.321Z: JOB_MESSAGE_DETAILED: Fusing consumer FilterOutSpammers/_UnpickledSideInput(MapToVoidKey0.out.0)/Values into FilterOutSpammers/_UnpickledSideInput(MapToVoidKey0.out.0)/GroupByKey/MergeBuckets
root: INFO: 2019-08-02T23:59:21.328Z: JOB_MESSAGE_DETAILED: Fusing consumer FilterOutSpammers/_UnpickledSideInput(MapToVoidKey0.out.0)/GroupByKey/MergeBuckets into FilterOutSpammers/_UnpickledSideInput(MapToVoidKey0.out.0)/GroupByKey/ReadStream
root: INFO: 2019-08-02T23:59:21.333Z: JOB_MESSAGE_DETAILED: Fusing consumer WriteAvgSessionLength/WriteToBigQuery/AppendDestination into WriteAvgSessionLength/ConvertToRow
root: INFO: 2019-08-02T23:59:21.339Z: JOB_MESSAGE_DETAILED: Fusing consumer WriteAvgSessionLength/ConvertToRow into FormatAvgSessionLength
root: INFO: 2019-08-02T23:59:21.344Z: JOB_MESSAGE_DETAILED: Fusing consumer FormatAvgSessionLength into CombineGlobally(MeanCombineFn)/UnKey
root: INFO: 2019-08-02T23:59:21.348Z: JOB_MESSAGE_DETAILED: Fusing consumer CombineGlobally(MeanCombineFn)/CombinePerKey/Combine into CombineGlobally(MeanCombineFn)/CombinePerKey/GroupByKey/MergeBuckets
root: INFO: 2019-08-02T23:59:21.352Z: JOB_MESSAGE_DETAILED: Fusing consumer CombineGlobally(MeanCombineFn)/CombinePerKey/GroupByKey/MergeBuckets into CombineGlobally(MeanCombineFn)/CombinePerKey/GroupByKey/ReadStream
root: INFO: 2019-08-02T23:59:21.356Z: JOB_MESSAGE_DETAILED: Fusing consumer CombineGlobally(MeanCombineFn)/CombinePerKey/GroupByKey/WriteStream into CombineGlobally(MeanCombineFn)/KeyWithVoid
root: INFO: 2019-08-02T23:59:21.360Z: JOB_MESSAGE_DETAILED: Fusing consumer CombineGlobally(MeanCombineFn)/KeyWithVoid into WindowToExtractSessionMean
root: INFO: 2019-08-02T23:59:21.363Z: JOB_MESSAGE_DETAILED: Fusing consumer WindowToExtractSessionMean into UserSessionActivity
root: INFO: 2019-08-02T23:59:21.366Z: JOB_MESSAGE_DETAILED: Fusing consumer UserSessionActivity into CombinePerKey(<lambda at game_stats.py:374>)/Combine
root: INFO: 2019-08-02T23:59:21.370Z: JOB_MESSAGE_DETAILED: Fusing consumer CombinePerKey(<lambda at game_stats.py:374>)/Combine into CombinePerKey(<lambda at game_stats.py:374>)/GroupByKey/MergeBuckets
root: INFO: 2019-08-02T23:59:21.373Z: JOB_MESSAGE_DETAILED: Fusing consumer WriteAvgSessionLength/WriteToBigQuery/StreamInsertRows/ParDo(BigQueryWriteFn) into WriteAvgSessionLength/WriteToBigQuery/AppendDestination
root: INFO: 2019-08-02T23:59:21.376Z: JOB_MESSAGE_DETAILED: Fusing consumer CombinePerKey(<lambda at game_stats.py:374>)/GroupByKey/WriteStream into WindowIntoSessions
root: INFO: 2019-08-02T23:59:21.380Z: JOB_MESSAGE_DETAILED: Fusing consumer CalculateSpammyUsers/CombineGlobally(MeanCombineFn)/UnKey into CalculateSpammyUsers/CombineGlobally(MeanCombineFn)/CombinePerKey/Combine
root: INFO: 2019-08-02T23:59:21.382Z: JOB_MESSAGE_DETAILED: Fusing consumer FilterOutSpammers/_UnpickledSideInput(MapToVoidKey0.out.0)/GroupByKey/WriteStream into FilterOutSpammers/_UnpickledSideInput(MapToVoidKey0.out.0)/PairWithVoidKey
root: INFO: 2019-08-02T23:59:21.388Z: JOB_MESSAGE_DETAILED: Fusing consumer FilterOutSpammers/_UnpickledSideInput(MapToVoidKey0.out.0)/PairWithVoidKey into FilterOutSpammers/MapToVoidKey0
root: INFO: 2019-08-02T23:59:21.391Z: JOB_MESSAGE_DETAILED: Fusing consumer WriteTeamScoreSums/WriteToBigQuery/StreamInsertRows/ParDo(BigQueryWriteFn) into WriteTeamScoreSums/WriteToBigQuery/AppendDestination
root: INFO: 2019-08-02T23:59:21.395Z: JOB_MESSAGE_DETAILED: Fusing consumer WriteTeamScoreSums/WriteToBigQuery/AppendDestination into WriteTeamScoreSums/ConvertToRow
root: INFO: 2019-08-02T23:59:21.399Z: JOB_MESSAGE_DETAILED: Fusing consumer WriteTeamScoreSums/ConvertToRow into TeamScoresDict
root: INFO: 2019-08-02T23:59:21.403Z: JOB_MESSAGE_DETAILED: Fusing consumer TeamScoresDict into ExtractAndSumScore/CombinePerKey(sum)/Combine
root: INFO: 2019-08-02T23:59:21.408Z: JOB_MESSAGE_DETAILED: Fusing consumer ExtractAndSumScore/CombinePerKey(sum)/GroupByKey/MergeBuckets into ExtractAndSumScore/CombinePerKey(sum)/GroupByKey/ReadStream
root: INFO: 2019-08-02T23:59:21.410Z: JOB_MESSAGE_DETAILED: Fusing consumer ExtractAndSumScore/CombinePerKey(sum)/GroupByKey/WriteStream into ExtractAndSumScore/Map(<lambda at game_stats.py:138>)
root: INFO: 2019-08-02T23:59:21.414Z: JOB_MESSAGE_DETAILED: Fusing consumer ExtractAndSumScore/Map(<lambda at game_stats.py:138>) into FilterOutSpammers/FilterOutSpammers
root: INFO: 2019-08-02T23:59:21.416Z: JOB_MESSAGE_DETAILED: Fusing consumer FilterOutSpammers/_UnpickledSideInput(MapToVoidKey0.out.0)/StreamingPCollectionViewWriter into FilterOutSpammers/_UnpickledSideInput(MapToVoidKey0.out.0)/Values
root: INFO: 2019-08-02T23:59:21.418Z: JOB_MESSAGE_DETAILED: Fusing consumer CalculateSpammyUsers/SumUsersScores/GroupByKey/WriteStream into UserFixedWindows
root: INFO: 2019-08-02T23:59:21.421Z: JOB_MESSAGE_DETAILED: Fusing consumer CalculateSpammyUsers/CombineGlobally(MeanCombineFn)/CombinePerKey/Combine into CalculateSpammyUsers/CombineGlobally(MeanCombineFn)/CombinePerKey/GroupByKey/MergeBuckets
root: INFO: 2019-08-02T23:59:21.423Z: JOB_MESSAGE_DETAILED: Fusing consumer CombineGlobally(MeanCombineFn)/UnKey into CombineGlobally(MeanCombineFn)/CombinePerKey/Combine
root: INFO: 2019-08-02T23:59:21.425Z: JOB_MESSAGE_DETAILED: Fusing consumer CreateSpammersView/UnKey into CreateSpammersView/CombinePerKey/Combine
root: INFO: 2019-08-02T23:59:21.430Z: JOB_MESSAGE_DETAILED: Fusing consumer CalculateSpammyUsers/CombineGlobally(MeanCombineFn)/CombinePerKey/GroupByKey/MergeBuckets into CalculateSpammyUsers/CombineGlobally(MeanCombineFn)/CombinePerKey/GroupByKey/ReadStream
root: INFO: 2019-08-02T23:59:21.433Z: JOB_MESSAGE_DETAILED: Fusing consumer CreateSpammersView/CombinePerKey/Combine into CreateSpammersView/CombinePerKey/GroupByKey/MergeBuckets
root: INFO: 2019-08-02T23:59:21.438Z: JOB_MESSAGE_DETAILED: Fusing consumer CreateSpammersView/CombinePerKey/GroupByKey/MergeBuckets into CreateSpammersView/CombinePerKey/GroupByKey/ReadStream
root: INFO: 2019-08-02T23:59:21.441Z: JOB_MESSAGE_DETAILED: Fusing consumer CombinePerKey(<lambda at game_stats.py:374>)/GroupByKey/MergeBuckets into CombinePerKey(<lambda at game_stats.py:374>)/GroupByKey/ReadStream
root: INFO: 2019-08-02T23:59:21.447Z: JOB_MESSAGE_DETAILED: Fusing consumer CreateSpammersView/KeyWithVoid into CalculateSpammyUsers/ProcessAndFilter/ProcessAndFilter
root: INFO: 2019-08-02T23:59:21.451Z: JOB_MESSAGE_DETAILED: Fusing consumer CalculateSpammyUsers/ProcessAndFilter/_UnpickledSideInput(MapToVoidKey0.out.0)/StreamingPCollectionViewWriter into CalculateSpammyUsers/ProcessAndFilter/_UnpickledSideInput(MapToVoidKey0.out.0)/Values
root: INFO: 2019-08-02T23:59:21.459Z: JOB_MESSAGE_DETAILED: Fusing consumer ExtractAndSumScore/CombinePerKey(sum)/Combine into ExtractAndSumScore/CombinePerKey(sum)/GroupByKey/MergeBuckets
root: INFO: 2019-08-02T23:59:21.465Z: JOB_MESSAGE_DETAILED: Fusing consumer CalculateSpammyUsers/ProcessAndFilter/_UnpickledSideInput(MapToVoidKey0.out.0)/GroupByKey/MergeBuckets into CalculateSpammyUsers/ProcessAndFilter/_UnpickledSideInput(MapToVoidKey0.out.0)/GroupByKey/ReadStream
root: INFO: 2019-08-02T23:59:21.471Z: JOB_MESSAGE_DETAILED: Fusing consumer CalculateSpammyUsers/ProcessAndFilter/_UnpickledSideInput(MapToVoidKey0.out.0)/GroupByKey/WriteStream into CalculateSpammyUsers/ProcessAndFilter/_UnpickledSideInput(MapToVoidKey0.out.0)/PairWithVoidKey
root: INFO: 2019-08-02T23:59:21.476Z: JOB_MESSAGE_DETAILED: Fusing consumer CalculateSpammyUsers/ProcessAndFilter/_UnpickledSideInput(MapToVoidKey0.out.0)/PairWithVoidKey into CalculateSpammyUsers/ProcessAndFilter/MapToVoidKey0
root: INFO: 2019-08-02T23:59:21.484Z: JOB_MESSAGE_DETAILED: Fusing consumer DecodeString into ReadPubSub/Read
root: INFO: 2019-08-02T23:59:21.487Z: JOB_MESSAGE_DETAILED: Fusing consumer CalculateSpammyUsers/CombineGlobally(MeanCombineFn)/CombinePerKey/GroupByKey/WriteStream into CalculateSpammyUsers/CombineGlobally(MeanCombineFn)/KeyWithVoid
root: INFO: 2019-08-02T23:59:21.491Z: JOB_MESSAGE_DETAILED: Fusing consumer CreateSpammersView/CombinePerKey/GroupByKey/WriteStream into CreateSpammersView/KeyWithVoid
root: INFO: 2019-08-02T23:59:21.497Z: JOB_MESSAGE_DETAILED: Fusing consumer CalculateSpammyUsers/CombineGlobally(MeanCombineFn)/KeyWithVoid into CalculateSpammyUsers/Values
root: INFO: 2019-08-02T23:59:21.504Z: JOB_MESSAGE_DETAILED: Fusing consumer ParseGameEventFn into DecodeString
root: INFO: 2019-08-02T23:59:21.510Z: JOB_MESSAGE_DETAILED: Fusing consumer CalculateSpammyUsers/ProcessAndFilter/_UnpickledSideInput(MapToVoidKey0.out.0)/Values into CalculateSpammyUsers/ProcessAndFilter/_UnpickledSideInput(MapToVoidKey0.out.0)/GroupByKey/MergeBuckets
root: INFO: 2019-08-02T23:59:21.514Z: JOB_MESSAGE_DETAILED: Fusing consumer CalculateSpammyUsers/SumUsersScores/Combine into CalculateSpammyUsers/SumUsersScores/GroupByKey/MergeBuckets
root: INFO: 2019-08-02T23:59:21.558Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
root: INFO: 2019-08-02T23:59:21.621Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
root: INFO: 2019-08-02T23:59:21.643Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
root: INFO: 2019-08-02T23:59:21.927Z: JOB_MESSAGE_DEBUG: Executing wait step start2
root: INFO: 2019-08-02T23:59:21.959Z: JOB_MESSAGE_DEBUG: Starting worker pool setup.
root: INFO: 2019-08-02T23:59:21.968Z: JOB_MESSAGE_BASIC: Starting 1 workers...
root: INFO: 2019-08-02T23:59:25.513Z: JOB_MESSAGE_BASIC: Executing operation CalculateSpammyUsers/ProcessAndFilter/_UnpickledSideInput(MapToVoidKey0.out.0)/GroupByKey/ReadStream+CalculateSpammyUsers/ProcessAndFilter/_UnpickledSideInput(MapToVoidKey0.out.0)/GroupByKey/MergeBuckets+CalculateSpammyUsers/ProcessAndFilter/_UnpickledSideInput(MapToVoidKey0.out.0)/Values+CalculateSpammyUsers/ProcessAndFilter/_UnpickledSideInput(MapToVoidKey0.out.0)/StreamingPCollectionViewWriter
root: INFO: 2019-08-02T23:59:25.518Z: JOB_MESSAGE_BASIC: Executing operation CalculateSpammyUsers/SumUsersScores/GroupByKey/ReadStream+CalculateSpammyUsers/SumUsersScores/GroupByKey/MergeBuckets+CalculateSpammyUsers/SumUsersScores/Combine+CalculateSpammyUsers/Values+CalculateSpammyUsers/CombineGlobally(MeanCombineFn)/KeyWithVoid+CalculateSpammyUsers/CombineGlobally(MeanCombineFn)/CombinePerKey/GroupByKey/WriteStream+CalculateSpammyUsers/SumUsersScores/Combine.out/FromValue/WriteStream
root: INFO: 2019-08-02T23:59:25.518Z: JOB_MESSAGE_BASIC: Executing operation CreateSpammersView/CombinePerKey/GroupByKey/ReadStream+CreateSpammersView/CombinePerKey/GroupByKey/MergeBuckets+CreateSpammersView/CombinePerKey/Combine+CreateSpammersView/UnKey+FilterOutSpammers/MapToVoidKey0+FilterOutSpammers/MapToVoidKey0+FilterOutSpammers/_UnpickledSideInput(MapToVoidKey0.out.0)/PairWithVoidKey+FilterOutSpammers/_UnpickledSideInput(MapToVoidKey0.out.0)/GroupByKey/WriteStream
root: INFO: 2019-08-02T23:59:25.518Z: JOB_MESSAGE_BASIC: Executing operation WindowIntoFixedWindows.out/FromValue/ReadStream+FilterOutSpammers/FilterOutSpammers+ExtractAndSumScore/Map(<lambda at game_stats.py:138>)+ExtractAndSumScore/CombinePerKey(sum)/GroupByKey/WriteStream
root: INFO: 2019-08-02T23:59:25.518Z: JOB_MESSAGE_BASIC: Executing operation CalculateSpammyUsers/SumUsersScores/Combine.out/FromValue/ReadStream+CalculateSpammyUsers/ProcessAndFilter/ProcessAndFilter+CreateSpammersView/KeyWithVoid+CreateSpammersView/CombinePerKey/GroupByKey/WriteStream
root: INFO: 2019-08-02T23:59:25.519Z: JOB_MESSAGE_BASIC: Executing operation CombinePerKey(<lambda at game_stats.py:374>)/GroupByKey/ReadStream+CombinePerKey(<lambda at game_stats.py:374>)/GroupByKey/MergeBuckets+CombinePerKey(<lambda at game_stats.py:374>)/Combine+UserSessionActivity+WindowToExtractSessionMean+CombineGlobally(MeanCombineFn)/KeyWithVoid+CombineGlobally(MeanCombineFn)/CombinePerKey/GroupByKey/WriteStream
root: INFO: 2019-08-02T23:59:25.519Z: JOB_MESSAGE_BASIC: Executing operation CalculateSpammyUsers/CombineGlobally(MeanCombineFn)/CombinePerKey/GroupByKey/ReadStream+CalculateSpammyUsers/CombineGlobally(MeanCombineFn)/CombinePerKey/GroupByKey/MergeBuckets+CalculateSpammyUsers/CombineGlobally(MeanCombineFn)/CombinePerKey/Combine+CalculateSpammyUsers/CombineGlobally(MeanCombineFn)/UnKey+CalculateSpammyUsers/ProcessAndFilter/MapToVoidKey0+CalculateSpammyUsers/ProcessAndFilter/MapToVoidKey0+CalculateSpammyUsers/ProcessAndFilter/_UnpickledSideInput(MapToVoidKey0.out.0)/PairWithVoidKey+CalculateSpammyUsers/ProcessAndFilter/_UnpickledSideInput(MapToVoidKey0.out.0)/GroupByKey/WriteStream
root: INFO: 2019-08-02T23:59:25.519Z: JOB_MESSAGE_BASIC: Executing operation FilterOutSpammers/_UnpickledSideInput(MapToVoidKey0.out.0)/GroupByKey/ReadStream+FilterOutSpammers/_UnpickledSideInput(MapToVoidKey0.out.0)/GroupByKey/MergeBuckets+FilterOutSpammers/_UnpickledSideInput(MapToVoidKey0.out.0)/Values+FilterOutSpammers/_UnpickledSideInput(MapToVoidKey0.out.0)/StreamingPCollectionViewWriter
root: INFO: 2019-08-02T23:59:25.519Z: JOB_MESSAGE_BASIC: Executing operation ExtractAndSumScore/CombinePerKey(sum)/GroupByKey/ReadStream+ExtractAndSumScore/CombinePerKey(sum)/GroupByKey/MergeBuckets+ExtractAndSumScore/CombinePerKey(sum)/Combine+TeamScoresDict+WriteTeamScoreSums/ConvertToRow+WriteTeamScoreSums/WriteToBigQuery/AppendDestination+WriteTeamScoreSums/WriteToBigQuery/StreamInsertRows/ParDo(BigQueryWriteFn)
root: INFO: 2019-08-02T23:59:25.525Z: JOB_MESSAGE_BASIC: Executing operation CombineGlobally(MeanCombineFn)/CombinePerKey/GroupByKey/ReadStream+CombineGlobally(MeanCombineFn)/CombinePerKey/GroupByKey/MergeBuckets+CombineGlobally(MeanCombineFn)/CombinePerKey/Combine+CombineGlobally(MeanCombineFn)/UnKey+FormatAvgSessionLength+WriteAvgSessionLength/ConvertToRow+WriteAvgSessionLength/WriteToBigQuery/AppendDestination+WriteAvgSessionLength/WriteToBigQuery/StreamInsertRows/ParDo(BigQueryWriteFn)
root: INFO: 2019-08-02T23:59:25.526Z: JOB_MESSAGE_BASIC: Executing operation ReadPubSub/Read+DecodeString+ParseGameEventFn+AddEventTimestamps+WindowIntoFixedWindows+ExtractUserScores+WindowIntoSessions+UserFixedWindows+CombinePerKey(<lambda at game_stats.py:374>)/GroupByKey/WriteStream+CalculateSpammyUsers/SumUsersScores/GroupByKey/WriteStream+WindowIntoFixedWindows.out/FromValue/WriteStream
root: INFO: 2019-08-02T23:59:50.162Z: JOB_MESSAGE_DEBUG: Executing input step topology_init_attach_disk_input_step
root: INFO: 2019-08-02T23:59:50.437Z: JOB_MESSAGE_DETAILED: Checking permissions granted to controller Service Account.
root: INFO: 2019-08-02T23:59:51.046Z: JOB_MESSAGE_BASIC: Worker configuration: n1-standard-4 in us-central1-a.
root: INFO: 2019-08-03T00:00:07.409Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
root: INFO: 2019-08-03T00:02:51.354Z: JOB_MESSAGE_BASIC: Worker configuration: n1-standard-4 in us-central1-a.
root: INFO: 2019-08-03T00:03:01.944Z: JOB_MESSAGE_ERROR: Workflow failed.
root: INFO: 2019-08-03T00:03:02.001Z: JOB_MESSAGE_BASIC: Finished operation WindowIntoFixedWindows.out/FromValue/ReadStream+FilterOutSpammers/FilterOutSpammers+ExtractAndSumScore/Map(<lambda at game_stats.py:138>)+ExtractAndSumScore/CombinePerKey(sum)/GroupByKey/WriteStream
root: INFO: 2019-08-03T00:03:02.001Z: JOB_MESSAGE_BASIC: Finished operation CalculateSpammyUsers/CombineGlobally(MeanCombineFn)/CombinePerKey/GroupByKey/ReadStream+CalculateSpammyUsers/CombineGlobally(MeanCombineFn)/CombinePerKey/GroupByKey/MergeBuckets+CalculateSpammyUsers/CombineGlobally(MeanCombineFn)/CombinePerKey/Combine+CalculateSpammyUsers/CombineGlobally(MeanCombineFn)/UnKey+CalculateSpammyUsers/ProcessAndFilter/MapToVoidKey0+CalculateSpammyUsers/ProcessAndFilter/MapToVoidKey0+CalculateSpammyUsers/ProcessAndFilter/_UnpickledSideInput(MapToVoidKey0.out.0)/PairWithVoidKey+CalculateSpammyUsers/ProcessAndFilter/_UnpickledSideInput(MapToVoidKey0.out.0)/GroupByKey/WriteStream
root: INFO: 2019-08-03T00:03:02.001Z: JOB_MESSAGE_BASIC: Finished operation CombinePerKey(<lambda at game_stats.py:374>)/GroupByKey/ReadStream+CombinePerKey(<lambda at game_stats.py:374>)/GroupByKey/MergeBuckets+CombinePerKey(<lambda at game_stats.py:374>)/Combine+UserSessionActivity+WindowToExtractSessionMean+CombineGlobally(MeanCombineFn)/KeyWithVoid+CombineGlobally(MeanCombineFn)/CombinePerKey/GroupByKey/WriteStream
root: INFO: 2019-08-03T00:03:02.001Z: JOB_MESSAGE_BASIC: Finished operation CombineGlobally(MeanCombineFn)/CombinePerKey/GroupByKey/ReadStream+CombineGlobally(MeanCombineFn)/CombinePerKey/GroupByKey/MergeBuckets+CombineGlobally(MeanCombineFn)/CombinePerKey/Combine+CombineGlobally(MeanCombineFn)/UnKey+FormatAvgSessionLength+WriteAvgSessionLength/ConvertToRow+WriteAvgSessionLength/WriteToBigQuery/AppendDestination+WriteAvgSessionLength/WriteToBigQuery/StreamInsertRows/ParDo(BigQueryWriteFn)
root: INFO: 2019-08-03T00:03:02.001Z: JOB_MESSAGE_BASIC: Finished operation CreateSpammersView/CombinePerKey/GroupByKey/ReadStream+CreateSpammersView/CombinePerKey/GroupByKey/MergeBuckets+CreateSpammersView/CombinePerKey/Combine+CreateSpammersView/UnKey+FilterOutSpammers/MapToVoidKey0+FilterOutSpammers/MapToVoidKey0+FilterOutSpammers/_UnpickledSideInput(MapToVoidKey0.out.0)/PairWithVoidKey+FilterOutSpammers/_UnpickledSideInput(MapToVoidKey0.out.0)/GroupByKey/WriteStream
root: INFO: 2019-08-03T00:03:02.002Z: JOB_MESSAGE_BASIC: Finished operation ReadPubSub/Read+DecodeString+ParseGameEventFn+AddEventTimestamps+WindowIntoFixedWindows+ExtractUserScores+WindowIntoSessions+UserFixedWindows+CombinePerKey(<lambda at game_stats.py:374>)/GroupByKey/WriteStream+CalculateSpammyUsers/SumUsersScores/GroupByKey/WriteStream+WindowIntoFixedWindows.out/FromValue/WriteStream
root: INFO: 2019-08-03T00:03:02.008Z: JOB_MESSAGE_BASIC: Finished operation CalculateSpammyUsers/SumUsersScores/Combine.out/FromValue/ReadStream+CalculateSpammyUsers/ProcessAndFilter/ProcessAndFilter+CreateSpammersView/KeyWithVoid+CreateSpammersView/CombinePerKey/GroupByKey/WriteStream
root: INFO: 2019-08-03T00:03:02.008Z: JOB_MESSAGE_BASIC: Finished operation FilterOutSpammers/_UnpickledSideInput(MapToVoidKey0.out.0)/GroupByKey/ReadStream+FilterOutSpammers/_UnpickledSideInput(MapToVoidKey0.out.0)/GroupByKey/MergeBuckets+FilterOutSpammers/_UnpickledSideInput(MapToVoidKey0.out.0)/Values+FilterOutSpammers/_UnpickledSideInput(MapToVoidKey0.out.0)/StreamingPCollectionViewWriter
root: INFO: 2019-08-03T00:03:02.008Z: JOB_MESSAGE_BASIC: Finished operation ExtractAndSumScore/CombinePerKey(sum)/GroupByKey/ReadStream+ExtractAndSumScore/CombinePerKey(sum)/GroupByKey/MergeBuckets+ExtractAndSumScore/CombinePerKey(sum)/Combine+TeamScoresDict+WriteTeamScoreSums/ConvertToRow+WriteTeamScoreSums/WriteToBigQuery/AppendDestination+WriteTeamScoreSums/WriteToBigQuery/StreamInsertRows/ParDo(BigQueryWriteFn)
root: INFO: 2019-08-03T00:03:02.008Z: JOB_MESSAGE_BASIC: Finished operation CalculateSpammyUsers/SumUsersScores/GroupByKey/ReadStream+CalculateSpammyUsers/SumUsersScores/GroupByKey/MergeBuckets+CalculateSpammyUsers/SumUsersScores/Combine+CalculateSpammyUsers/Values+CalculateSpammyUsers/CombineGlobally(MeanCombineFn)/KeyWithVoid+CalculateSpammyUsers/CombineGlobally(MeanCombineFn)/CombinePerKey/GroupByKey/WriteStream+CalculateSpammyUsers/SumUsersScores/Combine.out/FromValue/WriteStream
root: INFO: 2019-08-03T00:03:02.009Z: JOB_MESSAGE_BASIC: Finished operation CalculateSpammyUsers/ProcessAndFilter/_UnpickledSideInput(MapToVoidKey0.out.0)/GroupByKey/ReadStream+CalculateSpammyUsers/ProcessAndFilter/_UnpickledSideInput(MapToVoidKey0.out.0)/GroupByKey/MergeBuckets+CalculateSpammyUsers/ProcessAndFilter/_UnpickledSideInput(MapToVoidKey0.out.0)/Values+CalculateSpammyUsers/ProcessAndFilter/_UnpickledSideInput(MapToVoidKey0.out.0)/StreamingPCollectionViewWriter
root: INFO: 2019-08-03T00:03:02.164Z: JOB_MESSAGE_DETAILED: Cleaning up.
root: INFO: 2019-08-03T00:03:02.225Z: JOB_MESSAGE_BASIC: Worker pool stopped.
root: INFO: 2019-08-03T00:03:02.241Z: JOB_MESSAGE_DEBUG: Tearing down pending resources...
root: INFO: 2019-08-03T00:03:02.249Z: JOB_MESSAGE_DEBUG: Starting worker pool teardown.
root: INFO: 2019-08-03T00:03:02.263Z: JOB_MESSAGE_BASIC: Stopping worker pool...
root: INFO: 2019-08-03T00:05:50.439Z: JOB_MESSAGE_DETAILED: Checking permissions granted to controller Service Account.
root: INFO: 2019-08-03T00:05:51.032Z: JOB_MESSAGE_BASIC: Worker configuration: n1-standard-4 in us-central1-a.
root: INFO: 2019-08-03T00:06:37.608Z: JOB_MESSAGE_BASIC: Worker pool stopped.
root: INFO: Job 2019-08-02_16_59_15-5237574317351094133 is in state JOB_STATE_FAILED
google.auth.transport._http_client: DEBUG: Making request: GET http://169.254.169.254
google.auth.transport._http_client: DEBUG: Making request: GET http://metadata.google.internal/computeMetadata/v1/project/project-id
urllib3.util.retry: DEBUG: Converted retries value: 3 -> Retry(total=3, connect=None, read=None, redirect=None, status=None)
google.auth.transport.requests: DEBUG: Making request: GET http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/?recursive=true
urllib3.connectionpool: DEBUG: Starting new HTTP connection (1): metadata.google.internal:80
urllib3.connectionpool: DEBUG: http://metadata.google.internal:80 "GET /computeMetadata/v1/instance/service-accounts/default/?recursive=true HTTP/1.1" 200 144
google.auth.transport.requests: DEBUG: Making request: GET http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/844138762903-compute@developer.gserviceaccount.com/token
urllib3.connectionpool: DEBUG: http://metadata.google.internal:80 "GET /computeMetadata/v1/instance/service-accounts/844138762903-compute@developer.gserviceaccount.com/token HTTP/1.1" 200 176
urllib3.connectionpool: DEBUG: Starting new HTTPS connection (1): www.googleapis.com:443
urllib3.connectionpool: DEBUG: https://www.googleapis.com:443 "DELETE /bigquery/v2/projects/apache-beam-testing/datasets/game_stats_it_dataset15647903454734?deleteContents=true HTTP/1.1" 204 0
--------------------- >> end captured logging << ---------------------
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-08-02_16_59_15-5237574317351094133?project=apache-beam-testing.
<https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1145: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-08-02_17_06_57-6240930238880192690?project=apache-beam-testing.
experiments = p.options.view_as(DebugOptions).experiments or []
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-08-02_17_16_24-17227500894604525567?project=apache-beam-testing.
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-08-02_17_25_02-16304762929609261447?project=apache-beam-testing.
<https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:572: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
<https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/fileio_test.py>:232: FutureWarning: MatchAll is experimental.
| 'GetPath' >> beam.Map(lambda metadata: metadata.path))
<https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/fileio_test.py>:243: FutureWarning: MatchAll is experimental.
| 'Checksums' >> beam.Map(compute_hash))
<https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/fileio_test.py>:243: FutureWarning: ReadMatches is experimental.
| 'Checksums' >> beam.Map(compute_hash))
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-08-02_16_59_12-5424155622884951817?project=apache-beam-testing.
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-08-02_17_21_30-1240453820890898516?project=apache-beam-testing.
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-08-02_17_29_52-12436556650842695919?project=apache-beam-testing.
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-08-02_17_37_55-7571970429594611487?project=apache-beam-testing.
<https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py>:695: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=transform.kms_key))
<https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1145: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-08-02_16_59_14-15351413865709239865?project=apache-beam-testing.
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-08-02_17_12_30-940094071724050972?project=apache-beam-testing.
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-08-02_17_20_51-14534886349008326060?project=apache-beam-testing.
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-08-02_17_29_12-360527994168535258?project=apache-beam-testing.
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-08-02_16_59_13-11220016134016623822?project=apache-beam-testing.
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-08-02_17_22_40-11470863786815100166?project=apache-beam-testing.
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-08-02_17_30_53-13762247642179254617?project=apache-beam-testing.
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-08-02_16_59_12-6027138758792993446?project=apache-beam-testing.
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-08-02_17_08_53-14545471349992964790?project=apache-beam-testing.
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-08-02_17_17_10-16626805096610807813?project=apache-beam-testing.
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-08-02_17_25_58-6444037675998809730?project=apache-beam-testing.
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-08-02_17_34_24-1654418361259797919?project=apache-beam-testing.
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-08-02_16_59_12-12919427812658234368?project=apache-beam-testing.
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-08-02_17_07_18-9461337746435191738?project=apache-beam-testing.
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-08-02_17_15_24-1661951621427522840?project=apache-beam-testing.
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-08-02_17_24_26-15705546279864806539?project=apache-beam-testing.
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-08-02_17_33_12-3404148904329367100?project=apache-beam-testing.
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-08-02_17_41_53-7111764932666666762?project=apache-beam-testing.
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-08-02_16_59_14-17912452092272033518?project=apache-beam-testing.
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-08-02_17_10_06-6060132283684691533?project=apache-beam-testing.
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-08-02_17_19_12-4986150420367570567?project=apache-beam-testing.
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-08-02_17_28_21-1394770922314569581?project=apache-beam-testing.
<https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py>:695: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=transform.kms_key))
<https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:73: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=kms_key))
<https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py>:695: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=transform.kms_key))
<https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1145: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
<https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:642: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-08-02_16_59_13-8768102259979260566?project=apache-beam-testing.
<https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1145: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
<https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:642: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-08-02_17_09_22-3531684497575492489?project=apache-beam-testing.
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-08-02_17_20_39-744401494917671613?project=apache-beam-testing.
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-08-02_17_29_11-14173851463182203689?project=apache-beam-testing.
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-08-02_17_38_17-7476741002522797296?project=apache-beam-testing.
----------------------------------------------------------------------
XML: nosetests-postCommitIT-df-py36.xml
----------------------------------------------------------------------
XML: <https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 42 tests in 3059.545s
FAILED (SKIP=5, errors=1)
> Task :sdks:python:test-suites:dataflow:py36:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/dataflow/py36/build.gradle'> line: 49
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 51m 53s
63 actionable tasks: 46 executed, 17 from cache
Publishing build scan...
https://gradle.com/s/hrvfxoqof4n6a
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Jenkins build is back to normal : beam_PostCommit_Python36 #126
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Python36/126/display/redirect>
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org