You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "Ankur Goenka (JIRA)" <ji...@apache.org> on 2019/05/01 20:33:00 UTC
[jira] [Resolved] (BEAM-6051) Tensor flow Chicago taxi failing for
256MB data failing with disk exhausted
[ https://issues.apache.org/jira/browse/BEAM-6051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ankur Goenka resolved BEAM-6051.
--------------------------------
Resolution: Cannot Reproduce
> Tensor flow Chicago taxi failing for 256MB data failing with disk exhausted
> ---------------------------------------------------------------------------
>
> Key: BEAM-6051
> URL: https://issues.apache.org/jira/browse/BEAM-6051
> Project: Beam
> Issue Type: Bug
> Components: java-fn-execution, runner-flink
> Reporter: Ankur Goenka
> Assignee: Ankur Goenka
> Priority: Major
> Fix For: Not applicable
>
>
> Chicago taxi example is failing when running on 1 node cluster with 256MB data.
> Disk usage:
> {code:java}
> goenka@goenka:/home/build$ ls -lha /tmp/flink-io-b9f13afc-0c5a-40ab-9a29-037d35068c1c/
> total 8.5G
> drwxr-xr-x 2 goenka primarygroup 4.0K Nov 12 20:27 .
> drwxrwxrwt 37 root root 11M Nov 12 20:27 ..
> -rw-r--r-- 1 goenka primarygroup 2.2G Nov 12 20:27 550b3393ce2a4c35ba37135c20ebccb3.channel
> -rw-r--r-- 1 goenka primarygroup 2.2G Nov 12 20:27 573f0fcb90732308857025108ffc74f6.channel
> -rw-r--r-- 1 goenka primarygroup 2.2G Nov 12 20:27 fb08a9ea467a1045f6087088b395ea8d.channel
> -rw-r--r-- 1 goenka primarygroup 2.2G Nov 12 20:27 fcca6cbf619f712d852d2371d9cb7046.channel
> {code}
> Error in logs:
> {code:java}
> 2018-11-12 20:45:09,344 ERROR org.apache.flink.runtime.operators.BatchTask - Error in task code: GroupReduce (GroupReduce at GenerateStatistics/RunStatsGenerators/CommonStatsGenerator/CombinePerKey/GroupByKey) (1/1) java.lang.Exception: The data preparation for task 'GroupReduce (GroupReduce at GenerateStatistics/RunStatsGenerators/CommonStatsGenerator/CombinePerKey/GroupByKey)' , caused an error: Error obtaining the sorted input: Thread 'SortMerger Reading Thread' terminated due to an exception: The record exceeds the maximum s ize of a sort buffer (current maximum: 30310400 bytes). at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:479) at org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:368) at org.apache.flink.runtime.taskmanager.Task.run(Task.java:712) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.RuntimeException: Error obtaining the sorted input: Thread 'SortMerger Reading Thread' terminated due to an exception: The record exceeds the maximum size of a sort buffer (current maximum: 30310400 bytes). at org.apache.flink.runtime.operators.sort.UnilateralSortMerger.getIterator(UnilateralSortMerger.java:650) at org.apache.flink.runtime.operators.BatchTask.getInput(BatchTask.java:1108) at org.apache.flink.runtime.operators.GroupReduceDriver.prepare(GroupReduceDriver.java:99) at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:473) ... 3 more Caused by: java.io.IOException: Thread 'SortMerger Reading Thread' terminated due to an exception: The record exceeds the maximum size of a sort buffer (current maximum: 30310400 bytes). at org.apache.flink.runtime.operators.sort.UnilateralSortMerger$ThreadBase.run(UnilateralSortMerger.java:831) Caused by: java.io.IOException: The record exceeds the maximum size of a sort buffer (current maximum: 30310400 bytes). at org.apache.flink.runtime.operators.sort.UnilateralSortMerger$ReadingThread.go(UnilateralSortMerger.java:986) at org.apache.flink.runtime.operators.sort.UnilateralSortMerger$ThreadBase.run(UnilateralSortMerger.java:827) 2018-11-12 20:45:09,348 INFO org.apache.flink.runtime.taskmanager.Task - GroupReduce (GroupReduce at GenerateStatistics/RunStatsGenerators/CommonStatsGenerator/CombinePerKey/GroupByKey) (1/1) (dcd3891fa9540bb4bf594a1a3d631e2e) switched from RUNNING to FAILED. java.lang.Exception: The data preparation for task 'GroupReduce (GroupReduce at GenerateStatistics/RunStatsGenerators/CommonStatsGenerator/CombinePerKey/GroupByKey)' , caused an error: Error obtaining the sorted input: Thread 'SortMerger Reading Thread' terminated due to an exception: The record exceeds the maximum s ize of a sort buffer (current maximum: 30310400 bytes). at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:479) at org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:368) at org.apache.flink.runtime.taskmanager.Task.run(Task.java:712) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.RuntimeException: Error obtaining the sorted input: Thread 'SortMerger Reading Thread' terminated due to an exception: The record exceeds the maximum size of a sort buffer (current maximum: 30310400 bytes). at org.apache.flink.runtime.operators.sort.UnilateralSortMerger.getIterator(UnilateralSortMerger.java:650) at org.apache.flink.runtime.operators.BatchTask.getInput(BatchTask.java:1108) at org.apache.flink.runtime.operators.GroupReduceDriver.prepare(GroupReduceDriver.java:99) at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:473) ... 3 more Caused by: java.io.IOException: Thread 'SortMerger Reading Thread' terminated due to an exception: The record exceeds the maximum size of a sort buffer (current maximum: 30310400 bytes). at org.apache.flink.runtime.operators.sort.UnilateralSortMerger$ThreadBase.run(UnilateralSortMerger.java:831) Caused by: java.io.IOException: The record exceeds the maximum size of a sort buffer (current maximum: 30310400 bytes). at org.apache.flink.runtime.operators.sort.UnilateralSortMerger$ReadingThread.go(UnilateralSortMerger.java:986) at org.apache.flink.runtime.operators.sort.UnilateralSortMerger$ThreadBase.run(UnilateralSortMerger.java:827) 2018-11-12 20:45:09,352 INFO org.apache.flink.runtime.taskmanager.Task - Freeing task resources for GroupReduce (GroupReduce at GenerateStatistics/RunStatsGenerators/CommonStatsGenerator/CombinePerKey/GroupByKey) (1/1) (dcd3891fa9540bb4bf594a1a3d631e2e). 2018-11-12 20:45:09,357 WARN bundle_processor.create_operation - No unique name set for transform fn/write/ref_PCollection_PCollection_77:0 2018-11-12 20:45:09,361 ERROR org.apache.flink.runtime.operators.BatchTask - Error in task code: GroupReduce (GroupReduce at GenerateStatistics/RunStatsGenerators/NumericStatsGenerator/CombinePerKey/GroupByKey) (1/1) java.lang.Exception: The data preparation for task 'GroupReduce (GroupReduce at GenerateStatistics/RunStatsGenerators/NumericStatsGenerator/CombinePerKey/GroupByKey)' , caused an error: Error obtaining the sorted input: Thread 'SortMerger Reading Thread' terminated due to an exception: The record exceeds the maximum size of a sort buffer (current maximum: 30310400 bytes). at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:479) at org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:368) at org.apache.flink.runtime.taskmanager.Task.run(Task.java:712) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.RuntimeException: Error obtaining the sorted input: Thread 'SortMerger Reading Thread' terminated due to an exception: The record exceeds the maximum size of a sort buffer (current maximum: 30310400 bytes). at org.apache.flink.runtime.operators.sort.UnilateralSortMerger.getIterator(UnilateralSortMerger.java:650) at org.apache.flink.runtime.operators.BatchTask.getInput(BatchTask.java:1108) at org.apache.flink.runtime.operators.GroupReduceDriver.prepare(GroupReduceDriver.java:99) at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:473) ... 3 more Caused by: java.io.IOException: Thread 'SortMerger Reading Thread' terminated due to an exception: The record exceeds the maximum size of a sort buffer (current maximum: 30310400 bytes). at org.apache.flink.runtime.operators.sort.UnilateralSortMerger$ThreadBase.run(UnilateralSortMerger.java:831) Caused by: java.io.IOException: The record exceeds the maximum size of a sort buffer (current maximum: 30310400 bytes). at org.apache.flink.runtime.operators.sort.UnilateralSortMerger$ReadingThread.go(UnilateralSortMerger.java:986) at org.apache.flink.runtime.operators.sort.UnilateralSortMerger$ThreadBase.run(UnilateralSortMerger.java:827) 2018-11-12 20:45:09,362 INFO org.apache.flink.runtime.taskmanager.Task - GroupReduce (GroupReduce at GenerateStatistics/RunStatsGenerators/NumericStatsGenerator/CombinePerKey/GroupByKey) (1/1) (51ce95407b2dd1e5b37501663e255dbd) switched from RUNNING to FAILED. java.lang.Exception: The data preparation for task 'GroupReduce (GroupReduce at GenerateStatistics/RunStatsGenerators/NumericStatsGenerator/CombinePerKey/GroupByKey)' , caused an error: Error obtaining the sorted input: Thread 'SortMerger Reading Thread' terminated due to an exception: The record exceeds the maximum size of a sort buffer (current maximum: 30310400 bytes). at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:479) at org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:368) at org.apache.flink.runtime.taskmanager.Task.run(Task.java:712) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.RuntimeException: Error obtaining the sorted input: Thread 'SortMerger Reading Thread' terminated due to an exception: The record exceeds the maximum size of a sort buffer (current maximum: 30310400 bytes). at org.apache.flink.runtime.operators.sort.UnilateralSortMerger.getIterator(UnilateralSortMerger.java:650) at org.apache.flink.runtime.operators.BatchTask.getInput(BatchTask.java:1108) at org.apache.flink.runtime.operators.GroupReduceDriver.prepare(GroupReduceDriver.java:99) at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:473) ... 3 more Caused by: java.io.IOException: Thread 'SortMerger Reading Thread' terminated due to an exception: The record exceeds the maximum size of a sort buffer (current maximum: 30310400 bytes). at org.apache.flink.runtime.operators.sort.UnilateralSortMerger$ThreadBase.run(UnilateralSortMerger.java:831) Caused by: java.io.IOException: The record exceeds the maximum size of a sort buffer (current maximum: 30310400 bytes). at org.apache.flink.runtime.operators.sort.UnilateralSortMerger$ReadingThread.go(UnilateralSortMerger.java:986) at org.apache.flink.runtime.operators.sort.UnilateralSortMerger$ThreadBase.run(UnilateralSortMerger.java:827) 2018-11-12 20:45:09,363 WARN bundle_processor.create_operation - No unique name set for transform fn/read/ref_PCollection_PCollection_74:0
> {code}
> Job Stats:
> {code:java}
> {
> "jid": "cbbca35225bdde85aeeb7146ef19ce69",
> "name": "BeamApp-goenka-1113041702-c9ef27fe",
> "isStoppable": false,
> "state": "FAILED",
> "start-time": 1542082624238,
> "end-time": 1542083249151,
> "duration": 624913,
> "now": 1542083464093,
> "timestamps": {
> "RUNNING": 1542082624264,
> "CANCELLING": 0,
> "FAILED": 1542083249151,
> "FINISHED": 0,
> "SUSPENDING": 0,
> "CREATED": 1542082624238,
> "SUSPENDED": 0,
> "RESTARTING": 0,
> "CANCELED": 0,
> "RECONCILING": 0,
> "FAILING": 1542083247152
> },
> "vertices": [
> {
> "id": "58c1d4699c3aef634832ff9b7ea3b704",
> "name": "DataSource (at DecodeData/InferFeatureTypes/DoOnce/Impulse (org.apache.beam.runners.flink.translation.wrappers.ImpulseInputFormat))",
> "parallelism": 1,
> "status": "FINISHED",
> "start-time": 1542082624293,
> "end-time": 1542082624891,
> "duration": 598,
> "tasks": {
> "SCHEDULED": 0,
> "RUNNING": 0,
> "CANCELING": 0,
> "CANCELED": 0,
> "FAILED": 0,
> "CREATED": 0,
> "DEPLOYING": 0,
> "RECONCILING": 0,
> "FINISHED": 1
> },
> "metrics": {
> "read-bytes": 0,
> "read-bytes-complete": true,
> "write-bytes": 0,
> "write-bytes-complete": true,
> "read-records": 0,
> "read-records-complete": true,
> "write-records": 1,
> "write-records-complete": true
> }
> },
> {
> "id": "d4ffa588405cb2c615c35edbb049faee",
> "name": "DataSource (at GenerateStatistics/RunStatsGenerators/MergeDatasetFeatureStatisticsProtos/DoOnce/Impulse (org.apache.beam.runners.flink.translation.wrappers.Impuls)",
> "parallelism": 1,
> "status": "FINISHED",
> "start-time": 1542082624292,
> "end-time": 1542082624890,
> "duration": 598,
> "tasks": {
> "SCHEDULED": 0,
> "RUNNING": 0,
> "CANCELING": 0,
> "CANCELED": 0,
> "FAILED": 0,
> "CREATED": 0,
> "DEPLOYING": 0,
> "RECONCILING": 0,
> "FINISHED": 1
> },
> "metrics": {
> "read-bytes": 0,
> "read-bytes-complete": true,
> "write-bytes": 0,
> "write-bytes-complete": true,
> "read-records": 0,
> "read-records-complete": true,
> "write-records": 1,
> "write-records-complete": true
> }
> },
> {
> "id": "2654a83491f298762b2651817b7a3fca",
> "name": "DataSource (at ReadData/Read/Impulse (org.apache.beam.runners.flink.translation.wrappers.ImpulseInputFormat))",
> "parallelism": 1,
> "status": "FINISHED",
> "start-time": 1542082624292,
> "end-time": 1542082624889,
> "duration": 597,
> "tasks": {
> "SCHEDULED": 0,
> "RUNNING": 0,
> "CANCELING": 0,
> "CANCELED": 0,
> "FAILED": 0,
> "CREATED": 0,
> "DEPLOYING": 0,
> "RECONCILING": 0,
> "FINISHED": 1
> },
> "metrics": {
> "read-bytes": 0,
> "read-bytes-complete": true,
> "write-bytes": 0,
> "write-bytes-complete": true,
> "read-records": 0,
> "read-records-complete": true,
> "write-records": 1,
> "write-records-complete": true
> }
> },
> {
> "id": "e1d5f5310bd2666573d2b7c2faf170cb",
> "name": "DataSource (at GenerateStatistics/RunStatsGenerators/NumericStatsGenerator/DoOnce/Impulse (org.apache.beam.runners.flink.translation.wrappers.ImpulseInputFormat))",
> "parallelism": 1,
> "status": "FINISHED",
> "start-time": 1542082624292,
> "end-time": 1542082624888,
> "duration": 596,
> "tasks": {
> "SCHEDULED": 0,
> "RUNNING": 0,
> "CANCELING": 0,
> "CANCELED": 0,
> "FAILED": 0,
> "CREATED": 0,
> "DEPLOYING": 0,
> "RECONCILING": 0,
> "FINISHED": 1
> },
> "metrics": {
> "read-bytes": 0,
> "read-bytes-complete": true,
> "write-bytes": 0,
> "write-bytes-complete": true,
> "read-records": 0,
> "read-records-complete": true,
> "write-records": 1,
> "write-records-complete": true
> }
> },
> {
> "id": "f8111ccb6bd1714295230f2f26e70526",
> "name": "DataSource (at GenerateStatistics/RunStatsGenerators/CommonStatsGenerator/DoOnce/Impulse (org.apache.beam.runners.flink.translation.wrappers.ImpulseInputFormat))",
> "parallelism": 1,
> "status": "FINISHED",
> "start-time": 1542082624292,
> "end-time": 1542082624884,
> "duration": 592,
> "tasks": {
> "SCHEDULED": 0,
> "RUNNING": 0,
> "CANCELING": 0,
> "CANCELED": 0,
> "FAILED": 0,
> "CREATED": 0,
> "DEPLOYING": 0,
> "RECONCILING": 0,
> "FINISHED": 1
> },
> "metrics": {
> "read-bytes": 0,
> "read-bytes-complete": true,
> "write-bytes": 0,
> "write-bytes-complete": true,
> "read-records": 0,
> "read-records-complete": true,
> "write-records": 1,
> "write-records-complete": true
> }
> },
> {
> "id": "e2e9488c10ee82bb3299dff6498221d6",
> "name": "DataSource (at GenerateStatistics/RunStatsGenerators/StringStatsGenerator/DoOnce/Impulse (org.apache.beam.runners.flink.translation.wrappers.ImpulseInputFormat))",
> "parallelism": 1,
> "status": "FINISHED",
> "start-time": 1542082624292,
> "end-time": 1542082624883,
> "duration": 591,
> "tasks": {
> "SCHEDULED": 0,
> "RUNNING": 0,
> "CANCELING": 0,
> "CANCELED": 0,
> "FAILED": 0,
> "CREATED": 0,
> "DEPLOYING": 0,
> "RECONCILING": 0,
> "FINISHED": 1
> },
> "metrics": {
> "read-bytes": 0,
> "read-bytes-complete": true,
> "write-bytes": 0,
> "write-bytes-complete": true,
> "read-records": 0,
> "read-records-complete": true,
> "write-records": 1,
> "write-records-complete": true
> }
> },
> {
> "id": "b81b935c3dbeb33defbe2a8cc43faf2a",
> "name": "DataSource (at WriteStatsOutput/Write/WriteImpl/DoOnce/Impulse (org.apache.beam.runners.flink.translation.wrappers.ImpulseInputFormat))",
> "parallelism": 1,
> "status": "FINISHED",
> "start-time": 1542082624292,
> "end-time": 1542082624875,
> "duration": 583,
> "tasks": {
> "SCHEDULED": 0,
> "RUNNING": 0,
> "CANCELING": 0,
> "CANCELED": 0,
> "FAILED": 0,
> "CREATED": 0,
> "DEPLOYING": 0,
> "RECONCILING": 0,
> "FINISHED": 1
> },
> "metrics": {
> "read-bytes": 0,
> "read-bytes-complete": true,
> "write-bytes": 0,
> "write-bytes-complete": true,
> "read-records": 0,
> "read-records-complete": true,
> "write-records": 1,
> "write-records-complete": true
> }
> },
> {
> "id": "298e41af9e9ebc1b922447d1d6473eaf",
> "name": "CHAIN MapPartition (MapPartition at 43DecodeData/InferFeatureTypes/DoOnce/Impulse.None/beam:env:docker:v1:0) -> FlatMap (FlatMap at 43DecodeData/InferFeatureTypes/DoOnce/Impulse.None/beam:env:docker:v1:0/out.0)",
> "parallelism": 1,
> "status": "FINISHED",
> "start-time": 1542082624868,
> "end-time": 1542082631331,
> "duration": 6463,
> "tasks": {
> "SCHEDULED": 0,
> "RUNNING": 0,
> "CANCELING": 0,
> "CANCELED": 0,
> "FAILED": 0,
> "CREATED": 0,
> "DEPLOYING": 0,
> "RECONCILING": 0,
> "FINISHED": 1
> },
> "metrics": {
> "read-bytes": 22,
> "read-bytes-complete": true,
> "write-bytes": 0,
> "write-bytes-complete": true,
> "read-records": 1,
> "read-records-complete": true,
> "write-records": 1,
> "write-records-complete": true
> }
> },
> {
> "id": "d188f20c85ac7be031abc25e1fb5b148",
> "name": "CHAIN MapPartition (MapPartition at 88GenerateStatistics/RunStatsGenerators/MergeDatasetFeatureStatisticsProtos/DoOnce/Impulse.None/beam:env:docker:v1:0) -> FlatMap (FlatMap at 88GenerateStatistics/RunStatsGenerators/MergeDatasetFeatureStatisticsProtos/DoOnce/Impulse.None/beam:env:docker:v1:0/out.0)",
> "parallelism": 1,
> "status": "FINISHED",
> "start-time": 1542082624868,
> "end-time": 1542082631366,
> "duration": 6498,
> "tasks": {
> "SCHEDULED": 0,
> "RUNNING": 0,
> "CANCELING": 0,
> "CANCELED": 0,
> "FAILED": 0,
> "CREATED": 0,
> "DEPLOYING": 0,
> "RECONCILING": 0,
> "FINISHED": 1
> },
> "metrics": {
> "read-bytes": 22,
> "read-bytes-complete": true,
> "write-bytes": 0,
> "write-bytes-complete": true,
> "read-records": 1,
> "read-records-complete": true,
> "write-records": 1,
> "write-records-complete": true
> }
> },
> {
> "id": "cec5f17742db186f0123405e18da30b3",
> "name": "CHAIN MapPartition (MapPartition at 21ReadData/Read/Impulse.None/beam:env:docker:v1:0) -> FlatMap (FlatMap at 21ReadData/Read/Impulse.None/beam:env:docker:v1:0/out.0) -> Map (Key Extractor) -> GroupCombine (GroupCombine at GroupCombine: ReadData/Read/Reshuffle/ReshufflePerKey/GroupByKey) -> Map (Key Extractor)",
> "parallelism": 1,
> "status": "FINISHED",
> "start-time": 1542082624870,
> "end-time": 1542082632467,
> "duration": 7597,
> "tasks": {
> "SCHEDULED": 0,
> "RUNNING": 0,
> "CANCELING": 0,
> "CANCELED": 0,
> "FAILED": 0,
> "CREATED": 0,
> "DEPLOYING": 0,
> "RECONCILING": 0,
> "FINISHED": 1
> },
> "metrics": {
> "read-bytes": 22,
> "read-bytes-complete": true,
> "write-bytes": 0,
> "write-bytes-complete": true,
> "read-records": 1,
> "read-records-complete": true,
> "write-records": 24,
> "write-records-complete": true
> }
> },
> {
> "id": "f2b94c0ad43c028476beca14f5b03871",
> "name": "GroupReduce (GroupReduce at ReadData/Read/Reshuffle/ReshufflePerKey/GroupByKey)",
> "parallelism": 1,
> "status": "FINISHED",
> "start-time": 1542082632465,
> "end-time": 1542082632500,
> "duration": 35,
> "tasks": {
> "SCHEDULED": 0,
> "RUNNING": 0,
> "CANCELING": 0,
> "CANCELED": 0,
> "FAILED": 0,
> "CREATED": 0,
> "DEPLOYING": 0,
> "RECONCILING": 0,
> "FINISHED": 1
> },
> "metrics": {
> "read-bytes": 10724,
> "read-bytes-complete": true,
> "write-bytes": 0,
> "write-bytes-complete": true,
> "read-records": 12,
> "read-records-complete": true,
> "write-records": 12,
> "write-records-complete": true
> }
> },
> {
> "id": "c6485ed8e08800c2f8ae0ef7e6411255",
> "name": "CHAIN MapPartition (MapPartition at 64ReadData/Read/Reshuffle/ReshufflePerKey/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0) -> FlatMap (FlatMap at 64ReadData/Read/Reshuffle/ReshufflePerKey/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0/out.0)",
> "parallelism": 1,
> "status": "FINISHED",
> "start-time": 1542082632499,
> "end-time": 1542082670922,
> "duration": 38423,
> "tasks": {
> "SCHEDULED": 0,
> "RUNNING": 0,
> "CANCELING": 0,
> "CANCELED": 0,
> "FAILED": 0,
> "CREATED": 0,
> "DEPLOYING": 0,
> "RECONCILING": 0,
> "FINISHED": 1
> },
> "metrics": {
> "read-bytes": 10539,
> "read-bytes-complete": true,
> "write-bytes": 364773376,
> "write-bytes-complete": true,
> "read-records": 12,
> "read-records-complete": true,
> "write-records": 1125972,
> "write-records-complete": true
> }
> },
> {
> "id": "4983cfb36a4f03aebd68726bad8c05a5",
> "name": "CHAIN MapPartition (MapPartition at 26DecodeData/ParseCSVRecords.None/beam:env:docker:v1:1) -> FlatMap (FlatMap at 26DecodeData/ParseCSVRecords.None/beam:env:docker:v1:1/out.0) -> Map (Key Extractor) -> GroupCombine (GroupCombine at GroupCombine: DecodeData/InferFeatureTypes/CombinePerKey/GroupByKey) -> Map (Key Extractor)",
> "parallelism": 1,
> "status": "FINISHED",
> "start-time": 1542082670924,
> "end-time": 1542082692698,
> "duration": 21774,
> "tasks": {
> "SCHEDULED": 0,
> "RUNNING": 0,
> "CANCELING": 0,
> "CANCELED": 0,
> "FAILED": 0,
> "CREATED": 0,
> "DEPLOYING": 0,
> "RECONCILING": 0,
> "FINISHED": 1
> },
> "metrics": {
> "read-bytes": 182394152,
> "read-bytes-complete": true,
> "write-bytes": 163250176,
> "write-bytes-complete": true,
> "read-records": 1125972,
> "read-records-complete": true,
> "write-records": 1125979,
> "write-records-complete": true
> }
> },
> {
> "id": "ab40a7291893738a7c7ecae2bcdd07a9",
> "name": "GroupReduce (GroupReduce at DecodeData/InferFeatureTypes/CombinePerKey/GroupByKey)",
> "parallelism": 1,
> "status": "FINISHED",
> "start-time": 1542082675389,
> "end-time": 1542082695000,
> "duration": 19611,
> "tasks": {
> "SCHEDULED": 0,
> "RUNNING": 0,
> "CANCELING": 0,
> "CANCELED": 0,
> "FAILED": 0,
> "CREATED": 0,
> "DEPLOYING": 0,
> "RECONCILING": 0,
> "FINISHED": 1
> },
> "metrics": {
> "read-bytes": 163252831,
> "read-bytes-complete": true,
> "write-bytes": 163250176,
> "write-bytes-complete": true,
> "read-records": 7,
> "read-records-complete": true,
> "write-records": 1,
> "write-records-complete": true
> }
> },
> {
> "id": "bf9097c62f41786b45bd4d64d00ecf33",
> "name": "CHAIN MapPartition (MapPartition at 67DecodeData/InferFeatureTypes/CombinePerKey/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0) -> FlatMap (FlatMap at 67DecodeData/InferFeatureTypes/CombinePerKey/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0/out.0)",
> "parallelism": 1,
> "status": "FINISHED",
> "start-time": 1542082694812,
> "end-time": 1542082734034,
> "duration": 39222,
> "tasks": {
> "SCHEDULED": 0,
> "RUNNING": 0,
> "CANCELING": 0,
> "CANCELED": 0,
> "FAILED": 0,
> "CREATED": 0,
> "DEPLOYING": 0,
> "RECONCILING": 0,
> "FINISHED": 1
> },
> "metrics": {
> "read-bytes": 163252651,
> "read-bytes-complete": true,
> "write-bytes": 0,
> "write-bytes-complete": true,
> "read-records": 1,
> "read-records-complete": true,
> "write-records": 1,
> "write-records-complete": true
> }
> },
> {
> "id": "7f96f31248e59e2ef247f45568169783",
> "name": "CHAIN MapPartition (MapPartition at 47DecodeData/InferFeatureTypes/DoOnce/Map(decode).None/beam:env:docker:v1:0) -> FlatMap (FlatMap at 47DecodeData/InferFeatureTypes/DoOnce/Map(decode).None/beam:env:docker:v1:0/out.0)",
> "parallelism": 1,
> "status": "FINISHED",
> "start-time": 1542082631308,
> "end-time": 1542082734132,
> "duration": 102824,
> "tasks": {
> "SCHEDULED": 0,
> "RUNNING": 0,
> "CANCELING": 0,
> "CANCELED": 0,
> "FAILED": 0,
> "CREATED": 0,
> "DEPLOYING": 0,
> "RECONCILING": 0,
> "FINISHED": 1
> },
> "metrics": {
> "read-bytes": 1658,
> "read-bytes-complete": true,
> "write-bytes": 0,
> "write-bytes-complete": true,
> "read-records": 1,
> "read-records-complete": true,
> "write-records": 1,
> "write-records-complete": true
> }
> },
> {
> "id": "9e5d4b9b9aee5486a04c9c6fa18cb684",
> "name": "MapPartition (MapPartition at 26DecodeData/ParseCSVRecords.None/beam:env:docker:v1:0)",
> "parallelism": 1,
> "status": "FINISHED",
> "start-time": 1542082670924,
> "end-time": 1542083241328,
> "duration": 570404,
> "tasks": {
> "SCHEDULED": 0,
> "RUNNING": 0,
> "CANCELING": 0,
> "CANCELED": 0,
> "FAILED": 0,
> "CREATED": 0,
> "DEPLOYING": 0,
> "RECONCILING": 0,
> "FINISHED": 1
> },
> "metrics": {
> "read-bytes": 182395787,
> "read-bytes-complete": true,
> "write-bytes": 11357388800,
> "write-bytes-complete": true,
> "read-records": 1125972,
> "read-records-complete": true,
> "write-records": 3713276,
> "write-records-complete": true
> }
> },
> {
> "id": "16d3da7754788feb2d6b67fcfc3d9f40",
> "name": "CHAIN FlatMap (FlatMap at 26DecodeData/ParseCSVRecords.None/beam:env:docker:v1:0/out.1) -> Map (Key Extractor) -> GroupCombine (GroupCombine at GroupCombine: GenerateStatistics/RunStatsGenerators/NumericStatsGenerator/CombinePerKey/GroupByKey) -> Map (Key Extractor)",
> "parallelism": 1,
> "status": "CANCELED",
> "start-time": 1542083241330,
> "end-time": 1542083248774,
> "duration": 7444,
> "tasks": {
> "SCHEDULED": 0,
> "RUNNING": 0,
> "CANCELING": 0,
> "CANCELED": 1,
> "FAILED": 0,
> "CREATED": 0,
> "DEPLOYING": 0,
> "RECONCILING": 0,
> "FINISHED": 0
> },
> "metrics": {
> "read-bytes": 390037504,
> "read-bytes-complete": true,
> "write-bytes": 99876864,
> "write-bytes-complete": true,
> "read-records": 502044,
> "read-records-complete": true,
> "write-records": 1095,
> "write-records-complete": true
> }
> },
> {
> "id": "b66363d1cd8ed075c6e4526d72af9176",
> "name": "GroupReduce (GroupReduce at GenerateStatistics/RunStatsGenerators/NumericStatsGenerator/CombinePerKey/GroupByKey)",
> "parallelism": 1,
> "status": "CANCELED",
> "start-time": 1542083242335,
> "end-time": 1542083248772,
> "duration": 6437,
> "tasks": {
> "SCHEDULED": 0,
> "RUNNING": 0,
> "CANCELING": 0,
> "CANCELED": 1,
> "FAILED": 0,
> "CREATED": 0,
> "DEPLOYING": 0,
> "RECONCILING": 0,
> "FINISHED": 0
> },
> "metrics": {
> "read-bytes": 90013696,
> "read-bytes-complete": true,
> "write-bytes": 0,
> "write-bytes-complete": true,
> "read-records": 0,
> "read-records-complete": true,
> "write-records": 0,
> "write-records-complete": true
> }
> },
> {
> "id": "d17e147a7808e6fffb562f5ad608348c",
> "name": "CHAIN MapPartition (MapPartition at 98GenerateStatistics/RunStatsGenerators/NumericStatsGenerator/CombinePerKey/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0) -> FlatMap (FlatMap at 98GenerateStatistics/RunStatsGenerators/NumericStatsGenerator/CombinePerKey/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0/out.0)",
> "parallelism": 1,
> "status": "CANCELED",
> "start-time": -1,
> "end-time": -1,
> "duration": -1,
> "tasks": {
> "SCHEDULED": 0,
> "RUNNING": 0,
> "CANCELING": 0,
> "CANCELED": 1,
> "FAILED": 0,
> "CREATED": 0,
> "DEPLOYING": 0,
> "RECONCILING": 0,
> "FINISHED": 0
> },
> "metrics": {
> "read-bytes": 0,
> "read-bytes-complete": true,
> "write-bytes": 0,
> "write-bytes-complete": true,
> "read-records": 0,
> "read-records-complete": true,
> "write-records": 0,
> "write-records-complete": true
> }
> },
> {
> "id": "510b5ba27b288217d588462cdfdb1c89",
> "name": "CHAIN FlatMap (FlatMap at 26DecodeData/ParseCSVRecords.None/beam:env:docker:v1:0/out.3) -> Map (Key Extractor) -> GroupCombine (GroupCombine at GroupCombine: GenerateStatistics/RunStatsGenerators/TopKStatsGenerator/TopK_CountFeatureNameValueTuple/CombinePerKey(CountCombineFn)/GroupByKey) -> Map (Key Extractor)",
> "parallelism": 1,
> "status": "CANCELED",
> "start-time": 1542083241330,
> "end-time": 1542083248782,
> "duration": 7452,
> "tasks": {
> "SCHEDULED": 0,
> "RUNNING": 0,
> "CANCELING": 0,
> "CANCELED": 1,
> "FAILED": 0,
> "CREATED": 0,
> "DEPLOYING": 0,
> "RECONCILING": 0,
> "FINISHED": 0
> },
> "metrics": {
> "read-bytes": 615284736,
> "read-bytes-complete": true,
> "write-bytes": 589824,
> "write-bytes-complete": true,
> "read-records": 875071,
> "read-records-complete": true,
> "write-records": 375219,
> "write-records-complete": true
> }
> },
> {
> "id": "59b4d70dcb043f75e8a71e0f60368044",
> "name": "GroupReduce (GroupReduce at GenerateStatistics/RunStatsGenerators/TopKStatsGenerator/TopK_CountFeatureNameValueTuple/CombinePerKey(CountCombineFn)/GroupByKey)",
> "parallelism": 1,
> "status": "CANCELED",
> "start-time": -1,
> "end-time": -1,
> "duration": -1,
> "tasks": {
> "SCHEDULED": 0,
> "RUNNING": 0,
> "CANCELING": 0,
> "CANCELED": 1,
> "FAILED": 0,
> "CREATED": 0,
> "DEPLOYING": 0,
> "RECONCILING": 0,
> "FINISHED": 0
> },
> "metrics": {
> "read-bytes": 0,
> "read-bytes-complete": true,
> "write-bytes": 0,
> "write-bytes-complete": true,
> "read-records": 0,
> "read-records-complete": true,
> "write-records": 0,
> "write-records-complete": true
> }
> },
> {
> "id": "b16f8e2ec206664fb6b95c49f57a04c2",
> "name": "CHAIN MapPartition (MapPartition at 143GenerateStatistics/RunStatsGenerators/TopKStatsGenerator/TopK_CountFeatureNameValueTuple/CombinePerKey(CountCombineFn)/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0) -> FlatMap (FlatMap at 143GenerateStatistics/RunStatsGenerators/TopKStatsGenerator/TopK_CountFeatureNameValueTuple/CombinePerKey(CountCombineFn)/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0/out.0) -> Map (Key Extractor) -> GroupCombine (GroupCombine at GroupCombine: GenerateStatistics/RunStatsGenerators/TopKStatsGenerator/TopK_GetTopK/CombinePerKey(TopCombineFn)/GroupByKey) -> Map (Key Extractor)",
> "parallelism": 1,
> "status": "CANCELED",
> "start-time": -1,
> "end-time": -1,
> "duration": -1,
> "tasks": {
> "SCHEDULED": 0,
> "RUNNING": 0,
> "CANCELING": 0,
> "CANCELED": 1,
> "FAILED": 0,
> "CREATED": 0,
> "DEPLOYING": 0,
> "RECONCILING": 0,
> "FINISHED": 0
> },
> "metrics": {
> "read-bytes": 0,
> "read-bytes-complete": true,
> "write-bytes": 0,
> "write-bytes-complete": true,
> "read-records": 0,
> "read-records-complete": true,
> "write-records": 0,
> "write-records-complete": true
> }
> },
> {
> "id": "24ef3f0717eb591a4a46823c63b305c3",
> "name": "GroupReduce (GroupReduce at GenerateStatistics/RunStatsGenerators/TopKStatsGenerator/TopK_GetTopK/CombinePerKey(TopCombineFn)/GroupByKey)",
> "parallelism": 1,
> "status": "CANCELED",
> "start-time": -1,
> "end-time": -1,
> "duration": -1,
> "tasks": {
> "SCHEDULED": 0,
> "RUNNING": 0,
> "CANCELING": 0,
> "CANCELED": 1,
> "FAILED": 0,
> "CREATED": 0,
> "DEPLOYING": 0,
> "RECONCILING": 0,
> "FINISHED": 0
> },
> "metrics": {
> "read-bytes": 0,
> "read-bytes-complete": true,
> "write-bytes": 0,
> "write-bytes-complete": true,
> "read-records": 0,
> "read-records-complete": true,
> "write-records": 0,
> "write-records-complete": true
> }
> },
> {
> "id": "18fc0b71278713b5c0f18c4299931157",
> "name": "CHAIN MapPartition (MapPartition at 122GenerateStatistics/RunStatsGenerators/TopKStatsGenerator/TopK_GetTopK/CombinePerKey(TopCombineFn)/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0) -> FlatMap (FlatMap at 122GenerateStatistics/RunStatsGenerators/TopKStatsGenerator/TopK_GetTopK/CombinePerKey(TopCombineFn)/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0/out.0)",
> "parallelism": 1,
> "status": "CANCELED",
> "start-time": -1,
> "end-time": -1,
> "duration": -1,
> "tasks": {
> "SCHEDULED": 0,
> "RUNNING": 0,
> "CANCELING": 0,
> "CANCELED": 1,
> "FAILED": 0,
> "CREATED": 0,
> "DEPLOYING": 0,
> "RECONCILING": 0,
> "FINISHED": 0
> },
> "metrics": {
> "read-bytes": 0,
> "read-bytes-complete": true,
> "write-bytes": 0,
> "write-bytes-complete": true,
> "read-records": 0,
> "read-records-complete": true,
> "write-records": 0,
> "write-records-complete": true
> }
> },
> {
> "id": "68f880071c774596985fcd67729f0f1a",
> "name": "CHAIN FlatMap (FlatMap at 26DecodeData/ParseCSVRecords.None/beam:env:docker:v1:0/out.2) -> Map (Key Extractor) -> GroupCombine (GroupCombine at GroupCombine: GenerateStatistics/RunStatsGenerators/StringStatsGenerator/CombinePerKey/GroupByKey) -> Map (Key Extractor)",
> "parallelism": 1,
> "status": "CANCELED",
> "start-time": 1542083241330,
> "end-time": 1542083248778,
> "duration": 7448,
> "tasks": {
> "SCHEDULED": 0,
> "RUNNING": 0,
> "CANCELING": 0,
> "CANCELED": 1,
> "FAILED": 0,
> "CREATED": 0,
> "DEPLOYING": 0,
> "RECONCILING": 0,
> "FINISHED": 0
> },
> "metrics": {
> "read-bytes": 400556032,
> "read-bytes-complete": true,
> "write-bytes": 120324096,
> "write-bytes-complete": true,
> "read-records": 502127,
> "read-records-complete": true,
> "write-records": 1095,
> "write-records-complete": true
> }
> },
> {
> "id": "2481c440912b84ca49e8f8b72da1752d",
> "name": "GroupReduce (GroupReduce at GenerateStatistics/RunStatsGenerators/StringStatsGenerator/CombinePerKey/GroupByKey)",
> "parallelism": 1,
> "status": "CANCELED",
> "start-time": 1542083242364,
> "end-time": 1542083248770,
> "duration": 6406,
> "tasks": {
> "SCHEDULED": 0,
> "RUNNING": 0,
> "CANCELING": 0,
> "CANCELED": 1,
> "FAILED": 0,
> "CREATED": 0,
> "DEPLOYING": 0,
> "RECONCILING": 0,
> "FINISHED": 0
> },
> "metrics": {
> "read-bytes": 90013696,
> "read-bytes-complete": true,
> "write-bytes": 0,
> "write-bytes-complete": true,
> "read-records": 0,
> "read-records-complete": true,
> "write-records": 0,
> "write-records-complete": true
> }
> },
> {
> "id": "e8d11f0a437ebd15b951c95626e4e052",
> "name": "CHAIN MapPartition (MapPartition at 97GenerateStatistics/RunStatsGenerators/StringStatsGenerator/CombinePerKey/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0) -> FlatMap (FlatMap at 97GenerateStatistics/RunStatsGenerators/StringStatsGenerator/CombinePerKey/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0/out.0)",
> "parallelism": 1,
> "status": "CANCELED",
> "start-time": -1,
> "end-time": -1,
> "duration": -1,
> "tasks": {
> "SCHEDULED": 0,
> "RUNNING": 0,
> "CANCELING": 0,
> "CANCELED": 1,
> "FAILED": 0,
> "CREATED": 0,
> "DEPLOYING": 0,
> "RECONCILING": 0,
> "FINISHED": 0
> },
> "metrics": {
> "read-bytes": 0,
> "read-bytes-complete": true,
> "write-bytes": 0,
> "write-bytes-complete": true,
> "read-records": 0,
> "read-records-complete": true,
> "write-records": 0,
> "write-records-complete": true
> }
> },
> {
> "id": "b5521a56f6f1fa2bc3a06a975a873c99",
> "name": "CHAIN FlatMap (FlatMap at 26DecodeData/ParseCSVRecords.None/beam:env:docker:v1:0/out.0) -> Map (Key Extractor) -> GroupCombine (GroupCombine at GroupCombine: GenerateStatistics/RunStatsGenerators/CommonStatsGenerator/CombinePerKey/GroupByKey) -> Map (Key Extractor)",
> "parallelism": 1,
> "status": "CANCELED",
> "start-time": 1542083241329,
> "end-time": 1542083248777,
> "duration": 7448,
> "tasks": {
> "SCHEDULED": 0,
> "RUNNING": 0,
> "CANCELING": 0,
> "CANCELED": 1,
> "FAILED": 0,
> "CREATED": 0,
> "DEPLOYING": 0,
> "RECONCILING": 0,
> "FINISHED": 0
> },
> "metrics": {
> "read-bytes": 583434240,
> "read-bytes-complete": true,
> "write-bytes": 180944896,
> "write-bytes-complete": true,
> "read-records": 505313,
> "read-records-complete": true,
> "write-records": 2265,
> "write-records-complete": true
> }
> },
> {
> "id": "18daf2682316240371747d029e2aa07f",
> "name": "GroupReduce (GroupReduce at GenerateStatistics/RunStatsGenerators/CommonStatsGenerator/CombinePerKey/GroupByKey)",
> "parallelism": 1,
> "status": "FAILED",
> "start-time": 1542083241961,
> "end-time": 1542083247152,
> "duration": 5191,
> "tasks": {
> "SCHEDULED": 0,
> "RUNNING": 0,
> "CANCELING": 0,
> "CANCELED": 0,
> "FAILED": 1,
> "CREATED": 0,
> "DEPLOYING": 0,
> "RECONCILING": 0,
> "FINISHED": 0
> },
> "metrics": {
> "read-bytes": 90013696,
> "read-bytes-complete": true,
> "write-bytes": 0,
> "write-bytes-complete": true,
> "read-records": 0,
> "read-records-complete": true,
> "write-records": 0,
> "write-records-complete": true
> }
> },
> {
> "id": "ee2a7ee51779773ead459855cd6fdf9a",
> "name": "CHAIN MapPartition (MapPartition at 97GenerateStatistics/RunStatsGenerators/CommonStatsGenerator/CombinePerKey/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0) -> FlatMap (FlatMap at 97GenerateStatistics/RunStatsGenerators/CommonStatsGenerator/CombinePerKey/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0/out.0)",
> "parallelism": 1,
> "status": "CANCELED",
> "start-time": -1,
> "end-time": -1,
> "duration": -1,
> "tasks": {
> "SCHEDULED": 0,
> "RUNNING": 0,
> "CANCELING": 0,
> "CANCELED": 1,
> "FAILED": 0,
> "CREATED": 0,
> "DEPLOYING": 0,
> "RECONCILING": 0,
> "FINISHED": 0
> },
> "metrics": {
> "read-bytes": 0,
> "read-bytes-complete": true,
> "write-bytes": 0,
> "write-bytes-complete": true,
> "read-records": 0,
> "read-records-complete": true,
> "write-records": 0,
> "write-records-complete": true
> }
> },
> {
> "id": "18c48f3b69f7182a426812bf0a129a91",
> "name": "CHAIN FlatMap (FlatMap at 26DecodeData/ParseCSVRecords.None/beam:env:docker:v1:0/out.4) -> Map (Key Extractor) -> GroupCombine (GroupCombine at GroupCombine: GenerateStatistics/RunStatsGenerators/UniquesStatsGenerator/Uniques_CountFeatureNameValueTuple/CombinePerKey(CountCombineFn)/GroupByKey) -> Map (Key Extractor)",
> "parallelism": 1,
> "status": "CANCELED",
> "start-time": 1542083241330,
> "end-time": 1542083248779,
> "duration": 7449,
> "tasks": {
> "SCHEDULED": 0,
> "RUNNING": 0,
> "CANCELING": 0,
> "CANCELED": 1,
> "FAILED": 0,
> "CREATED": 0,
> "DEPLOYING": 0,
> "RECONCILING": 0,
> "FINISHED": 0
> },
> "metrics": {
> "read-bytes": 603586560,
> "read-bytes-complete": true,
> "write-bytes": 753664,
> "write-bytes-complete": true,
> "read-records": 631916,
> "read-records-complete": true,
> "write-records": 375889,
> "write-records-complete": true
> }
> },
> {
> "id": "988381bcd910a9dabb5670e82d9fde7f",
> "name": "GroupReduce (GroupReduce at GenerateStatistics/RunStatsGenerators/UniquesStatsGenerator/Uniques_CountFeatureNameValueTuple/CombinePerKey(CountCombineFn)/GroupByKey)",
> "parallelism": 1,
> "status": "CANCELED",
> "start-time": -1,
> "end-time": -1,
> "duration": -1,
> "tasks": {
> "SCHEDULED": 0,
> "RUNNING": 0,
> "CANCELING": 0,
> "CANCELED": 1,
> "FAILED": 0,
> "CREATED": 0,
> "DEPLOYING": 0,
> "RECONCILING": 0,
> "FINISHED": 0
> },
> "metrics": {
> "read-bytes": 0,
> "read-bytes-complete": true,
> "write-bytes": 0,
> "write-bytes-complete": true,
> "read-records": 0,
> "read-records-complete": true,
> "write-records": 0,
> "write-records-complete": true
> }
> },
> {
> "id": "6fbe81468dce31c8ca2853fdba8a8760",
> "name": "CHAIN MapPartition (MapPartition at 149GenerateStatistics/RunStatsGenerators/UniquesStatsGenerator/Uniques_CountFeatureNameValueTuple/CombinePerKey(CountCombineFn)/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0) -> FlatMap (FlatMap at 149GenerateStatistics/RunStatsGenerators/UniquesStatsGenerator/Uniques_CountFeatureNameValueTuple/CombinePerKey(CountCombineFn)/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0/out.0) -> Map (Key Extractor) -> GroupCombine (GroupCombine at GroupCombine: GenerateStatistics/RunStatsGenerators/UniquesStatsGenerator/Uniques_CountPerFeatureName/CombinePerKey(CountCombineFn)/GroupByKey) -> Map (Key Extractor)",
> "parallelism": 1,
> "status": "CANCELED",
> "start-time": -1,
> "end-time": -1,
> "duration": -1,
> "tasks": {
> "SCHEDULED": 0,
> "RUNNING": 0,
> "CANCELING": 0,
> "CANCELED": 1,
> "FAILED": 0,
> "CREATED": 0,
> "DEPLOYING": 0,
> "RECONCILING": 0,
> "FINISHED": 0
> },
> "metrics": {
> "read-bytes": 0,
> "read-bytes-complete": true,
> "write-bytes": 0,
> "write-bytes-complete": true,
> "read-records": 0,
> "read-records-complete": true,
> "write-records": 0,
> "write-records-complete": true
> }
> },
> {
> "id": "3fe989337a8ccda9c573d1410ead4dd0",
> "name": "GroupReduce (GroupReduce at GenerateStatistics/RunStatsGenerators/UniquesStatsGenerator/Uniques_CountPerFeatureName/CombinePerKey(CountCombineFn)/GroupByKey)",
> "parallelism": 1,
> "status": "CANCELED",
> "start-time": -1,
> "end-time": -1,
> "duration": -1,
> "tasks": {
> "SCHEDULED": 0,
> "RUNNING": 0,
> "CANCELING": 0,
> "CANCELED": 1,
> "FAILED": 0,
> "CREATED": 0,
> "DEPLOYING": 0,
> "RECONCILING": 0,
> "FINISHED": 0
> },
> "metrics": {
> "read-bytes": 0,
> "read-bytes-complete": true,
> "write-bytes": 0,
> "write-bytes-complete": true,
> "read-records": 0,
> "read-records-complete": true,
> "write-records": 0,
> "write-records-complete": true
> }
> },
> {
> "id": "88b81a3cae9eab85c0ea89ba0c01f1e5",
> "name": "CHAIN MapPartition (MapPartition at 142GenerateStatistics/RunStatsGenerators/UniquesStatsGenerator/Uniques_CountPerFeatureName/CombinePerKey(CountCombineFn)/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0) -> FlatMap (FlatMap at 142GenerateStatistics/RunStatsGenerators/UniquesStatsGenerator/Uniques_CountPerFeatureName/CombinePerKey(CountCombineFn)/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0/out.0)",
> "parallelism": 1,
> "status": "CANCELED",
> "start-time": -1,
> "end-time": -1,
> "duration": -1,
> "tasks": {
> "SCHEDULED": 0,
> "RUNNING": 0,
> "CANCELING": 0,
> "CANCELED": 1,
> "FAILED": 0,
> "CREATED": 0,
> "DEPLOYING": 0,
> "RECONCILING": 0,
> "FINISHED": 0
> },
> "metrics": {
> "read-bytes": 0,
> "read-bytes-complete": true,
> "write-bytes": 0,
> "write-bytes-complete": true,
> "read-records": 0,
> "read-records-complete": true,
> "write-records": 0,
> "write-records-complete": true
> }
> },
> {
> "id": "42dee93d9fa1826c4fe5b74bbc2b20be",
> "name": "CHAIN MapPartition (MapPartition at 74GenerateStatistics/RunStatsGenerators/NumericStatsGenerator/DoOnce/Impulse.None/beam:env:docker:v1:0) -> FlatMap (FlatMap at 74GenerateStatistics/RunStatsGenerators/NumericStatsGenerator/DoOnce/Impulse.None/beam:env:docker:v1:0/out.0)",
> "parallelism": 1,
> "status": "FINISHED",
> "start-time": 1542082624867,
> "end-time": 1542082631350,
> "duration": 6483,
> "tasks": {
> "SCHEDULED": 0,
> "RUNNING": 0,
> "CANCELING": 0,
> "CANCELED": 0,
> "FAILED": 0,
> "CREATED": 0,
> "DEPLOYING": 0,
> "RECONCILING": 0,
> "FINISHED": 1
> },
> "metrics": {
> "read-bytes": 22,
> "read-bytes-complete": true,
> "write-bytes": 0,
> "write-bytes-complete": true,
> "read-records": 1,
> "read-records-complete": true,
> "write-records": 1,
> "write-records-complete": true
> }
> },
> {
> "id": "e8a3cc8c502ce75b685d3e2927a1c97f",
> "name": "CHAIN MapPartition (MapPartition at 78GenerateStatistics/RunStatsGenerators/NumericStatsGenerator/DoOnce/Map(decode).None/beam:env:docker:v1:0) -> FlatMap (FlatMap at 78GenerateStatistics/RunStatsGenerators/NumericStatsGenerator/DoOnce/Map(decode).None/beam:env:docker:v1:0/out.0)",
> "parallelism": 1,
> "status": "CANCELED",
> "start-time": 1542082631346,
> "end-time": 1542083248779,
> "duration": 617433,
> "tasks": {
> "SCHEDULED": 0,
> "RUNNING": 0,
> "CANCELING": 0,
> "CANCELED": 1,
> "FAILED": 0,
> "CREATED": 0,
> "DEPLOYING": 0,
> "RECONCILING": 0,
> "FINISHED": 0
> },
> "metrics": {
> "read-bytes": 0,
> "read-bytes-complete": true,
> "write-bytes": 0,
> "write-bytes-complete": true,
> "read-records": 0,
> "read-records-complete": true,
> "write-records": 0,
> "write-records-complete": true
> }
> },
> {
> "id": "3f481475a4117a27391074a639a24c77",
> "name": "CHAIN MapPartition (MapPartition at 73GenerateStatistics/RunStatsGenerators/CommonStatsGenerator/DoOnce/Impulse.None/beam:env:docker:v1:0) -> FlatMap (FlatMap at 73GenerateStatistics/RunStatsGenerators/CommonStatsGenerator/DoOnce/Impulse.None/beam:env:docker:v1:0/out.0)",
> "parallelism": 1,
> "status": "FINISHED",
> "start-time": 1542082624866,
> "end-time": 1542082631332,
> "duration": 6466,
> "tasks": {
> "SCHEDULED": 0,
> "RUNNING": 0,
> "CANCELING": 0,
> "CANCELED": 0,
> "FAILED": 0,
> "CREATED": 0,
> "DEPLOYING": 0,
> "RECONCILING": 0,
> "FINISHED": 1
> },
> "metrics": {
> "read-bytes": 22,
> "read-bytes-complete": true,
> "write-bytes": 0,
> "write-bytes-complete": true,
> "read-records": 1,
> "read-records-complete": true,
> "write-records": 1,
> "write-records-complete": true
> }
> },
> {
> "id": "3bd851095315b142e35707a497509f11",
> "name": "CHAIN MapPartition (MapPartition at 77GenerateStatistics/RunStatsGenerators/CommonStatsGenerator/DoOnce/Map(decode).None/beam:env:docker:v1:0) -> FlatMap (FlatMap at 77GenerateStatistics/RunStatsGenerators/CommonStatsGenerator/DoOnce/Map(decode).None/beam:env:docker:v1:0/out.0)",
> "parallelism": 1,
> "status": "CANCELED",
> "start-time": 1542082631308,
> "end-time": 1542083248780,
> "duration": 617472,
> "tasks": {
> "SCHEDULED": 0,
> "RUNNING": 0,
> "CANCELING": 0,
> "CANCELED": 1,
> "FAILED": 0,
> "CREATED": 0,
> "DEPLOYING": 0,
> "RECONCILING": 0,
> "FINISHED": 0
> },
> "metrics": {
> "read-bytes": 0,
> "read-bytes-complete": true,
> "write-bytes": 0,
> "write-bytes-complete": true,
> "read-records": 0,
> "read-records-complete": true,
> "write-records": 0,
> "write-records-complete": true
> }
> },
> {
> "id": "35bad9de5c3986d1aa37eb321996f9e1",
> "name": "CHAIN MapPartition (MapPartition at 73GenerateStatistics/RunStatsGenerators/StringStatsGenerator/DoOnce/Impulse.None/beam:env:docker:v1:0) -> FlatMap (FlatMap at 73GenerateStatistics/RunStatsGenerators/StringStatsGenerator/DoOnce/Impulse.None/beam:env:docker:v1:0/out.0)",
> "parallelism": 1,
> "status": "FINISHED",
> "start-time": 1542082624866,
> "end-time": 1542082631223,
> "duration": 6357,
> "tasks": {
> "SCHEDULED": 0,
> "RUNNING": 0,
> "CANCELING": 0,
> "CANCELED": 0,
> "FAILED": 0,
> "CREATED": 0,
> "DEPLOYING": 0,
> "RECONCILING": 0,
> "FINISHED": 1
> },
> "metrics": {
> "read-bytes": 22,
> "read-bytes-complete": true,
> "write-bytes": 0,
> "write-bytes-complete": true,
> "read-records": 1,
> "read-records-complete": true,
> "write-records": 1,
> "write-records-complete": true
> }
> },
> {
> "id": "6db900e63ce3b00bbed15ce6268976b0",
> "name": "CHAIN MapPartition (MapPartition at 77GenerateStatistics/RunStatsGenerators/StringStatsGenerator/DoOnce/Map(decode).None/beam:env:docker:v1:0) -> FlatMap (FlatMap at 77GenerateStatistics/RunStatsGenerators/StringStatsGenerator/DoOnce/Map(decode).None/beam:env:docker:v1:0/out.0)",
> "parallelism": 1,
> "status": "CANCELED",
> "start-time": 1542082631208,
> "end-time": 1542083248781,
> "duration": 617573,
> "tasks": {
> "SCHEDULED": 0,
> "RUNNING": 0,
> "CANCELING": 0,
> "CANCELED": 1,
> "FAILED": 0,
> "CREATED": 0,
> "DEPLOYING": 0,
> "RECONCILING": 0,
> "FINISHED": 0
> },
> "metrics": {
> "read-bytes": 0,
> "read-bytes-complete": true,
> "write-bytes": 0,
> "write-bytes-complete": true,
> "read-records": 0,
> "read-records-complete": true,
> "write-records": 0,
> "write-records-complete": true
> }
> },
> {
> "id": "bb036e3688efb54b9875caf74fe7017b",
> "name": "CHAIN Filter (UnionFixFilter) -> Map (Key Extractor) -> GroupCombine (GroupCombine at GroupCombine: GenerateStatistics/RunStatsGenerators/MergeDatasetFeatureStatisticsProtos/CombinePerKey/GroupByKey) -> Map (Key Extractor)",
> "parallelism": 1,
> "status": "CANCELED",
> "start-time": -1,
> "end-time": -1,
> "duration": -1,
> "tasks": {
> "SCHEDULED": 0,
> "RUNNING": 0,
> "CANCELING": 0,
> "CANCELED": 1,
> "FAILED": 0,
> "CREATED": 0,
> "DEPLOYING": 0,
> "RECONCILING": 0,
> "FINISHED": 0
> },
> "metrics": {
> "read-bytes": 0,
> "read-bytes-complete": true,
> "write-bytes": 0,
> "write-bytes-complete": true,
> "read-records": 0,
> "read-records-complete": true,
> "write-records": 0,
> "write-records-complete": true
> }
> },
> {
> "id": "1e4111a2c49845717e0275467fc1f09f",
> "name": "GroupReduce (GroupReduce at GenerateStatistics/RunStatsGenerators/MergeDatasetFeatureStatisticsProtos/CombinePerKey/GroupByKey)",
> "parallelism": 1,
> "status": "CANCELED",
> "start-time": -1,
> "end-time": -1,
> "duration": -1,
> "tasks": {
> "SCHEDULED": 0,
> "RUNNING": 0,
> "CANCELING": 0,
> "CANCELED": 1,
> "FAILED": 0,
> "CREATED": 0,
> "DEPLOYING": 0,
> "RECONCILING": 0,
> "FINISHED": 0
> },
> "metrics": {
> "read-bytes": 0,
> "read-bytes-complete": true,
> "write-bytes": 0,
> "write-bytes-complete": true,
> "read-records": 0,
> "read-records-complete": true,
> "write-records": 0,
> "write-records-complete": true
> }
> },
> {
> "id": "f6ef3a5be1f89eca633122d825dde2e4",
> "name": "CHAIN MapPartition (MapPartition at 112GenerateStatistics/RunStatsGenerators/MergeDatasetFeatureStatisticsProtos/CombinePerKey/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0) -> FlatMap (FlatMap at 112GenerateStatistics/RunStatsGenerators/MergeDatasetFeatureStatisticsProtos/CombinePerKey/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0/out.0)",
> "parallelism": 1,
> "status": "CANCELED",
> "start-time": -1,
> "end-time": -1,
> "duration": -1,
> "tasks": {
> "SCHEDULED": 0,
> "RUNNING": 0,
> "CANCELING": 0,
> "CANCELED": 1,
> "FAILED": 0,
> "CREATED": 0,
> "DEPLOYING": 0,
> "RECONCILING": 0,
> "FINISHED": 0
> },
> "metrics": {
> "read-bytes": 0,
> "read-bytes-complete": true,
> "write-bytes": 0,
> "write-bytes-complete": true,
> "read-records": 0,
> "read-records-complete": true,
> "write-records": 0,
> "write-records-complete": true
> }
> },
> {
> "id": "6948037a86b77d33fed35777dad327c3",
> "name": "CHAIN MapPartition (MapPartition at 92GenerateStatistics/RunStatsGenerators/MergeDatasetFeatureStatisticsProtos/DoOnce/Map(decode).None/beam:env:docker:v1:0) -> FlatMap (FlatMap at 92GenerateStatistics/RunStatsGenerators/MergeDatasetFeatureStatisticsProtos/DoOnce/Map(decode).None/beam:env:docker:v1:0/out.0) -> Map (Key Extractor) -> GroupCombine (GroupCombine at GroupCombine: WriteStatsOutput/Write/WriteImpl/GroupByKey) -> Map (Key Extractor)",
> "parallelism": 1,
> "status": "CANCELED",
> "start-time": 1542082631354,
> "end-time": 1542083248781,
> "duration": 617427,
> "tasks": {
> "SCHEDULED": 0,
> "RUNNING": 0,
> "CANCELING": 0,
> "CANCELED": 1,
> "FAILED": 0,
> "CREATED": 0,
> "DEPLOYING": 0,
> "RECONCILING": 0,
> "FINISHED": 0
> },
> "metrics": {
> "read-bytes": 0,
> "read-bytes-complete": true,
> "write-bytes": 0,
> "write-bytes-complete": true,
> "read-records": 0,
> "read-records-complete": true,
> "write-records": 0,
> "write-records-complete": true
> }
> },
> {
> "id": "a633dac13fbf4c7fd8bd521edc7ace83",
> "name": "GroupReduce (GroupReduce at WriteStatsOutput/Write/WriteImpl/GroupByKey)",
> "parallelism": 1,
> "status": "CANCELED",
> "start-time": -1,
> "end-time": -1,
> "duration": -1,
> "tasks": {
> "SCHEDULED": 0,
> "RUNNING": 0,
> "CANCELING": 0,
> "CANCELED": 1,
> "FAILED": 0,
> "CREATED": 0,
> "DEPLOYING": 0,
> "RECONCILING": 0,
> "FINISHED": 0
> },
> "metrics": {
> "read-bytes": 0,
> "read-bytes-complete": true,
> "write-bytes": 0,
> "write-bytes-complete": true,
> "read-records": 0,
> "read-records-complete": true,
> "write-records": 0,
> "write-records-complete": true
> }
> },
> {
> "id": "766e63063a9b847725557fe8646ec382",
> "name": "CHAIN MapPartition (MapPartition at 47WriteStatsOutput/Write/WriteImpl/DoOnce/Impulse.None/beam:env:docker:v1:0) -> FlatMap (FlatMap at 47WriteStatsOutput/Write/WriteImpl/DoOnce/Impulse.None/beam:env:docker:v1:0/out.0)",
> "parallelism": 1,
> "status": "FINISHED",
> "start-time": 1542082624864,
> "end-time": 1542082631303,
> "duration": 6439,
> "tasks": {
> "SCHEDULED": 0,
> "RUNNING": 0,
> "CANCELING": 0,
> "CANCELED": 0,
> "FAILED": 0,
> "CREATED": 0,
> "DEPLOYING": 0,
> "RECONCILING": 0,
> "FINISHED": 1
> },
> "metrics": {
> "read-bytes": 22,
> "read-bytes-complete": true,
> "write-bytes": 0,
> "write-bytes-complete": true,
> "read-records": 1,
> "read-records-complete": true,
> "write-records": 1,
> "write-records-complete": true
> }
> },
> {
> "id": "12012f2c6a9d768ad5651fd2d01f4276",
> "name": "CHAIN MapPartition (MapPartition at 51WriteStatsOutput/Write/WriteImpl/DoOnce/Map(decode).None/beam:env:docker:v1:0) -> FlatMap (FlatMap at 51WriteStatsOutput/Write/WriteImpl/DoOnce/Map(decode).None/beam:env:docker:v1:0/out.0)",
> "parallelism": 1,
> "status": "FINISHED",
> "start-time": 1542082631304,
> "end-time": 1542082631370,
> "duration": 66,
> "tasks": {
> "SCHEDULED": 0,
> "RUNNING": 0,
> "CANCELING": 0,
> "CANCELED": 0,
> "FAILED": 0,
> "CREATED": 0,
> "DEPLOYING": 0,
> "RECONCILING": 0,
> "FINISHED": 1
> },
> "metrics": {
> "read-bytes": 23,
> "read-bytes-complete": true,
> "write-bytes": 0,
> "write-bytes-complete": true,
> "read-records": 1,
> "read-records-complete": true,
> "write-records": 1,
> "write-records-complete": true
> }
> },
> {
> "id": "03641b59a0998396c201212176db6d3f",
> "name": "CHAIN MapPartition (MapPartition at 57WriteStatsOutput/Write/WriteImpl/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0) -> FlatMap (FlatMap at 57WriteStatsOutput/Write/WriteImpl/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0/out.0)",
> "parallelism": 1,
> "status": "CANCELED",
> "start-time": 1542082631371,
> "end-time": 1542083249150,
> "duration": 617779,
> "tasks": {
> "SCHEDULED": 0,
> "RUNNING": 0,
> "CANCELING": 0,
> "CANCELED": 1,
> "FAILED": 0,
> "CREATED": 0,
> "DEPLOYING": 0,
> "RECONCILING": 0,
> "FINISHED": 0
> },
> "metrics": {
> "read-bytes": 177,
> "read-bytes-complete": true,
> "write-bytes": 0,
> "write-bytes-complete": true,
> "read-records": 0,
> "read-records-complete": true,
> "write-records": 0,
> "write-records-complete": true
> }
> },
> {
> "id": "55d331be8cfc0f9ab69860e8289ad9c9",
> "name": "CHAIN MapPartition (MapPartition at 51WriteStatsOutput/Write/WriteImpl/DoOnce/Map(decode).None/beam:env:docker:v1:1) -> FlatMap (FlatMap at 51WriteStatsOutput/Write/WriteImpl/DoOnce/Map(decode).None/beam:env:docker:v1:1/out.0)",
> "parallelism": 1,
> "status": "CANCELED",
> "start-time": 1542082631304,
> "end-time": 1542083248782,
> "duration": 617478,
> "tasks": {
> "SCHEDULED": 0,
> "RUNNING": 0,
> "CANCELING": 0,
> "CANCELED": 1,
> "FAILED": 0,
> "CREATED": 0,
> "DEPLOYING": 0,
> "RECONCILING": 0,
> "FINISHED": 0
> },
> "metrics": {
> "read-bytes": 177,
> "read-bytes-complete": true,
> "write-bytes": 0,
> "write-bytes-complete": true,
> "read-records": 0,
> "read-records-complete": true,
> "write-records": 0,
> "write-records-complete": true
> }
> },
> {
> "id": "1e2ed63c2e5714f106bcbed1130c3957",
> "name": "MapPartition (MapPartition at 51WriteStatsOutput/Write/WriteImpl/DoOnce/Map(decode).None/beam:env:docker:v1:2)",
> "parallelism": 1,
> "status": "CANCELED",
> "start-time": 1542082631304,
> "end-time": 1542083248783,
> "duration": 617479,
> "tasks": {
> "SCHEDULED": 0,
> "RUNNING": 0,
> "CANCELING": 0,
> "CANCELED": 1,
> "FAILED": 0,
> "CREATED": 0,
> "DEPLOYING": 0,
> "RECONCILING": 0,
> "FINISHED": 0
> },
> "metrics": {
> "read-bytes": 177,
> "read-bytes-complete": true,
> "write-bytes": 0,
> "write-bytes-complete": true,
> "read-records": 0,
> "read-records-complete": true,
> "write-records": 0,
> "write-records-complete": true
> }
> },
> {
> "id": "27ea7d6f7814e0c565930b8fd6577fba",
> "name": "DataSink (org.apache.flink.api.java.io.DiscardingOutputFormat@32ec80e7)",
> "parallelism": 1,
> "status": "CANCELED",
> "start-time": -1,
> "end-time": -1,
> "duration": -1,
> "tasks": {
> "SCHEDULED": 0,
> "RUNNING": 0,
> "CANCELING": 0,
> "CANCELED": 1,
> "FAILED": 0,
> "CREATED": 0,
> "DEPLOYING": 0,
> "RECONCILING": 0,
> "FINISHED": 0
> },
> "metrics": {
> "read-bytes": 0,
> "read-bytes-complete": true,
> "write-bytes": 0,
> "write-bytes-complete": true,
> "read-records": 0,
> "read-records-complete": true,
> "write-records": 0,
> "write-records-complete": true
> }
> }
> ],
> "status-counts": {
> "SCHEDULED": 0,
> "RUNNING": 0,
> "CANCELING": 0,
> "CANCELED": 30,
> "FAILED": 1,
> "CREATED": 0,
> "DEPLOYING": 0,
> "RECONCILING": 0,
> "FINISHED": 22
> },
> "plan": {
> "jid": "cbbca35225bdde85aeeb7146ef19ce69",
> "name": "BeamApp-goenka-1113041702-c9ef27fe",
> "nodes": [
> {
> "id": "c6485ed8e08800c2f8ae0ef7e6411255",
> "parallelism": 1,
> "operator": "MapPartition -> FlatMap",
> "operator_strategy": "Map Partition<br/> -> FlatMap",
> "description": "MapPartition (MapPartition at 64ReadData/Read/Reshuffle/ReshufflePerKey/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0)<br/> -> FlatMap (FlatMap at 64ReadData/Read/Reshuffle/ReshufflePerKey/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0/out.0)",
> "inputs": [
> {
> "num": 0,
> "id": "f2b94c0ad43c028476beca14f5b03871",
> "ship_strategy": "Forward",
> "exchange": "pipelined"
> }
> ],
> "optimizer_properties": {
> "global_properties": [
> {
> "name": "Partitioning",
> "value": "RANDOM_PARTITIONED"
> },
> {
> "name": "Partitioning Order",
> "value": "(none)"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "local_properties": [
> {
> "name": "Order",
> "value": "(none)"
> },
> {
> "name": "Grouping",
> "value": "not grouped"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "estimates": [
> {
> "name": "Est. Output Size",
> "value": "(unknown)"
> },
> {
> "name": "Est. Cardinality",
> "value": "(unknown)"
> }
> ],
> "costs": [
> {
> "name": "Network",
> "value": "0.0"
> },
> {
> "name": "Disk I/O",
> "value": "0.0"
> },
> {
> "name": "CPU",
> "value": "0.0"
> },
> {
> "name": "Cumulative Network",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative Disk I/O",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative CPU",
> "value": "(unknown)"
> }
> ],
> "compiler_hints": [
> {
> "name": "Output Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Output Cardinality",
> "value": "(none)"
> },
> {
> "name": "Avg. Output Record Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Filter Factor",
> "value": "(none)"
> }
> ]
> }
> },
> {
> "id": "e8a3cc8c502ce75b685d3e2927a1c97f",
> "parallelism": 1,
> "operator": "MapPartition -> FlatMap",
> "operator_strategy": "Map Partition<br/> -> FlatMap",
> "description": "MapPartition (MapPartition at 78GenerateStatistics/RunStatsGenerators/NumericStatsGenerator/DoOnce/Map(decode).None/beam:env:docker:v1:0)<br/> -> FlatMap (FlatMap at 78GenerateStatistics/RunStatsGenerators/NumericStatsGenerator/DoOnce/Map(decode).None/beam:env:docker:v1:0/out.0)",
> "inputs": [
> {
> "num": 0,
> "id": "42dee93d9fa1826c4fe5b74bbc2b20be",
> "ship_strategy": "Forward",
> "exchange": "pipelined"
> },
> {
> "num": 1,
> "id": "d17e147a7808e6fffb562f5ad608348c",
> "ship_strategy": "Broadcast",
> "exchange": "pipelined"
> }
> ],
> "optimizer_properties": {
> "global_properties": [
> {
> "name": "Partitioning",
> "value": "RANDOM_PARTITIONED"
> },
> {
> "name": "Partitioning Order",
> "value": "(none)"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "local_properties": [
> {
> "name": "Order",
> "value": "(none)"
> },
> {
> "name": "Grouping",
> "value": "not grouped"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "estimates": [
> {
> "name": "Est. Output Size",
> "value": "(unknown)"
> },
> {
> "name": "Est. Cardinality",
> "value": "(unknown)"
> }
> ],
> "costs": [
> {
> "name": "Network",
> "value": "0.0"
> },
> {
> "name": "Disk I/O",
> "value": "0.0"
> },
> {
> "name": "CPU",
> "value": "0.0"
> },
> {
> "name": "Cumulative Network",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative Disk I/O",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative CPU",
> "value": "(unknown)"
> }
> ],
> "compiler_hints": [
> {
> "name": "Output Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Output Cardinality",
> "value": "(none)"
> },
> {
> "name": "Avg. Output Record Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Filter Factor",
> "value": "(none)"
> }
> ]
> }
> },
> {
> "id": "bb036e3688efb54b9875caf74fe7017b",
> "parallelism": 1,
> "operator": "Filter -> Map -> GroupCombine -> Map",
> "operator_strategy": "FlatMap<br/> -> Map<br/> -> Sorted Combine<br/> -> Map",
> "description": "Filter (UnionFixFilter)<br/> -> Map (Key Extractor)<br/> -> GroupCombine (GroupCombine at GroupCombine: GenerateStatistics/RunStatsGenerators/MergeDatasetFeatureStatisticsProtos/CombinePerKey/GroupByKey)<br/> -> Map (Key Extractor)",
> "inputs": [
> {
> "num": 0,
> "id": "e8a3cc8c502ce75b685d3e2927a1c97f",
> "ship_strategy": "Forward",
> "exchange": "pipelined"
> },
> {
> "num": 1,
> "id": "18fc0b71278713b5c0f18c4299931157",
> "ship_strategy": "Forward",
> "exchange": "pipelined"
> },
> {
> "num": 2,
> "id": "6db900e63ce3b00bbed15ce6268976b0",
> "ship_strategy": "Forward",
> "exchange": "pipelined"
> },
> {
> "num": 3,
> "id": "3bd851095315b142e35707a497509f11",
> "ship_strategy": "Forward",
> "exchange": "pipelined"
> },
> {
> "num": 4,
> "id": "88b81a3cae9eab85c0ea89ba0c01f1e5",
> "ship_strategy": "Forward",
> "exchange": "pipelined"
> }
> ],
> "optimizer_properties": {
> "global_properties": [
> {
> "name": "Partitioning",
> "value": "RANDOM_PARTITIONED"
> },
> {
> "name": "Partitioning Order",
> "value": "(none)"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "local_properties": [
> {
> "name": "Order",
> "value": "(none)"
> },
> {
> "name": "Grouping",
> "value": "not grouped"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "estimates": [
> {
> "name": "Est. Output Size",
> "value": "(unknown)"
> },
> {
> "name": "Est. Cardinality",
> "value": "(unknown)"
> }
> ],
> "costs": [
> {
> "name": "Network",
> "value": "0.0"
> },
> {
> "name": "Disk I/O",
> "value": "0.0"
> },
> {
> "name": "CPU",
> "value": "0.0"
> },
> {
> "name": "Cumulative Network",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative Disk I/O",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative CPU",
> "value": "(unknown)"
> }
> ],
> "compiler_hints": [
> {
> "name": "Output Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Output Cardinality",
> "value": "(none)"
> },
> {
> "name": "Avg. Output Record Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Filter Factor",
> "value": "(none)"
> }
> ]
> }
> },
> {
> "id": "88b81a3cae9eab85c0ea89ba0c01f1e5",
> "parallelism": 1,
> "operator": "MapPartition -> FlatMap",
> "operator_strategy": "Map Partition<br/> -> FlatMap",
> "description": "MapPartition (MapPartition at 142GenerateStatistics/RunStatsGenerators/UniquesStatsGenerator/Uniques_CountPerFeatureName/CombinePerKey(CountCombineFn)/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0)<br/> -> FlatMap (FlatMap at 142GenerateStatistics/RunStatsGenerators/UniquesStatsGenerator/Uniques_CountPerFeatureName/CombinePerKey(CountCombineFn)/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0/out.0)",
> "inputs": [
> {
> "num": 0,
> "id": "3fe989337a8ccda9c573d1410ead4dd0",
> "ship_strategy": "Forward",
> "exchange": "pipelined"
> }
> ],
> "optimizer_properties": {
> "global_properties": [
> {
> "name": "Partitioning",
> "value": "RANDOM_PARTITIONED"
> },
> {
> "name": "Partitioning Order",
> "value": "(none)"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "local_properties": [
> {
> "name": "Order",
> "value": "(none)"
> },
> {
> "name": "Grouping",
> "value": "not grouped"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "estimates": [
> {
> "name": "Est. Output Size",
> "value": "(unknown)"
> },
> {
> "name": "Est. Cardinality",
> "value": "(unknown)"
> }
> ],
> "costs": [
> {
> "name": "Network",
> "value": "0.0"
> },
> {
> "name": "Disk I/O",
> "value": "0.0"
> },
> {
> "name": "CPU",
> "value": "0.0"
> },
> {
> "name": "Cumulative Network",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative Disk I/O",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative CPU",
> "value": "(unknown)"
> }
> ],
> "compiler_hints": [
> {
> "name": "Output Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Output Cardinality",
> "value": "(none)"
> },
> {
> "name": "Avg. Output Record Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Filter Factor",
> "value": "(none)"
> }
> ]
> }
> },
> {
> "id": "58c1d4699c3aef634832ff9b7ea3b704",
> "parallelism": 1,
> "operator": "Data Source",
> "operator_strategy": "(none)",
> "description": "DataSource (at DecodeData/InferFeatureTypes/DoOnce/Impulse (org.apache.beam.runners.flink.translation.wrappers.ImpulseInputFormat))",
> "optimizer_properties": {
> "global_properties": [
> {
> "name": "Partitioning",
> "value": "RANDOM_PARTITIONED"
> },
> {
> "name": "Partitioning Order",
> "value": "(none)"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "local_properties": [
> {
> "name": "Order",
> "value": "(none)"
> },
> {
> "name": "Grouping",
> "value": "not grouped"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "estimates": [
> {
> "name": "Est. Output Size",
> "value": "1.00"
> },
> {
> "name": "Est. Cardinality",
> "value": "1.00"
> }
> ],
> "costs": [
> {
> "name": "Network",
> "value": "0.0"
> },
> {
> "name": "Disk I/O",
> "value": "0.0"
> },
> {
> "name": "CPU",
> "value": "0.0"
> },
> {
> "name": "Cumulative Network",
> "value": "0.0"
> },
> {
> "name": "Cumulative Disk I/O",
> "value": "0.0"
> },
> {
> "name": "Cumulative CPU",
> "value": "0.0"
> }
> ],
> "compiler_hints": [
> {
> "name": "Output Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Output Cardinality",
> "value": "(none)"
> },
> {
> "name": "Avg. Output Record Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Filter Factor",
> "value": "(none)"
> }
> ]
> }
> },
> {
> "id": "3f481475a4117a27391074a639a24c77",
> "parallelism": 1,
> "operator": "MapPartition -> FlatMap",
> "operator_strategy": "Map Partition<br/> -> FlatMap",
> "description": "MapPartition (MapPartition at 73GenerateStatistics/RunStatsGenerators/CommonStatsGenerator/DoOnce/Impulse.None/beam:env:docker:v1:0)<br/> -> FlatMap (FlatMap at 73GenerateStatistics/RunStatsGenerators/CommonStatsGenerator/DoOnce/Impulse.None/beam:env:docker:v1:0/out.0)",
> "inputs": [
> {
> "num": 0,
> "id": "f8111ccb6bd1714295230f2f26e70526",
> "ship_strategy": "Forward",
> "exchange": "pipelined"
> }
> ],
> "optimizer_properties": {
> "global_properties": [
> {
> "name": "Partitioning",
> "value": "RANDOM_PARTITIONED"
> },
> {
> "name": "Partitioning Order",
> "value": "(none)"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "local_properties": [
> {
> "name": "Order",
> "value": "(none)"
> },
> {
> "name": "Grouping",
> "value": "not grouped"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "estimates": [
> {
> "name": "Est. Output Size",
> "value": "(unknown)"
> },
> {
> "name": "Est. Cardinality",
> "value": "(unknown)"
> }
> ],
> "costs": [
> {
> "name": "Network",
> "value": "0.0"
> },
> {
> "name": "Disk I/O",
> "value": "0.0"
> },
> {
> "name": "CPU",
> "value": "0.0"
> },
> {
> "name": "Cumulative Network",
> "value": "0.0"
> },
> {
> "name": "Cumulative Disk I/O",
> "value": "0.0"
> },
> {
> "name": "Cumulative CPU",
> "value": "0.0"
> }
> ],
> "compiler_hints": [
> {
> "name": "Output Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Output Cardinality",
> "value": "(none)"
> },
> {
> "name": "Avg. Output Record Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Filter Factor",
> "value": "(none)"
> }
> ]
> }
> },
> {
> "id": "55d331be8cfc0f9ab69860e8289ad9c9",
> "parallelism": 1,
> "operator": "MapPartition -> FlatMap",
> "operator_strategy": "Map Partition<br/> -> FlatMap",
> "description": "MapPartition (MapPartition at 51WriteStatsOutput/Write/WriteImpl/DoOnce/Map(decode).None/beam:env:docker:v1:1)<br/> -> FlatMap (FlatMap at 51WriteStatsOutput/Write/WriteImpl/DoOnce/Map(decode).None/beam:env:docker:v1:1/out.0)",
> "inputs": [
> {
> "num": 0,
> "id": "766e63063a9b847725557fe8646ec382",
> "ship_strategy": "Forward",
> "exchange": "blocking"
> },
> {
> "num": 1,
> "id": "12012f2c6a9d768ad5651fd2d01f4276",
> "ship_strategy": "Broadcast",
> "exchange": "blocking"
> },
> {
> "num": 2,
> "id": "03641b59a0998396c201212176db6d3f",
> "ship_strategy": "Broadcast",
> "exchange": "blocking"
> }
> ],
> "optimizer_properties": {
> "global_properties": [
> {
> "name": "Partitioning",
> "value": "RANDOM_PARTITIONED"
> },
> {
> "name": "Partitioning Order",
> "value": "(none)"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "local_properties": [
> {
> "name": "Order",
> "value": "(none)"
> },
> {
> "name": "Grouping",
> "value": "not grouped"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "estimates": [
> {
> "name": "Est. Output Size",
> "value": "(unknown)"
> },
> {
> "name": "Est. Cardinality",
> "value": "(unknown)"
> }
> ],
> "costs": [
> {
> "name": "Network",
> "value": "0.0"
> },
> {
> "name": "Disk I/O",
> "value": "0.0"
> },
> {
> "name": "CPU",
> "value": "0.0"
> },
> {
> "name": "Cumulative Network",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative Disk I/O",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative CPU",
> "value": "(unknown)"
> }
> ],
> "compiler_hints": [
> {
> "name": "Output Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Output Cardinality",
> "value": "(none)"
> },
> {
> "name": "Avg. Output Record Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Filter Factor",
> "value": "(none)"
> }
> ]
> }
> },
> {
> "id": "d17e147a7808e6fffb562f5ad608348c",
> "parallelism": 1,
> "operator": "MapPartition -> FlatMap",
> "operator_strategy": "Map Partition<br/> -> FlatMap",
> "description": "MapPartition (MapPartition at 98GenerateStatistics/RunStatsGenerators/NumericStatsGenerator/CombinePerKey/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0)<br/> -> FlatMap (FlatMap at 98GenerateStatistics/RunStatsGenerators/NumericStatsGenerator/CombinePerKey/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0/out.0)",
> "inputs": [
> {
> "num": 0,
> "id": "b66363d1cd8ed075c6e4526d72af9176",
> "ship_strategy": "Forward",
> "exchange": "pipelined"
> }
> ],
> "optimizer_properties": {
> "global_properties": [
> {
> "name": "Partitioning",
> "value": "RANDOM_PARTITIONED"
> },
> {
> "name": "Partitioning Order",
> "value": "(none)"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "local_properties": [
> {
> "name": "Order",
> "value": "(none)"
> },
> {
> "name": "Grouping",
> "value": "not grouped"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "estimates": [
> {
> "name": "Est. Output Size",
> "value": "(unknown)"
> },
> {
> "name": "Est. Cardinality",
> "value": "(unknown)"
> }
> ],
> "costs": [
> {
> "name": "Network",
> "value": "0.0"
> },
> {
> "name": "Disk I/O",
> "value": "0.0"
> },
> {
> "name": "CPU",
> "value": "0.0"
> },
> {
> "name": "Cumulative Network",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative Disk I/O",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative CPU",
> "value": "(unknown)"
> }
> ],
> "compiler_hints": [
> {
> "name": "Output Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Output Cardinality",
> "value": "(none)"
> },
> {
> "name": "Avg. Output Record Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Filter Factor",
> "value": "(none)"
> }
> ]
> }
> },
> {
> "id": "d4ffa588405cb2c615c35edbb049faee",
> "parallelism": 1,
> "operator": "Data Source",
> "operator_strategy": "(none)",
> "description": "DataSource (at GenerateStatistics/RunStatsGenerators/MergeDatasetFeatureStatisticsProtos/DoOnce/Impulse (org.apache.beam.runners.flink.translation.wrappers.Impuls)",
> "optimizer_properties": {
> "global_properties": [
> {
> "name": "Partitioning",
> "value": "RANDOM_PARTITIONED"
> },
> {
> "name": "Partitioning Order",
> "value": "(none)"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "local_properties": [
> {
> "name": "Order",
> "value": "(none)"
> },
> {
> "name": "Grouping",
> "value": "not grouped"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "estimates": [
> {
> "name": "Est. Output Size",
> "value": "1.00"
> },
> {
> "name": "Est. Cardinality",
> "value": "1.00"
> }
> ],
> "costs": [
> {
> "name": "Network",
> "value": "0.0"
> },
> {
> "name": "Disk I/O",
> "value": "0.0"
> },
> {
> "name": "CPU",
> "value": "0.0"
> },
> {
> "name": "Cumulative Network",
> "value": "0.0"
> },
> {
> "name": "Cumulative Disk I/O",
> "value": "0.0"
> },
> {
> "name": "Cumulative CPU",
> "value": "0.0"
> }
> ],
> "compiler_hints": [
> {
> "name": "Output Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Output Cardinality",
> "value": "(none)"
> },
> {
> "name": "Avg. Output Record Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Filter Factor",
> "value": "(none)"
> }
> ]
> }
> },
> {
> "id": "16d3da7754788feb2d6b67fcfc3d9f40",
> "parallelism": 1,
> "operator": "FlatMap -> Map -> GroupCombine -> Map",
> "operator_strategy": "FlatMap<br/> -> Map<br/> -> Sorted Combine<br/> -> Map",
> "description": "FlatMap (FlatMap at 26DecodeData/ParseCSVRecords.None/beam:env:docker:v1:0/out.1)<br/> -> Map (Key Extractor)<br/> -> GroupCombine (GroupCombine at GroupCombine: GenerateStatistics/RunStatsGenerators/NumericStatsGenerator/CombinePerKey/GroupByKey)<br/> -> Map (Key Extractor)",
> "inputs": [
> {
> "num": 0,
> "id": "9e5d4b9b9aee5486a04c9c6fa18cb684",
> "ship_strategy": "Forward",
> "exchange": "blocking"
> }
> ],
> "optimizer_properties": {
> "global_properties": [
> {
> "name": "Partitioning",
> "value": "RANDOM_PARTITIONED"
> },
> {
> "name": "Partitioning Order",
> "value": "(none)"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "local_properties": [
> {
> "name": "Order",
> "value": "(none)"
> },
> {
> "name": "Grouping",
> "value": "not grouped"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "estimates": [
> {
> "name": "Est. Output Size",
> "value": "(unknown)"
> },
> {
> "name": "Est. Cardinality",
> "value": "(unknown)"
> }
> ],
> "costs": [
> {
> "name": "Network",
> "value": "0.0"
> },
> {
> "name": "Disk I/O",
> "value": "0.0"
> },
> {
> "name": "CPU",
> "value": "0.0"
> },
> {
> "name": "Cumulative Network",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative Disk I/O",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative CPU",
> "value": "(unknown)"
> }
> ],
> "compiler_hints": [
> {
> "name": "Output Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Output Cardinality",
> "value": "(none)"
> },
> {
> "name": "Avg. Output Record Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Filter Factor",
> "value": "(none)"
> }
> ]
> }
> },
> {
> "id": "a633dac13fbf4c7fd8bd521edc7ace83",
> "parallelism": 1,
> "operator": "GroupReduce",
> "operator_strategy": "Sorted Group Reduce",
> "description": "GroupReduce (GroupReduce at WriteStatsOutput/Write/WriteImpl/GroupByKey)",
> "inputs": [
> {
> "num": 0,
> "id": "6948037a86b77d33fed35777dad327c3",
> "ship_strategy": "Hash Partition on [0]",
> "local_strategy": "Sort on [0:ASC]",
> "exchange": "pipelined"
> }
> ],
> "optimizer_properties": {
> "global_properties": [
> {
> "name": "Partitioning",
> "value": "RANDOM_PARTITIONED"
> },
> {
> "name": "Partitioning Order",
> "value": "(none)"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "local_properties": [
> {
> "name": "Order",
> "value": "(none)"
> },
> {
> "name": "Grouping",
> "value": "not grouped"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "estimates": [
> {
> "name": "Est. Output Size",
> "value": "(unknown)"
> },
> {
> "name": "Est. Cardinality",
> "value": "(unknown)"
> }
> ],
> "costs": [
> {
> "name": "Network",
> "value": "(unknown)"
> },
> {
> "name": "Disk I/O",
> "value": "(unknown)"
> },
> {
> "name": "CPU",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative Network",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative Disk I/O",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative CPU",
> "value": "(unknown)"
> }
> ],
> "compiler_hints": [
> {
> "name": "Output Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Output Cardinality",
> "value": "(none)"
> },
> {
> "name": "Avg. Output Record Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Filter Factor",
> "value": "(none)"
> }
> ]
> }
> },
> {
> "id": "b5521a56f6f1fa2bc3a06a975a873c99",
> "parallelism": 1,
> "operator": "FlatMap -> Map -> GroupCombine -> Map",
> "operator_strategy": "FlatMap<br/> -> Map<br/> -> Sorted Combine<br/> -> Map",
> "description": "FlatMap (FlatMap at 26DecodeData/ParseCSVRecords.None/beam:env:docker:v1:0/out.0)<br/> -> Map (Key Extractor)<br/> -> GroupCombine (GroupCombine at GroupCombine: GenerateStatistics/RunStatsGenerators/CommonStatsGenerator/CombinePerKey/GroupByKey)<br/> -> Map (Key Extractor)",
> "inputs": [
> {
> "num": 0,
> "id": "9e5d4b9b9aee5486a04c9c6fa18cb684",
> "ship_strategy": "Forward",
> "exchange": "blocking"
> }
> ],
> "optimizer_properties": {
> "global_properties": [
> {
> "name": "Partitioning",
> "value": "RANDOM_PARTITIONED"
> },
> {
> "name": "Partitioning Order",
> "value": "(none)"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "local_properties": [
> {
> "name": "Order",
> "value": "(none)"
> },
> {
> "name": "Grouping",
> "value": "not grouped"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "estimates": [
> {
> "name": "Est. Output Size",
> "value": "(unknown)"
> },
> {
> "name": "Est. Cardinality",
> "value": "(unknown)"
> }
> ],
> "costs": [
> {
> "name": "Network",
> "value": "0.0"
> },
> {
> "name": "Disk I/O",
> "value": "0.0"
> },
> {
> "name": "CPU",
> "value": "0.0"
> },
> {
> "name": "Cumulative Network",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative Disk I/O",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative CPU",
> "value": "(unknown)"
> }
> ],
> "compiler_hints": [
> {
> "name": "Output Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Output Cardinality",
> "value": "(none)"
> },
> {
> "name": "Avg. Output Record Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Filter Factor",
> "value": "(none)"
> }
> ]
> }
> },
> {
> "id": "3fe989337a8ccda9c573d1410ead4dd0",
> "parallelism": 1,
> "operator": "GroupReduce",
> "operator_strategy": "Sorted Group Reduce",
> "description": "GroupReduce (GroupReduce at GenerateStatistics/RunStatsGenerators/UniquesStatsGenerator/Uniques_CountPerFeatureName/CombinePerKey(CountCombineFn)/GroupByKey)",
> "inputs": [
> {
> "num": 0,
> "id": "6fbe81468dce31c8ca2853fdba8a8760",
> "ship_strategy": "Hash Partition on [0]",
> "local_strategy": "Sort on [0:ASC]",
> "exchange": "pipelined"
> }
> ],
> "optimizer_properties": {
> "global_properties": [
> {
> "name": "Partitioning",
> "value": "RANDOM_PARTITIONED"
> },
> {
> "name": "Partitioning Order",
> "value": "(none)"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "local_properties": [
> {
> "name": "Order",
> "value": "(none)"
> },
> {
> "name": "Grouping",
> "value": "not grouped"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "estimates": [
> {
> "name": "Est. Output Size",
> "value": "(unknown)"
> },
> {
> "name": "Est. Cardinality",
> "value": "(unknown)"
> }
> ],
> "costs": [
> {
> "name": "Network",
> "value": "(unknown)"
> },
> {
> "name": "Disk I/O",
> "value": "(unknown)"
> },
> {
> "name": "CPU",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative Network",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative Disk I/O",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative CPU",
> "value": "(unknown)"
> }
> ],
> "compiler_hints": [
> {
> "name": "Output Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Output Cardinality",
> "value": "(none)"
> },
> {
> "name": "Avg. Output Record Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Filter Factor",
> "value": "(none)"
> }
> ]
> }
> },
> {
> "id": "f2b94c0ad43c028476beca14f5b03871",
> "parallelism": 1,
> "operator": "GroupReduce",
> "operator_strategy": "Sorted Group Reduce",
> "description": "GroupReduce (GroupReduce at ReadData/Read/Reshuffle/ReshufflePerKey/GroupByKey)",
> "inputs": [
> {
> "num": 0,
> "id": "cec5f17742db186f0123405e18da30b3",
> "ship_strategy": "Hash Partition on [0]",
> "local_strategy": "Sort on [0:ASC]",
> "exchange": "pipelined"
> }
> ],
> "optimizer_properties": {
> "global_properties": [
> {
> "name": "Partitioning",
> "value": "RANDOM_PARTITIONED"
> },
> {
> "name": "Partitioning Order",
> "value": "(none)"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "local_properties": [
> {
> "name": "Order",
> "value": "(none)"
> },
> {
> "name": "Grouping",
> "value": "not grouped"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "estimates": [
> {
> "name": "Est. Output Size",
> "value": "(unknown)"
> },
> {
> "name": "Est. Cardinality",
> "value": "(unknown)"
> }
> ],
> "costs": [
> {
> "name": "Network",
> "value": "(unknown)"
> },
> {
> "name": "Disk I/O",
> "value": "(unknown)"
> },
> {
> "name": "CPU",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative Network",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative Disk I/O",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative CPU",
> "value": "(unknown)"
> }
> ],
> "compiler_hints": [
> {
> "name": "Output Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Output Cardinality",
> "value": "(none)"
> },
> {
> "name": "Avg. Output Record Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Filter Factor",
> "value": "(none)"
> }
> ]
> }
> },
> {
> "id": "cec5f17742db186f0123405e18da30b3",
> "parallelism": 1,
> "operator": "MapPartition -> FlatMap -> Map -> GroupCombine -> Map",
> "operator_strategy": "Map Partition<br/> -> FlatMap<br/> -> Map<br/> -> Sorted Combine<br/> -> Map",
> "description": "MapPartition (MapPartition at 21ReadData/Read/Impulse.None/beam:env:docker:v1:0)<br/> -> FlatMap (FlatMap at 21ReadData/Read/Impulse.None/beam:env:docker:v1:0/out.0)<br/> -> Map (Key Extractor)<br/> -> GroupCombine (GroupCombine at GroupCombine: ReadData/Read/Reshuffle/ReshufflePerKey/GroupByKey)<br/> -> Map (Key Extractor)",
> "inputs": [
> {
> "num": 0,
> "id": "2654a83491f298762b2651817b7a3fca",
> "ship_strategy": "Forward",
> "exchange": "pipelined"
> }
> ],
> "optimizer_properties": {
> "global_properties": [
> {
> "name": "Partitioning",
> "value": "RANDOM_PARTITIONED"
> },
> {
> "name": "Partitioning Order",
> "value": "(none)"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "local_properties": [
> {
> "name": "Order",
> "value": "(none)"
> },
> {
> "name": "Grouping",
> "value": "not grouped"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "estimates": [
> {
> "name": "Est. Output Size",
> "value": "(unknown)"
> },
> {
> "name": "Est. Cardinality",
> "value": "(unknown)"
> }
> ],
> "costs": [
> {
> "name": "Network",
> "value": "0.0"
> },
> {
> "name": "Disk I/O",
> "value": "0.0"
> },
> {
> "name": "CPU",
> "value": "0.0"
> },
> {
> "name": "Cumulative Network",
> "value": "0.0"
> },
> {
> "name": "Cumulative Disk I/O",
> "value": "0.0"
> },
> {
> "name": "Cumulative CPU",
> "value": "0.0"
> }
> ],
> "compiler_hints": [
> {
> "name": "Output Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Output Cardinality",
> "value": "(none)"
> },
> {
> "name": "Avg. Output Record Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Filter Factor",
> "value": "(none)"
> }
> ]
> }
> },
> {
> "id": "18c48f3b69f7182a426812bf0a129a91",
> "parallelism": 1,
> "operator": "FlatMap -> Map -> GroupCombine -> Map",
> "operator_strategy": "FlatMap<br/> -> Map<br/> -> Sorted Combine<br/> -> Map",
> "description": "FlatMap (FlatMap at 26DecodeData/ParseCSVRecords.None/beam:env:docker:v1:0/out.4)<br/> -> Map (Key Extractor)<br/> -> GroupCombine (GroupCombine at GroupCombine: GenerateStatistics/RunStatsGenerators/UniquesStatsGenerator/Uniques_CountFeatureNameValueTuple/CombinePerKey(CountCombineFn)/GroupByKey)<br/> -> Map (Key Extractor)",
> "inputs": [
> {
> "num": 0,
> "id": "9e5d4b9b9aee5486a04c9c6fa18cb684",
> "ship_strategy": "Forward",
> "exchange": "blocking"
> }
> ],
> "optimizer_properties": {
> "global_properties": [
> {
> "name": "Partitioning",
> "value": "RANDOM_PARTITIONED"
> },
> {
> "name": "Partitioning Order",
> "value": "(none)"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "local_properties": [
> {
> "name": "Order",
> "value": "(none)"
> },
> {
> "name": "Grouping",
> "value": "not grouped"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "estimates": [
> {
> "name": "Est. Output Size",
> "value": "(unknown)"
> },
> {
> "name": "Est. Cardinality",
> "value": "(unknown)"
> }
> ],
> "costs": [
> {
> "name": "Network",
> "value": "0.0"
> },
> {
> "name": "Disk I/O",
> "value": "0.0"
> },
> {
> "name": "CPU",
> "value": "0.0"
> },
> {
> "name": "Cumulative Network",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative Disk I/O",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative CPU",
> "value": "(unknown)"
> }
> ],
> "compiler_hints": [
> {
> "name": "Output Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Output Cardinality",
> "value": "(none)"
> },
> {
> "name": "Avg. Output Record Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Filter Factor",
> "value": "(none)"
> }
> ]
> }
> },
> {
> "id": "42dee93d9fa1826c4fe5b74bbc2b20be",
> "parallelism": 1,
> "operator": "MapPartition -> FlatMap",
> "operator_strategy": "Map Partition<br/> -> FlatMap",
> "description": "MapPartition (MapPartition at 74GenerateStatistics/RunStatsGenerators/NumericStatsGenerator/DoOnce/Impulse.None/beam:env:docker:v1:0)<br/> -> FlatMap (FlatMap at 74GenerateStatistics/RunStatsGenerators/NumericStatsGenerator/DoOnce/Impulse.None/beam:env:docker:v1:0/out.0)",
> "inputs": [
> {
> "num": 0,
> "id": "e1d5f5310bd2666573d2b7c2faf170cb",
> "ship_strategy": "Forward",
> "exchange": "pipelined"
> }
> ],
> "optimizer_properties": {
> "global_properties": [
> {
> "name": "Partitioning",
> "value": "RANDOM_PARTITIONED"
> },
> {
> "name": "Partitioning Order",
> "value": "(none)"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "local_properties": [
> {
> "name": "Order",
> "value": "(none)"
> },
> {
> "name": "Grouping",
> "value": "not grouped"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "estimates": [
> {
> "name": "Est. Output Size",
> "value": "(unknown)"
> },
> {
> "name": "Est. Cardinality",
> "value": "(unknown)"
> }
> ],
> "costs": [
> {
> "name": "Network",
> "value": "0.0"
> },
> {
> "name": "Disk I/O",
> "value": "0.0"
> },
> {
> "name": "CPU",
> "value": "0.0"
> },
> {
> "name": "Cumulative Network",
> "value": "0.0"
> },
> {
> "name": "Cumulative Disk I/O",
> "value": "0.0"
> },
> {
> "name": "Cumulative CPU",
> "value": "0.0"
> }
> ],
> "compiler_hints": [
> {
> "name": "Output Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Output Cardinality",
> "value": "(none)"
> },
> {
> "name": "Avg. Output Record Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Filter Factor",
> "value": "(none)"
> }
> ]
> }
> },
> {
> "id": "ee2a7ee51779773ead459855cd6fdf9a",
> "parallelism": 1,
> "operator": "MapPartition -> FlatMap",
> "operator_strategy": "Map Partition<br/> -> FlatMap",
> "description": "MapPartition (MapPartition at 97GenerateStatistics/RunStatsGenerators/CommonStatsGenerator/CombinePerKey/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0)<br/> -> FlatMap (FlatMap at 97GenerateStatistics/RunStatsGenerators/CommonStatsGenerator/CombinePerKey/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0/out.0)",
> "inputs": [
> {
> "num": 0,
> "id": "18daf2682316240371747d029e2aa07f",
> "ship_strategy": "Forward",
> "exchange": "pipelined"
> }
> ],
> "optimizer_properties": {
> "global_properties": [
> {
> "name": "Partitioning",
> "value": "RANDOM_PARTITIONED"
> },
> {
> "name": "Partitioning Order",
> "value": "(none)"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "local_properties": [
> {
> "name": "Order",
> "value": "(none)"
> },
> {
> "name": "Grouping",
> "value": "not grouped"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "estimates": [
> {
> "name": "Est. Output Size",
> "value": "(unknown)"
> },
> {
> "name": "Est. Cardinality",
> "value": "(unknown)"
> }
> ],
> "costs": [
> {
> "name": "Network",
> "value": "0.0"
> },
> {
> "name": "Disk I/O",
> "value": "0.0"
> },
> {
> "name": "CPU",
> "value": "0.0"
> },
> {
> "name": "Cumulative Network",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative Disk I/O",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative CPU",
> "value": "(unknown)"
> }
> ],
> "compiler_hints": [
> {
> "name": "Output Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Output Cardinality",
> "value": "(none)"
> },
> {
> "name": "Avg. Output Record Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Filter Factor",
> "value": "(none)"
> }
> ]
> }
> },
> {
> "id": "6948037a86b77d33fed35777dad327c3",
> "parallelism": 1,
> "operator": "MapPartition -> FlatMap -> Map -> GroupCombine -> Map",
> "operator_strategy": "Map Partition<br/> -> FlatMap<br/> -> Map<br/> -> Sorted Combine<br/> -> Map",
> "description": "MapPartition (MapPartition at 92GenerateStatistics/RunStatsGenerators/MergeDatasetFeatureStatisticsProtos/DoOnce/Map(decode).None/beam:env:docker:v1:0)<br/> -> FlatMap (FlatMap at 92GenerateStatistics/RunStatsGenerators/MergeDatasetFeatureStatisticsProtos/DoOnce/Map(decode).None/beam:env:docker:v1:0/out.0)<br/> -> Map (Key Extractor)<br/> -> GroupCombine (GroupCombine at GroupCombine: WriteStatsOutput/Write/WriteImpl/GroupByKey)<br/> -> Map (Key Extractor)",
> "inputs": [
> {
> "num": 0,
> "id": "d188f20c85ac7be031abc25e1fb5b148",
> "ship_strategy": "Forward",
> "exchange": "pipelined"
> },
> {
> "num": 1,
> "id": "f6ef3a5be1f89eca633122d825dde2e4",
> "ship_strategy": "Broadcast",
> "exchange": "pipelined"
> }
> ],
> "optimizer_properties": {
> "global_properties": [
> {
> "name": "Partitioning",
> "value": "RANDOM_PARTITIONED"
> },
> {
> "name": "Partitioning Order",
> "value": "(none)"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "local_properties": [
> {
> "name": "Order",
> "value": "(none)"
> },
> {
> "name": "Grouping",
> "value": "not grouped"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "estimates": [
> {
> "name": "Est. Output Size",
> "value": "(unknown)"
> },
> {
> "name": "Est. Cardinality",
> "value": "(unknown)"
> }
> ],
> "costs": [
> {
> "name": "Network",
> "value": "0.0"
> },
> {
> "name": "Disk I/O",
> "value": "0.0"
> },
> {
> "name": "CPU",
> "value": "0.0"
> },
> {
> "name": "Cumulative Network",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative Disk I/O",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative CPU",
> "value": "(unknown)"
> }
> ],
> "compiler_hints": [
> {
> "name": "Output Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Output Cardinality",
> "value": "(none)"
> },
> {
> "name": "Avg. Output Record Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Filter Factor",
> "value": "(none)"
> }
> ]
> }
> },
> {
> "id": "bf9097c62f41786b45bd4d64d00ecf33",
> "parallelism": 1,
> "operator": "MapPartition -> FlatMap",
> "operator_strategy": "Map Partition<br/> -> FlatMap",
> "description": "MapPartition (MapPartition at 67DecodeData/InferFeatureTypes/CombinePerKey/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0)<br/> -> FlatMap (FlatMap at 67DecodeData/InferFeatureTypes/CombinePerKey/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0/out.0)",
> "inputs": [
> {
> "num": 0,
> "id": "ab40a7291893738a7c7ecae2bcdd07a9",
> "ship_strategy": "Forward",
> "exchange": "pipelined"
> }
> ],
> "optimizer_properties": {
> "global_properties": [
> {
> "name": "Partitioning",
> "value": "RANDOM_PARTITIONED"
> },
> {
> "name": "Partitioning Order",
> "value": "(none)"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "local_properties": [
> {
> "name": "Order",
> "value": "(none)"
> },
> {
> "name": "Grouping",
> "value": "not grouped"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "estimates": [
> {
> "name": "Est. Output Size",
> "value": "(unknown)"
> },
> {
> "name": "Est. Cardinality",
> "value": "(unknown)"
> }
> ],
> "costs": [
> {
> "name": "Network",
> "value": "0.0"
> },
> {
> "name": "Disk I/O",
> "value": "0.0"
> },
> {
> "name": "CPU",
> "value": "0.0"
> },
> {
> "name": "Cumulative Network",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative Disk I/O",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative CPU",
> "value": "(unknown)"
> }
> ],
> "compiler_hints": [
> {
> "name": "Output Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Output Cardinality",
> "value": "(none)"
> },
> {
> "name": "Avg. Output Record Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Filter Factor",
> "value": "(none)"
> }
> ]
> }
> },
> {
> "id": "18daf2682316240371747d029e2aa07f",
> "parallelism": 1,
> "operator": "GroupReduce",
> "operator_strategy": "Sorted Group Reduce",
> "description": "GroupReduce (GroupReduce at GenerateStatistics/RunStatsGenerators/CommonStatsGenerator/CombinePerKey/GroupByKey)",
> "inputs": [
> {
> "num": 0,
> "id": "b5521a56f6f1fa2bc3a06a975a873c99",
> "ship_strategy": "Hash Partition on [0]",
> "local_strategy": "Sort on [0:ASC]",
> "exchange": "pipelined"
> }
> ],
> "optimizer_properties": {
> "global_properties": [
> {
> "name": "Partitioning",
> "value": "RANDOM_PARTITIONED"
> },
> {
> "name": "Partitioning Order",
> "value": "(none)"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "local_properties": [
> {
> "name": "Order",
> "value": "(none)"
> },
> {
> "name": "Grouping",
> "value": "not grouped"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "estimates": [
> {
> "name": "Est. Output Size",
> "value": "(unknown)"
> },
> {
> "name": "Est. Cardinality",
> "value": "(unknown)"
> }
> ],
> "costs": [
> {
> "name": "Network",
> "value": "(unknown)"
> },
> {
> "name": "Disk I/O",
> "value": "(unknown)"
> },
> {
> "name": "CPU",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative Network",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative Disk I/O",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative CPU",
> "value": "(unknown)"
> }
> ],
> "compiler_hints": [
> {
> "name": "Output Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Output Cardinality",
> "value": "(none)"
> },
> {
> "name": "Avg. Output Record Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Filter Factor",
> "value": "(none)"
> }
> ]
> }
> },
> {
> "id": "24ef3f0717eb591a4a46823c63b305c3",
> "parallelism": 1,
> "operator": "GroupReduce",
> "operator_strategy": "Sorted Group Reduce",
> "description": "GroupReduce (GroupReduce at GenerateStatistics/RunStatsGenerators/TopKStatsGenerator/TopK_GetTopK/CombinePerKey(TopCombineFn)/GroupByKey)",
> "inputs": [
> {
> "num": 0,
> "id": "b16f8e2ec206664fb6b95c49f57a04c2",
> "ship_strategy": "Hash Partition on [0]",
> "local_strategy": "Sort on [0:ASC]",
> "exchange": "pipelined"
> }
> ],
> "optimizer_properties": {
> "global_properties": [
> {
> "name": "Partitioning",
> "value": "RANDOM_PARTITIONED"
> },
> {
> "name": "Partitioning Order",
> "value": "(none)"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "local_properties": [
> {
> "name": "Order",
> "value": "(none)"
> },
> {
> "name": "Grouping",
> "value": "not grouped"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "estimates": [
> {
> "name": "Est. Output Size",
> "value": "(unknown)"
> },
> {
> "name": "Est. Cardinality",
> "value": "(unknown)"
> }
> ],
> "costs": [
> {
> "name": "Network",
> "value": "(unknown)"
> },
> {
> "name": "Disk I/O",
> "value": "(unknown)"
> },
> {
> "name": "CPU",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative Network",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative Disk I/O",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative CPU",
> "value": "(unknown)"
> }
> ],
> "compiler_hints": [
> {
> "name": "Output Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Output Cardinality",
> "value": "(none)"
> },
> {
> "name": "Avg. Output Record Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Filter Factor",
> "value": "(none)"
> }
> ]
> }
> },
> {
> "id": "988381bcd910a9dabb5670e82d9fde7f",
> "parallelism": 1,
> "operator": "GroupReduce",
> "operator_strategy": "Sorted Group Reduce",
> "description": "GroupReduce (GroupReduce at GenerateStatistics/RunStatsGenerators/UniquesStatsGenerator/Uniques_CountFeatureNameValueTuple/CombinePerKey(CountCombineFn)/GroupByKey)",
> "inputs": [
> {
> "num": 0,
> "id": "18c48f3b69f7182a426812bf0a129a91",
> "ship_strategy": "Hash Partition on [0]",
> "local_strategy": "Sort on [0:ASC]",
> "exchange": "pipelined"
> }
> ],
> "optimizer_properties": {
> "global_properties": [
> {
> "name": "Partitioning",
> "value": "RANDOM_PARTITIONED"
> },
> {
> "name": "Partitioning Order",
> "value": "(none)"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "local_properties": [
> {
> "name": "Order",
> "value": "(none)"
> },
> {
> "name": "Grouping",
> "value": "not grouped"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "estimates": [
> {
> "name": "Est. Output Size",
> "value": "(unknown)"
> },
> {
> "name": "Est. Cardinality",
> "value": "(unknown)"
> }
> ],
> "costs": [
> {
> "name": "Network",
> "value": "(unknown)"
> },
> {
> "name": "Disk I/O",
> "value": "(unknown)"
> },
> {
> "name": "CPU",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative Network",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative Disk I/O",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative CPU",
> "value": "(unknown)"
> }
> ],
> "compiler_hints": [
> {
> "name": "Output Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Output Cardinality",
> "value": "(none)"
> },
> {
> "name": "Avg. Output Record Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Filter Factor",
> "value": "(none)"
> }
> ]
> }
> },
> {
> "id": "766e63063a9b847725557fe8646ec382",
> "parallelism": 1,
> "operator": "MapPartition -> FlatMap",
> "operator_strategy": "Map Partition<br/> -> FlatMap",
> "description": "MapPartition (MapPartition at 47WriteStatsOutput/Write/WriteImpl/DoOnce/Impulse.None/beam:env:docker:v1:0)<br/> -> FlatMap (FlatMap at 47WriteStatsOutput/Write/WriteImpl/DoOnce/Impulse.None/beam:env:docker:v1:0/out.0)",
> "inputs": [
> {
> "num": 0,
> "id": "b81b935c3dbeb33defbe2a8cc43faf2a",
> "ship_strategy": "Forward",
> "exchange": "pipelined"
> }
> ],
> "optimizer_properties": {
> "global_properties": [
> {
> "name": "Partitioning",
> "value": "RANDOM_PARTITIONED"
> },
> {
> "name": "Partitioning Order",
> "value": "(none)"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "local_properties": [
> {
> "name": "Order",
> "value": "(none)"
> },
> {
> "name": "Grouping",
> "value": "not grouped"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "estimates": [
> {
> "name": "Est. Output Size",
> "value": "(unknown)"
> },
> {
> "name": "Est. Cardinality",
> "value": "(unknown)"
> }
> ],
> "costs": [
> {
> "name": "Network",
> "value": "0.0"
> },
> {
> "name": "Disk I/O",
> "value": "0.0"
> },
> {
> "name": "CPU",
> "value": "0.0"
> },
> {
> "name": "Cumulative Network",
> "value": "0.0"
> },
> {
> "name": "Cumulative Disk I/O",
> "value": "0.0"
> },
> {
> "name": "Cumulative CPU",
> "value": "0.0"
> }
> ],
> "compiler_hints": [
> {
> "name": "Output Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Output Cardinality",
> "value": "(none)"
> },
> {
> "name": "Avg. Output Record Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Filter Factor",
> "value": "(none)"
> }
> ]
> }
> },
> {
> "id": "2654a83491f298762b2651817b7a3fca",
> "parallelism": 1,
> "operator": "Data Source",
> "operator_strategy": "(none)",
> "description": "DataSource (at ReadData/Read/Impulse (org.apache.beam.runners.flink.translation.wrappers.ImpulseInputFormat))",
> "optimizer_properties": {
> "global_properties": [
> {
> "name": "Partitioning",
> "value": "RANDOM_PARTITIONED"
> },
> {
> "name": "Partitioning Order",
> "value": "(none)"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "local_properties": [
> {
> "name": "Order",
> "value": "(none)"
> },
> {
> "name": "Grouping",
> "value": "not grouped"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "estimates": [
> {
> "name": "Est. Output Size",
> "value": "1.00"
> },
> {
> "name": "Est. Cardinality",
> "value": "1.00"
> }
> ],
> "costs": [
> {
> "name": "Network",
> "value": "0.0"
> },
> {
> "name": "Disk I/O",
> "value": "0.0"
> },
> {
> "name": "CPU",
> "value": "0.0"
> },
> {
> "name": "Cumulative Network",
> "value": "0.0"
> },
> {
> "name": "Cumulative Disk I/O",
> "value": "0.0"
> },
> {
> "name": "Cumulative CPU",
> "value": "0.0"
> }
> ],
> "compiler_hints": [
> {
> "name": "Output Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Output Cardinality",
> "value": "(none)"
> },
> {
> "name": "Avg. Output Record Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Filter Factor",
> "value": "(none)"
> }
> ]
> }
> },
> {
> "id": "298e41af9e9ebc1b922447d1d6473eaf",
> "parallelism": 1,
> "operator": "MapPartition -> FlatMap",
> "operator_strategy": "Map Partition<br/> -> FlatMap",
> "description": "MapPartition (MapPartition at 43DecodeData/InferFeatureTypes/DoOnce/Impulse.None/beam:env:docker:v1:0)<br/> -> FlatMap (FlatMap at 43DecodeData/InferFeatureTypes/DoOnce/Impulse.None/beam:env:docker:v1:0/out.0)",
> "inputs": [
> {
> "num": 0,
> "id": "58c1d4699c3aef634832ff9b7ea3b704",
> "ship_strategy": "Forward",
> "exchange": "pipelined"
> }
> ],
> "optimizer_properties": {
> "global_properties": [
> {
> "name": "Partitioning",
> "value": "RANDOM_PARTITIONED"
> },
> {
> "name": "Partitioning Order",
> "value": "(none)"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "local_properties": [
> {
> "name": "Order",
> "value": "(none)"
> },
> {
> "name": "Grouping",
> "value": "not grouped"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "estimates": [
> {
> "name": "Est. Output Size",
> "value": "(unknown)"
> },
> {
> "name": "Est. Cardinality",
> "value": "(unknown)"
> }
> ],
> "costs": [
> {
> "name": "Network",
> "value": "0.0"
> },
> {
> "name": "Disk I/O",
> "value": "0.0"
> },
> {
> "name": "CPU",
> "value": "0.0"
> },
> {
> "name": "Cumulative Network",
> "value": "0.0"
> },
> {
> "name": "Cumulative Disk I/O",
> "value": "0.0"
> },
> {
> "name": "Cumulative CPU",
> "value": "0.0"
> }
> ],
> "compiler_hints": [
> {
> "name": "Output Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Output Cardinality",
> "value": "(none)"
> },
> {
> "name": "Avg. Output Record Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Filter Factor",
> "value": "(none)"
> }
> ]
> }
> },
> {
> "id": "4983cfb36a4f03aebd68726bad8c05a5",
> "parallelism": 1,
> "operator": "MapPartition -> FlatMap -> Map -> GroupCombine -> Map",
> "operator_strategy": "Map Partition<br/> -> FlatMap<br/> -> Map<br/> -> Sorted Combine<br/> -> Map",
> "description": "MapPartition (MapPartition at 26DecodeData/ParseCSVRecords.None/beam:env:docker:v1:1)<br/> -> FlatMap (FlatMap at 26DecodeData/ParseCSVRecords.None/beam:env:docker:v1:1/out.0)<br/> -> Map (Key Extractor)<br/> -> GroupCombine (GroupCombine at GroupCombine: DecodeData/InferFeatureTypes/CombinePerKey/GroupByKey)<br/> -> Map (Key Extractor)",
> "inputs": [
> {
> "num": 0,
> "id": "c6485ed8e08800c2f8ae0ef7e6411255",
> "ship_strategy": "Forward",
> "exchange": "blocking"
> }
> ],
> "optimizer_properties": {
> "global_properties": [
> {
> "name": "Partitioning",
> "value": "RANDOM_PARTITIONED"
> },
> {
> "name": "Partitioning Order",
> "value": "(none)"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "local_properties": [
> {
> "name": "Order",
> "value": "(none)"
> },
> {
> "name": "Grouping",
> "value": "not grouped"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "estimates": [
> {
> "name": "Est. Output Size",
> "value": "(unknown)"
> },
> {
> "name": "Est. Cardinality",
> "value": "(unknown)"
> }
> ],
> "costs": [
> {
> "name": "Network",
> "value": "0.0"
> },
> {
> "name": "Disk I/O",
> "value": "0.0"
> },
> {
> "name": "CPU",
> "value": "0.0"
> },
> {
> "name": "Cumulative Network",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative Disk I/O",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative CPU",
> "value": "(unknown)"
> }
> ],
> "compiler_hints": [
> {
> "name": "Output Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Output Cardinality",
> "value": "(none)"
> },
> {
> "name": "Avg. Output Record Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Filter Factor",
> "value": "(none)"
> }
> ]
> }
> },
> {
> "id": "03641b59a0998396c201212176db6d3f",
> "parallelism": 1,
> "operator": "MapPartition -> FlatMap",
> "operator_strategy": "Map Partition<br/> -> FlatMap",
> "description": "MapPartition (MapPartition at 57WriteStatsOutput/Write/WriteImpl/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0)<br/> -> FlatMap (FlatMap at 57WriteStatsOutput/Write/WriteImpl/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0/out.0)",
> "inputs": [
> {
> "num": 0,
> "id": "a633dac13fbf4c7fd8bd521edc7ace83",
> "ship_strategy": "Forward",
> "exchange": "pipelined"
> },
> {
> "num": 1,
> "id": "12012f2c6a9d768ad5651fd2d01f4276",
> "ship_strategy": "Broadcast",
> "exchange": "blocking"
> }
> ],
> "optimizer_properties": {
> "global_properties": [
> {
> "name": "Partitioning",
> "value": "RANDOM_PARTITIONED"
> },
> {
> "name": "Partitioning Order",
> "value": "(none)"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "local_properties": [
> {
> "name": "Order",
> "value": "(none)"
> },
> {
> "name": "Grouping",
> "value": "not grouped"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "estimates": [
> {
> "name": "Est. Output Size",
> "value": "(unknown)"
> },
> {
> "name": "Est. Cardinality",
> "value": "(unknown)"
> }
> ],
> "costs": [
> {
> "name": "Network",
> "value": "0.0"
> },
> {
> "name": "Disk I/O",
> "value": "0.0"
> },
> {
> "name": "CPU",
> "value": "0.0"
> },
> {
> "name": "Cumulative Network",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative Disk I/O",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative CPU",
> "value": "(unknown)"
> }
> ],
> "compiler_hints": [
> {
> "name": "Output Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Output Cardinality",
> "value": "(none)"
> },
> {
> "name": "Avg. Output Record Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Filter Factor",
> "value": "(none)"
> }
> ]
> }
> },
> {
> "id": "510b5ba27b288217d588462cdfdb1c89",
> "parallelism": 1,
> "operator": "FlatMap -> Map -> GroupCombine -> Map",
> "operator_strategy": "FlatMap<br/> -> Map<br/> -> Sorted Combine<br/> -> Map",
> "description": "FlatMap (FlatMap at 26DecodeData/ParseCSVRecords.None/beam:env:docker:v1:0/out.3)<br/> -> Map (Key Extractor)<br/> -> GroupCombine (GroupCombine at GroupCombine: GenerateStatistics/RunStatsGenerators/TopKStatsGenerator/TopK_CountFeatureNameValueTuple/CombinePerKey(CountCombineFn)/GroupByKey)<br/> -> Map (Key Extractor)",
> "inputs": [
> {
> "num": 0,
> "id": "9e5d4b9b9aee5486a04c9c6fa18cb684",
> "ship_strategy": "Forward",
> "exchange": "blocking"
> }
> ],
> "optimizer_properties": {
> "global_properties": [
> {
> "name": "Partitioning",
> "value": "RANDOM_PARTITIONED"
> },
> {
> "name": "Partitioning Order",
> "value": "(none)"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "local_properties": [
> {
> "name": "Order",
> "value": "(none)"
> },
> {
> "name": "Grouping",
> "value": "not grouped"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "estimates": [
> {
> "name": "Est. Output Size",
> "value": "(unknown)"
> },
> {
> "name": "Est. Cardinality",
> "value": "(unknown)"
> }
> ],
> "costs": [
> {
> "name": "Network",
> "value": "0.0"
> },
> {
> "name": "Disk I/O",
> "value": "0.0"
> },
> {
> "name": "CPU",
> "value": "0.0"
> },
> {
> "name": "Cumulative Network",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative Disk I/O",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative CPU",
> "value": "(unknown)"
> }
> ],
> "compiler_hints": [
> {
> "name": "Output Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Output Cardinality",
> "value": "(none)"
> },
> {
> "name": "Avg. Output Record Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Filter Factor",
> "value": "(none)"
> }
> ]
> }
> },
> {
> "id": "e1d5f5310bd2666573d2b7c2faf170cb",
> "parallelism": 1,
> "operator": "Data Source",
> "operator_strategy": "(none)",
> "description": "DataSource (at GenerateStatistics/RunStatsGenerators/NumericStatsGenerator/DoOnce/Impulse (org.apache.beam.runners.flink.translation.wrappers.ImpulseInputFormat))",
> "optimizer_properties": {
> "global_properties": [
> {
> "name": "Partitioning",
> "value": "RANDOM_PARTITIONED"
> },
> {
> "name": "Partitioning Order",
> "value": "(none)"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "local_properties": [
> {
> "name": "Order",
> "value": "(none)"
> },
> {
> "name": "Grouping",
> "value": "not grouped"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "estimates": [
> {
> "name": "Est. Output Size",
> "value": "1.00"
> },
> {
> "name": "Est. Cardinality",
> "value": "1.00"
> }
> ],
> "costs": [
> {
> "name": "Network",
> "value": "0.0"
> },
> {
> "name": "Disk I/O",
> "value": "0.0"
> },
> {
> "name": "CPU",
> "value": "0.0"
> },
> {
> "name": "Cumulative Network",
> "value": "0.0"
> },
> {
> "name": "Cumulative Disk I/O",
> "value": "0.0"
> },
> {
> "name": "Cumulative CPU",
> "value": "0.0"
> }
> ],
> "compiler_hints": [
> {
> "name": "Output Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Output Cardinality",
> "value": "(none)"
> },
> {
> "name": "Avg. Output Record Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Filter Factor",
> "value": "(none)"
> }
> ]
> }
> },
> {
> "id": "18fc0b71278713b5c0f18c4299931157",
> "parallelism": 1,
> "operator": "MapPartition -> FlatMap",
> "operator_strategy": "Map Partition<br/> -> FlatMap",
> "description": "MapPartition (MapPartition at 122GenerateStatistics/RunStatsGenerators/TopKStatsGenerator/TopK_GetTopK/CombinePerKey(TopCombineFn)/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0)<br/> -> FlatMap (FlatMap at 122GenerateStatistics/RunStatsGenerators/TopKStatsGenerator/TopK_GetTopK/CombinePerKey(TopCombineFn)/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0/out.0)",
> "inputs": [
> {
> "num": 0,
> "id": "24ef3f0717eb591a4a46823c63b305c3",
> "ship_strategy": "Forward",
> "exchange": "pipelined"
> }
> ],
> "optimizer_properties": {
> "global_properties": [
> {
> "name": "Partitioning",
> "value": "RANDOM_PARTITIONED"
> },
> {
> "name": "Partitioning Order",
> "value": "(none)"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "local_properties": [
> {
> "name": "Order",
> "value": "(none)"
> },
> {
> "name": "Grouping",
> "value": "not grouped"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "estimates": [
> {
> "name": "Est. Output Size",
> "value": "(unknown)"
> },
> {
> "name": "Est. Cardinality",
> "value": "(unknown)"
> }
> ],
> "costs": [
> {
> "name": "Network",
> "value": "0.0"
> },
> {
> "name": "Disk I/O",
> "value": "0.0"
> },
> {
> "name": "CPU",
> "value": "0.0"
> },
> {
> "name": "Cumulative Network",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative Disk I/O",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative CPU",
> "value": "(unknown)"
> }
> ],
> "compiler_hints": [
> {
> "name": "Output Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Output Cardinality",
> "value": "(none)"
> },
> {
> "name": "Avg. Output Record Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Filter Factor",
> "value": "(none)"
> }
> ]
> }
> },
> {
> "id": "9e5d4b9b9aee5486a04c9c6fa18cb684",
> "parallelism": 1,
> "operator": "MapPartition",
> "operator_strategy": "Map Partition",
> "description": "MapPartition (MapPartition at 26DecodeData/ParseCSVRecords.None/beam:env:docker:v1:0)",
> "inputs": [
> {
> "num": 0,
> "id": "c6485ed8e08800c2f8ae0ef7e6411255",
> "ship_strategy": "Forward",
> "exchange": "blocking"
> },
> {
> "num": 1,
> "id": "7f96f31248e59e2ef247f45568169783",
> "ship_strategy": "Broadcast",
> "exchange": "pipelined"
> }
> ],
> "optimizer_properties": {
> "global_properties": [
> {
> "name": "Partitioning",
> "value": "RANDOM_PARTITIONED"
> },
> {
> "name": "Partitioning Order",
> "value": "(none)"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "local_properties": [
> {
> "name": "Order",
> "value": "(none)"
> },
> {
> "name": "Grouping",
> "value": "not grouped"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "estimates": [
> {
> "name": "Est. Output Size",
> "value": "(unknown)"
> },
> {
> "name": "Est. Cardinality",
> "value": "(unknown)"
> }
> ],
> "costs": [
> {
> "name": "Network",
> "value": "0.0"
> },
> {
> "name": "Disk I/O",
> "value": "0.0"
> },
> {
> "name": "CPU",
> "value": "0.0"
> },
> {
> "name": "Cumulative Network",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative Disk I/O",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative CPU",
> "value": "(unknown)"
> }
> ],
> "compiler_hints": [
> {
> "name": "Output Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Output Cardinality",
> "value": "(none)"
> },
> {
> "name": "Avg. Output Record Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Filter Factor",
> "value": "(none)"
> }
> ]
> }
> },
> {
> "id": "1e2ed63c2e5714f106bcbed1130c3957",
> "parallelism": 1,
> "operator": "MapPartition",
> "operator_strategy": "Map Partition",
> "description": "MapPartition (MapPartition at 51WriteStatsOutput/Write/WriteImpl/DoOnce/Map(decode).None/beam:env:docker:v1:2)",
> "inputs": [
> {
> "num": 0,
> "id": "766e63063a9b847725557fe8646ec382",
> "ship_strategy": "Forward",
> "exchange": "blocking"
> },
> {
> "num": 1,
> "id": "12012f2c6a9d768ad5651fd2d01f4276",
> "ship_strategy": "Broadcast",
> "exchange": "blocking"
> },
> {
> "num": 2,
> "id": "03641b59a0998396c201212176db6d3f",
> "ship_strategy": "Broadcast",
> "exchange": "blocking"
> },
> {
> "num": 3,
> "id": "55d331be8cfc0f9ab69860e8289ad9c9",
> "ship_strategy": "Broadcast",
> "exchange": "pipelined"
> }
> ],
> "optimizer_properties": {
> "global_properties": [
> {
> "name": "Partitioning",
> "value": "RANDOM_PARTITIONED"
> },
> {
> "name": "Partitioning Order",
> "value": "(none)"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "local_properties": [
> {
> "name": "Order",
> "value": "(none)"
> },
> {
> "name": "Grouping",
> "value": "not grouped"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "estimates": [
> {
> "name": "Est. Output Size",
> "value": "(unknown)"
> },
> {
> "name": "Est. Cardinality",
> "value": "(unknown)"
> }
> ],
> "costs": [
> {
> "name": "Network",
> "value": "0.0"
> },
> {
> "name": "Disk I/O",
> "value": "0.0"
> },
> {
> "name": "CPU",
> "value": "0.0"
> },
> {
> "name": "Cumulative Network",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative Disk I/O",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative CPU",
> "value": "(unknown)"
> }
> ],
> "compiler_hints": [
> {
> "name": "Output Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Output Cardinality",
> "value": "(none)"
> },
> {
> "name": "Avg. Output Record Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Filter Factor",
> "value": "(none)"
> }
> ]
> }
> },
> {
> "id": "d188f20c85ac7be031abc25e1fb5b148",
> "parallelism": 1,
> "operator": "MapPartition -> FlatMap",
> "operator_strategy": "Map Partition<br/> -> FlatMap",
> "description": "MapPartition (MapPartition at 88GenerateStatistics/RunStatsGenerators/MergeDatasetFeatureStatisticsProtos/DoOnce/Impulse.None/beam:env:docker:v1:0)<br/> -> FlatMap (FlatMap at 88GenerateStatistics/RunStatsGenerators/MergeDatasetFeatureStatisticsProtos/DoOnce/Impulse.None/beam:env:docker:v1:0/out.0)",
> "inputs": [
> {
> "num": 0,
> "id": "d4ffa588405cb2c615c35edbb049faee",
> "ship_strategy": "Forward",
> "exchange": "pipelined"
> }
> ],
> "optimizer_properties": {
> "global_properties": [
> {
> "name": "Partitioning",
> "value": "RANDOM_PARTITIONED"
> },
> {
> "name": "Partitioning Order",
> "value": "(none)"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "local_properties": [
> {
> "name": "Order",
> "value": "(none)"
> },
> {
> "name": "Grouping",
> "value": "not grouped"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "estimates": [
> {
> "name": "Est. Output Size",
> "value": "(unknown)"
> },
> {
> "name": "Est. Cardinality",
> "value": "(unknown)"
> }
> ],
> "costs": [
> {
> "name": "Network",
> "value": "0.0"
> },
> {
> "name": "Disk I/O",
> "value": "0.0"
> },
> {
> "name": "CPU",
> "value": "0.0"
> },
> {
> "name": "Cumulative Network",
> "value": "0.0"
> },
> {
> "name": "Cumulative Disk I/O",
> "value": "0.0"
> },
> {
> "name": "Cumulative CPU",
> "value": "0.0"
> }
> ],
> "compiler_hints": [
> {
> "name": "Output Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Output Cardinality",
> "value": "(none)"
> },
> {
> "name": "Avg. Output Record Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Filter Factor",
> "value": "(none)"
> }
> ]
> }
> },
> {
> "id": "7f96f31248e59e2ef247f45568169783",
> "parallelism": 1,
> "operator": "MapPartition -> FlatMap",
> "operator_strategy": "Map Partition<br/> -> FlatMap",
> "description": "MapPartition (MapPartition at 47DecodeData/InferFeatureTypes/DoOnce/Map(decode).None/beam:env:docker:v1:0)<br/> -> FlatMap (FlatMap at 47DecodeData/InferFeatureTypes/DoOnce/Map(decode).None/beam:env:docker:v1:0/out.0)",
> "inputs": [
> {
> "num": 0,
> "id": "298e41af9e9ebc1b922447d1d6473eaf",
> "ship_strategy": "Forward",
> "exchange": "pipelined"
> },
> {
> "num": 1,
> "id": "bf9097c62f41786b45bd4d64d00ecf33",
> "ship_strategy": "Broadcast",
> "exchange": "pipelined"
> }
> ],
> "optimizer_properties": {
> "global_properties": [
> {
> "name": "Partitioning",
> "value": "RANDOM_PARTITIONED"
> },
> {
> "name": "Partitioning Order",
> "value": "(none)"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "local_properties": [
> {
> "name": "Order",
> "value": "(none)"
> },
> {
> "name": "Grouping",
> "value": "not grouped"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "estimates": [
> {
> "name": "Est. Output Size",
> "value": "(unknown)"
> },
> {
> "name": "Est. Cardinality",
> "value": "(unknown)"
> }
> ],
> "costs": [
> {
> "name": "Network",
> "value": "0.0"
> },
> {
> "name": "Disk I/O",
> "value": "0.0"
> },
> {
> "name": "CPU",
> "value": "0.0"
> },
> {
> "name": "Cumulative Network",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative Disk I/O",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative CPU",
> "value": "(unknown)"
> }
> ],
> "compiler_hints": [
> {
> "name": "Output Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Output Cardinality",
> "value": "(none)"
> },
> {
> "name": "Avg. Output Record Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Filter Factor",
> "value": "(none)"
> }
> ]
> }
> },
> {
> "id": "59b4d70dcb043f75e8a71e0f60368044",
> "parallelism": 1,
> "operator": "GroupReduce",
> "operator_strategy": "Sorted Group Reduce",
> "description": "GroupReduce (GroupReduce at GenerateStatistics/RunStatsGenerators/TopKStatsGenerator/TopK_CountFeatureNameValueTuple/CombinePerKey(CountCombineFn)/GroupByKey)",
> "inputs": [
> {
> "num": 0,
> "id": "510b5ba27b288217d588462cdfdb1c89",
> "ship_strategy": "Hash Partition on [0]",
> "local_strategy": "Sort on [0:ASC]",
> "exchange": "pipelined"
> }
> ],
> "optimizer_properties": {
> "global_properties": [
> {
> "name": "Partitioning",
> "value": "RANDOM_PARTITIONED"
> },
> {
> "name": "Partitioning Order",
> "value": "(none)"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "local_properties": [
> {
> "name": "Order",
> "value": "(none)"
> },
> {
> "name": "Grouping",
> "value": "not grouped"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "estimates": [
> {
> "name": "Est. Output Size",
> "value": "(unknown)"
> },
> {
> "name": "Est. Cardinality",
> "value": "(unknown)"
> }
> ],
> "costs": [
> {
> "name": "Network",
> "value": "(unknown)"
> },
> {
> "name": "Disk I/O",
> "value": "(unknown)"
> },
> {
> "name": "CPU",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative Network",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative Disk I/O",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative CPU",
> "value": "(unknown)"
> }
> ],
> "compiler_hints": [
> {
> "name": "Output Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Output Cardinality",
> "value": "(none)"
> },
> {
> "name": "Avg. Output Record Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Filter Factor",
> "value": "(none)"
> }
> ]
> }
> },
> {
> "id": "6fbe81468dce31c8ca2853fdba8a8760",
> "parallelism": 1,
> "operator": "MapPartition -> FlatMap -> Map -> GroupCombine -> Map",
> "operator_strategy": "Map Partition<br/> -> FlatMap<br/> -> Map<br/> -> Sorted Combine<br/> -> Map",
> "description": "MapPartition (MapPartition at 149GenerateStatistics/RunStatsGenerators/UniquesStatsGenerator/Uniques_CountFeatureNameValueTuple/CombinePerKey(CountCombineFn)/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0)<br/> -> FlatMap (FlatMap at 149GenerateStatistics/RunStatsGenerators/UniquesStatsGenerator/Uniques_CountFeatureNameValueTuple/CombinePerKey(CountCombineFn)/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0/out.0)<br/> -> Map (Key Extractor)<br/> -> GroupCombine (GroupCombine at GroupCombine: GenerateStatistics/RunStatsGenerators/UniquesStatsGenerator/Uniques_CountPerFeatureName/CombinePerKey(CountCombineFn)/GroupByKey)<br/> -> Map (Key Extractor)",
> "inputs": [
> {
> "num": 0,
> "id": "988381bcd910a9dabb5670e82d9fde7f",
> "ship_strategy": "Forward",
> "exchange": "pipelined"
> }
> ],
> "optimizer_properties": {
> "global_properties": [
> {
> "name": "Partitioning",
> "value": "RANDOM_PARTITIONED"
> },
> {
> "name": "Partitioning Order",
> "value": "(none)"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "local_properties": [
> {
> "name": "Order",
> "value": "(none)"
> },
> {
> "name": "Grouping",
> "value": "not grouped"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "estimates": [
> {
> "name": "Est. Output Size",
> "value": "(unknown)"
> },
> {
> "name": "Est. Cardinality",
> "value": "(unknown)"
> }
> ],
> "costs": [
> {
> "name": "Network",
> "value": "0.0"
> },
> {
> "name": "Disk I/O",
> "value": "0.0"
> },
> {
> "name": "CPU",
> "value": "0.0"
> },
> {
> "name": "Cumulative Network",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative Disk I/O",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative CPU",
> "value": "(unknown)"
> }
> ],
> "compiler_hints": [
> {
> "name": "Output Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Output Cardinality",
> "value": "(none)"
> },
> {
> "name": "Avg. Output Record Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Filter Factor",
> "value": "(none)"
> }
> ]
> }
> },
> {
> "id": "68f880071c774596985fcd67729f0f1a",
> "parallelism": 1,
> "operator": "FlatMap -> Map -> GroupCombine -> Map",
> "operator_strategy": "FlatMap<br/> -> Map<br/> -> Sorted Combine<br/> -> Map",
> "description": "FlatMap (FlatMap at 26DecodeData/ParseCSVRecords.None/beam:env:docker:v1:0/out.2)<br/> -> Map (Key Extractor)<br/> -> GroupCombine (GroupCombine at GroupCombine: GenerateStatistics/RunStatsGenerators/StringStatsGenerator/CombinePerKey/GroupByKey)<br/> -> Map (Key Extractor)",
> "inputs": [
> {
> "num": 0,
> "id": "9e5d4b9b9aee5486a04c9c6fa18cb684",
> "ship_strategy": "Forward",
> "exchange": "blocking"
> }
> ],
> "optimizer_properties": {
> "global_properties": [
> {
> "name": "Partitioning",
> "value": "RANDOM_PARTITIONED"
> },
> {
> "name": "Partitioning Order",
> "value": "(none)"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "local_properties": [
> {
> "name": "Order",
> "value": "(none)"
> },
> {
> "name": "Grouping",
> "value": "not grouped"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "estimates": [
> {
> "name": "Est. Output Size",
> "value": "(unknown)"
> },
> {
> "name": "Est. Cardinality",
> "value": "(unknown)"
> }
> ],
> "costs": [
> {
> "name": "Network",
> "value": "0.0"
> },
> {
> "name": "Disk I/O",
> "value": "0.0"
> },
> {
> "name": "CPU",
> "value": "0.0"
> },
> {
> "name": "Cumulative Network",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative Disk I/O",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative CPU",
> "value": "(unknown)"
> }
> ],
> "compiler_hints": [
> {
> "name": "Output Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Output Cardinality",
> "value": "(none)"
> },
> {
> "name": "Avg. Output Record Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Filter Factor",
> "value": "(none)"
> }
> ]
> }
> },
> {
> "id": "f8111ccb6bd1714295230f2f26e70526",
> "parallelism": 1,
> "operator": "Data Source",
> "operator_strategy": "(none)",
> "description": "DataSource (at GenerateStatistics/RunStatsGenerators/CommonStatsGenerator/DoOnce/Impulse (org.apache.beam.runners.flink.translation.wrappers.ImpulseInputFormat))",
> "optimizer_properties": {
> "global_properties": [
> {
> "name": "Partitioning",
> "value": "RANDOM_PARTITIONED"
> },
> {
> "name": "Partitioning Order",
> "value": "(none)"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "local_properties": [
> {
> "name": "Order",
> "value": "(none)"
> },
> {
> "name": "Grouping",
> "value": "not grouped"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "estimates": [
> {
> "name": "Est. Output Size",
> "value": "1.00"
> },
> {
> "name": "Est. Cardinality",
> "value": "1.00"
> }
> ],
> "costs": [
> {
> "name": "Network",
> "value": "0.0"
> },
> {
> "name": "Disk I/O",
> "value": "0.0"
> },
> {
> "name": "CPU",
> "value": "0.0"
> },
> {
> "name": "Cumulative Network",
> "value": "0.0"
> },
> {
> "name": "Cumulative Disk I/O",
> "value": "0.0"
> },
> {
> "name": "Cumulative CPU",
> "value": "0.0"
> }
> ],
> "compiler_hints": [
> {
> "name": "Output Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Output Cardinality",
> "value": "(none)"
> },
> {
> "name": "Avg. Output Record Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Filter Factor",
> "value": "(none)"
> }
> ]
> }
> },
> {
> "id": "e2e9488c10ee82bb3299dff6498221d6",
> "parallelism": 1,
> "operator": "Data Source",
> "operator_strategy": "(none)",
> "description": "DataSource (at GenerateStatistics/RunStatsGenerators/StringStatsGenerator/DoOnce/Impulse (org.apache.beam.runners.flink.translation.wrappers.ImpulseInputFormat))",
> "optimizer_properties": {
> "global_properties": [
> {
> "name": "Partitioning",
> "value": "RANDOM_PARTITIONED"
> },
> {
> "name": "Partitioning Order",
> "value": "(none)"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "local_properties": [
> {
> "name": "Order",
> "value": "(none)"
> },
> {
> "name": "Grouping",
> "value": "not grouped"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "estimates": [
> {
> "name": "Est. Output Size",
> "value": "1.00"
> },
> {
> "name": "Est. Cardinality",
> "value": "1.00"
> }
> ],
> "costs": [
> {
> "name": "Network",
> "value": "0.0"
> },
> {
> "name": "Disk I/O",
> "value": "0.0"
> },
> {
> "name": "CPU",
> "value": "0.0"
> },
> {
> "name": "Cumulative Network",
> "value": "0.0"
> },
> {
> "name": "Cumulative Disk I/O",
> "value": "0.0"
> },
> {
> "name": "Cumulative CPU",
> "value": "0.0"
> }
> ],
> "compiler_hints": [
> {
> "name": "Output Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Output Cardinality",
> "value": "(none)"
> },
> {
> "name": "Avg. Output Record Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Filter Factor",
> "value": "(none)"
> }
> ]
> }
> },
> {
> "id": "b16f8e2ec206664fb6b95c49f57a04c2",
> "parallelism": 1,
> "operator": "MapPartition -> FlatMap -> Map -> GroupCombine -> Map",
> "operator_strategy": "Map Partition<br/> -> FlatMap<br/> -> Map<br/> -> Sorted Combine<br/> -> Map",
> "description": "MapPartition (MapPartition at 143GenerateStatistics/RunStatsGenerators/TopKStatsGenerator/TopK_CountFeatureNameValueTuple/CombinePerKey(CountCombineFn)/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0)<br/> -> FlatMap (FlatMap at 143GenerateStatistics/RunStatsGenerators/TopKStatsGenerator/TopK_CountFeatureNameValueTuple/CombinePerKey(CountCombineFn)/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0/out.0)<br/> -> Map (Key Extractor)<br/> -> GroupCombine (GroupCombine at GroupCombine: GenerateStatistics/RunStatsGenerators/TopKStatsGenerator/TopK_GetTopK/CombinePerKey(TopCombineFn)/GroupByKey)<br/> -> Map (Key Extractor)",
> "inputs": [
> {
> "num": 0,
> "id": "59b4d70dcb043f75e8a71e0f60368044",
> "ship_strategy": "Forward",
> "exchange": "pipelined"
> }
> ],
> "optimizer_properties": {
> "global_properties": [
> {
> "name": "Partitioning",
> "value": "RANDOM_PARTITIONED"
> },
> {
> "name": "Partitioning Order",
> "value": "(none)"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "local_properties": [
> {
> "name": "Order",
> "value": "(none)"
> },
> {
> "name": "Grouping",
> "value": "not grouped"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "estimates": [
> {
> "name": "Est. Output Size",
> "value": "(unknown)"
> },
> {
> "name": "Est. Cardinality",
> "value": "(unknown)"
> }
> ],
> "costs": [
> {
> "name": "Network",
> "value": "0.0"
> },
> {
> "name": "Disk I/O",
> "value": "0.0"
> },
> {
> "name": "CPU",
> "value": "0.0"
> },
> {
> "name": "Cumulative Network",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative Disk I/O",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative CPU",
> "value": "(unknown)"
> }
> ],
> "compiler_hints": [
> {
> "name": "Output Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Output Cardinality",
> "value": "(none)"
> },
> {
> "name": "Avg. Output Record Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Filter Factor",
> "value": "(none)"
> }
> ]
> }
> },
> {
> "id": "1e4111a2c49845717e0275467fc1f09f",
> "parallelism": 1,
> "operator": "GroupReduce",
> "operator_strategy": "Sorted Group Reduce",
> "description": "GroupReduce (GroupReduce at GenerateStatistics/RunStatsGenerators/MergeDatasetFeatureStatisticsProtos/CombinePerKey/GroupByKey)",
> "inputs": [
> {
> "num": 0,
> "id": "bb036e3688efb54b9875caf74fe7017b",
> "ship_strategy": "Hash Partition on [0]",
> "local_strategy": "Sort on [0:ASC]",
> "exchange": "pipelined"
> }
> ],
> "optimizer_properties": {
> "global_properties": [
> {
> "name": "Partitioning",
> "value": "RANDOM_PARTITIONED"
> },
> {
> "name": "Partitioning Order",
> "value": "(none)"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "local_properties": [
> {
> "name": "Order",
> "value": "(none)"
> },
> {
> "name": "Grouping",
> "value": "not grouped"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "estimates": [
> {
> "name": "Est. Output Size",
> "value": "(unknown)"
> },
> {
> "name": "Est. Cardinality",
> "value": "(unknown)"
> }
> ],
> "costs": [
> {
> "name": "Network",
> "value": "(unknown)"
> },
> {
> "name": "Disk I/O",
> "value": "(unknown)"
> },
> {
> "name": "CPU",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative Network",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative Disk I/O",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative CPU",
> "value": "(unknown)"
> }
> ],
> "compiler_hints": [
> {
> "name": "Output Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Output Cardinality",
> "value": "(none)"
> },
> {
> "name": "Avg. Output Record Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Filter Factor",
> "value": "(none)"
> }
> ]
> }
> },
> {
> "id": "b66363d1cd8ed075c6e4526d72af9176",
> "parallelism": 1,
> "operator": "GroupReduce",
> "operator_strategy": "Sorted Group Reduce",
> "description": "GroupReduce (GroupReduce at GenerateStatistics/RunStatsGenerators/NumericStatsGenerator/CombinePerKey/GroupByKey)",
> "inputs": [
> {
> "num": 0,
> "id": "16d3da7754788feb2d6b67fcfc3d9f40",
> "ship_strategy": "Hash Partition on [0]",
> "local_strategy": "Sort on [0:ASC]",
> "exchange": "pipelined"
> }
> ],
> "optimizer_properties": {
> "global_properties": [
> {
> "name": "Partitioning",
> "value": "RANDOM_PARTITIONED"
> },
> {
> "name": "Partitioning Order",
> "value": "(none)"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "local_properties": [
> {
> "name": "Order",
> "value": "(none)"
> },
> {
> "name": "Grouping",
> "value": "not grouped"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "estimates": [
> {
> "name": "Est. Output Size",
> "value": "(unknown)"
> },
> {
> "name": "Est. Cardinality",
> "value": "(unknown)"
> }
> ],
> "costs": [
> {
> "name": "Network",
> "value": "(unknown)"
> },
> {
> "name": "Disk I/O",
> "value": "(unknown)"
> },
> {
> "name": "CPU",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative Network",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative Disk I/O",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative CPU",
> "value": "(unknown)"
> }
> ],
> "compiler_hints": [
> {
> "name": "Output Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Output Cardinality",
> "value": "(none)"
> },
> {
> "name": "Avg. Output Record Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Filter Factor",
> "value": "(none)"
> }
> ]
> }
> },
> {
> "id": "2481c440912b84ca49e8f8b72da1752d",
> "parallelism": 1,
> "operator": "GroupReduce",
> "operator_strategy": "Sorted Group Reduce",
> "description": "GroupReduce (GroupReduce at GenerateStatistics/RunStatsGenerators/StringStatsGenerator/CombinePerKey/GroupByKey)",
> "inputs": [
> {
> "num": 0,
> "id": "68f880071c774596985fcd67729f0f1a",
> "ship_strategy": "Hash Partition on [0]",
> "local_strategy": "Sort on [0:ASC]",
> "exchange": "pipelined"
> }
> ],
> "optimizer_properties": {
> "global_properties": [
> {
> "name": "Partitioning",
> "value": "RANDOM_PARTITIONED"
> },
> {
> "name": "Partitioning Order",
> "value": "(none)"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "local_properties": [
> {
> "name": "Order",
> "value": "(none)"
> },
> {
> "name": "Grouping",
> "value": "not grouped"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "estimates": [
> {
> "name": "Est. Output Size",
> "value": "(unknown)"
> },
> {
> "name": "Est. Cardinality",
> "value": "(unknown)"
> }
> ],
> "costs": [
> {
> "name": "Network",
> "value": "(unknown)"
> },
> {
> "name": "Disk I/O",
> "value": "(unknown)"
> },
> {
> "name": "CPU",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative Network",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative Disk I/O",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative CPU",
> "value": "(unknown)"
> }
> ],
> "compiler_hints": [
> {
> "name": "Output Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Output Cardinality",
> "value": "(none)"
> },
> {
> "name": "Avg. Output Record Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Filter Factor",
> "value": "(none)"
> }
> ]
> }
> },
> {
> "id": "ab40a7291893738a7c7ecae2bcdd07a9",
> "parallelism": 1,
> "operator": "GroupReduce",
> "operator_strategy": "Sorted Group Reduce",
> "description": "GroupReduce (GroupReduce at DecodeData/InferFeatureTypes/CombinePerKey/GroupByKey)",
> "inputs": [
> {
> "num": 0,
> "id": "4983cfb36a4f03aebd68726bad8c05a5",
> "ship_strategy": "Hash Partition on [0]",
> "local_strategy": "Sort on [0:ASC]",
> "exchange": "pipelined"
> }
> ],
> "optimizer_properties": {
> "global_properties": [
> {
> "name": "Partitioning",
> "value": "RANDOM_PARTITIONED"
> },
> {
> "name": "Partitioning Order",
> "value": "(none)"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "local_properties": [
> {
> "name": "Order",
> "value": "(none)"
> },
> {
> "name": "Grouping",
> "value": "not grouped"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "estimates": [
> {
> "name": "Est. Output Size",
> "value": "(unknown)"
> },
> {
> "name": "Est. Cardinality",
> "value": "(unknown)"
> }
> ],
> "costs": [
> {
> "name": "Network",
> "value": "(unknown)"
> },
> {
> "name": "Disk I/O",
> "value": "(unknown)"
> },
> {
> "name": "CPU",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative Network",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative Disk I/O",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative CPU",
> "value": "(unknown)"
> }
> ],
> "compiler_hints": [
> {
> "name": "Output Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Output Cardinality",
> "value": "(none)"
> },
> {
> "name": "Avg. Output Record Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Filter Factor",
> "value": "(none)"
> }
> ]
> }
> },
> {
> "id": "3bd851095315b142e35707a497509f11",
> "parallelism": 1,
> "operator": "MapPartition -> FlatMap",
> "operator_strategy": "Map Partition<br/> -> FlatMap",
> "description": "MapPartition (MapPartition at 77GenerateStatistics/RunStatsGenerators/CommonStatsGenerator/DoOnce/Map(decode).None/beam:env:docker:v1:0)<br/> -> FlatMap (FlatMap at 77GenerateStatistics/RunStatsGenerators/CommonStatsGenerator/DoOnce/Map(decode).None/beam:env:docker:v1:0/out.0)",
> "inputs": [
> {
> "num": 0,
> "id": "3f481475a4117a27391074a639a24c77",
> "ship_strategy": "Forward",
> "exchange": "pipelined"
> },
> {
> "num": 1,
> "id": "ee2a7ee51779773ead459855cd6fdf9a",
> "ship_strategy": "Broadcast",
> "exchange": "pipelined"
> }
> ],
> "optimizer_properties": {
> "global_properties": [
> {
> "name": "Partitioning",
> "value": "RANDOM_PARTITIONED"
> },
> {
> "name": "Partitioning Order",
> "value": "(none)"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "local_properties": [
> {
> "name": "Order",
> "value": "(none)"
> },
> {
> "name": "Grouping",
> "value": "not grouped"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "estimates": [
> {
> "name": "Est. Output Size",
> "value": "(unknown)"
> },
> {
> "name": "Est. Cardinality",
> "value": "(unknown)"
> }
> ],
> "costs": [
> {
> "name": "Network",
> "value": "0.0"
> },
> {
> "name": "Disk I/O",
> "value": "0.0"
> },
> {
> "name": "CPU",
> "value": "0.0"
> },
> {
> "name": "Cumulative Network",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative Disk I/O",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative CPU",
> "value": "(unknown)"
> }
> ],
> "compiler_hints": [
> {
> "name": "Output Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Output Cardinality",
> "value": "(none)"
> },
> {
> "name": "Avg. Output Record Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Filter Factor",
> "value": "(none)"
> }
> ]
> }
> },
> {
> "id": "b81b935c3dbeb33defbe2a8cc43faf2a",
> "parallelism": 1,
> "operator": "Data Source",
> "operator_strategy": "(none)",
> "description": "DataSource (at WriteStatsOutput/Write/WriteImpl/DoOnce/Impulse (org.apache.beam.runners.flink.translation.wrappers.ImpulseInputFormat))",
> "optimizer_properties": {
> "global_properties": [
> {
> "name": "Partitioning",
> "value": "RANDOM_PARTITIONED"
> },
> {
> "name": "Partitioning Order",
> "value": "(none)"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "local_properties": [
> {
> "name": "Order",
> "value": "(none)"
> },
> {
> "name": "Grouping",
> "value": "not grouped"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "estimates": [
> {
> "name": "Est. Output Size",
> "value": "1.00"
> },
> {
> "name": "Est. Cardinality",
> "value": "1.00"
> }
> ],
> "costs": [
> {
> "name": "Network",
> "value": "0.0"
> },
> {
> "name": "Disk I/O",
> "value": "0.0"
> },
> {
> "name": "CPU",
> "value": "0.0"
> },
> {
> "name": "Cumulative Network",
> "value": "0.0"
> },
> {
> "name": "Cumulative Disk I/O",
> "value": "0.0"
> },
> {
> "name": "Cumulative CPU",
> "value": "0.0"
> }
> ],
> "compiler_hints": [
> {
> "name": "Output Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Output Cardinality",
> "value": "(none)"
> },
> {
> "name": "Avg. Output Record Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Filter Factor",
> "value": "(none)"
> }
> ]
> }
> },
> {
> "id": "6db900e63ce3b00bbed15ce6268976b0",
> "parallelism": 1,
> "operator": "MapPartition -> FlatMap",
> "operator_strategy": "Map Partition<br/> -> FlatMap",
> "description": "MapPartition (MapPartition at 77GenerateStatistics/RunStatsGenerators/StringStatsGenerator/DoOnce/Map(decode).None/beam:env:docker:v1:0)<br/> -> FlatMap (FlatMap at 77GenerateStatistics/RunStatsGenerators/StringStatsGenerator/DoOnce/Map(decode).None/beam:env:docker:v1:0/out.0)",
> "inputs": [
> {
> "num": 0,
> "id": "35bad9de5c3986d1aa37eb321996f9e1",
> "ship_strategy": "Forward",
> "exchange": "pipelined"
> },
> {
> "num": 1,
> "id": "e8d11f0a437ebd15b951c95626e4e052",
> "ship_strategy": "Broadcast",
> "exchange": "pipelined"
> }
> ],
> "optimizer_properties": {
> "global_properties": [
> {
> "name": "Partitioning",
> "value": "RANDOM_PARTITIONED"
> },
> {
> "name": "Partitioning Order",
> "value": "(none)"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "local_properties": [
> {
> "name": "Order",
> "value": "(none)"
> },
> {
> "name": "Grouping",
> "value": "not grouped"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "estimates": [
> {
> "name": "Est. Output Size",
> "value": "(unknown)"
> },
> {
> "name": "Est. Cardinality",
> "value": "(unknown)"
> }
> ],
> "costs": [
> {
> "name": "Network",
> "value": "0.0"
> },
> {
> "name": "Disk I/O",
> "value": "0.0"
> },
> {
> "name": "CPU",
> "value": "0.0"
> },
> {
> "name": "Cumulative Network",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative Disk I/O",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative CPU",
> "value": "(unknown)"
> }
> ],
> "compiler_hints": [
> {
> "name": "Output Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Output Cardinality",
> "value": "(none)"
> },
> {
> "name": "Avg. Output Record Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Filter Factor",
> "value": "(none)"
> }
> ]
> }
> },
> {
> "id": "35bad9de5c3986d1aa37eb321996f9e1",
> "parallelism": 1,
> "operator": "MapPartition -> FlatMap",
> "operator_strategy": "Map Partition<br/> -> FlatMap",
> "description": "MapPartition (MapPartition at 73GenerateStatistics/RunStatsGenerators/StringStatsGenerator/DoOnce/Impulse.None/beam:env:docker:v1:0)<br/> -> FlatMap (FlatMap at 73GenerateStatistics/RunStatsGenerators/StringStatsGenerator/DoOnce/Impulse.None/beam:env:docker:v1:0/out.0)",
> "inputs": [
> {
> "num": 0,
> "id": "e2e9488c10ee82bb3299dff6498221d6",
> "ship_strategy": "Forward",
> "exchange": "pipelined"
> }
> ],
> "optimizer_properties": {
> "global_properties": [
> {
> "name": "Partitioning",
> "value": "RANDOM_PARTITIONED"
> },
> {
> "name": "Partitioning Order",
> "value": "(none)"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "local_properties": [
> {
> "name": "Order",
> "value": "(none)"
> },
> {
> "name": "Grouping",
> "value": "not grouped"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "estimates": [
> {
> "name": "Est. Output Size",
> "value": "(unknown)"
> },
> {
> "name": "Est. Cardinality",
> "value": "(unknown)"
> }
> ],
> "costs": [
> {
> "name": "Network",
> "value": "0.0"
> },
> {
> "name": "Disk I/O",
> "value": "0.0"
> },
> {
> "name": "CPU",
> "value": "0.0"
> },
> {
> "name": "Cumulative Network",
> "value": "0.0"
> },
> {
> "name": "Cumulative Disk I/O",
> "value": "0.0"
> },
> {
> "name": "Cumulative CPU",
> "value": "0.0"
> }
> ],
> "compiler_hints": [
> {
> "name": "Output Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Output Cardinality",
> "value": "(none)"
> },
> {
> "name": "Avg. Output Record Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Filter Factor",
> "value": "(none)"
> }
> ]
> }
> },
> {
> "id": "e8d11f0a437ebd15b951c95626e4e052",
> "parallelism": 1,
> "operator": "MapPartition -> FlatMap",
> "operator_strategy": "Map Partition<br/> -> FlatMap",
> "description": "MapPartition (MapPartition at 97GenerateStatistics/RunStatsGenerators/StringStatsGenerator/CombinePerKey/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0)<br/> -> FlatMap (FlatMap at 97GenerateStatistics/RunStatsGenerators/StringStatsGenerator/CombinePerKey/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0/out.0)",
> "inputs": [
> {
> "num": 0,
> "id": "2481c440912b84ca49e8f8b72da1752d",
> "ship_strategy": "Forward",
> "exchange": "pipelined"
> }
> ],
> "optimizer_properties": {
> "global_properties": [
> {
> "name": "Partitioning",
> "value": "RANDOM_PARTITIONED"
> },
> {
> "name": "Partitioning Order",
> "value": "(none)"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "local_properties": [
> {
> "name": "Order",
> "value": "(none)"
> },
> {
> "name": "Grouping",
> "value": "not grouped"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "estimates": [
> {
> "name": "Est. Output Size",
> "value": "(unknown)"
> },
> {
> "name": "Est. Cardinality",
> "value": "(unknown)"
> }
> ],
> "costs": [
> {
> "name": "Network",
> "value": "0.0"
> },
> {
> "name": "Disk I/O",
> "value": "0.0"
> },
> {
> "name": "CPU",
> "value": "0.0"
> },
> {
> "name": "Cumulative Network",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative Disk I/O",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative CPU",
> "value": "(unknown)"
> }
> ],
> "compiler_hints": [
> {
> "name": "Output Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Output Cardinality",
> "value": "(none)"
> },
> {
> "name": "Avg. Output Record Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Filter Factor",
> "value": "(none)"
> }
> ]
> }
> },
> {
> "id": "27ea7d6f7814e0c565930b8fd6577fba",
> "parallelism": 1,
> "operator": "Data Sink",
> "operator_strategy": "(none)",
> "description": "DataSink (org.apache.flink.api.java.io.DiscardingOutputFormat@32ec80e7)",
> "inputs": [
> {
> "num": 0,
> "id": "1e2ed63c2e5714f106bcbed1130c3957",
> "ship_strategy": "Forward",
> "exchange": "pipelined"
> }
> ],
> "optimizer_properties": {
> "global_properties": [
> {
> "name": "Partitioning",
> "value": "RANDOM_PARTITIONED"
> },
> {
> "name": "Partitioning Order",
> "value": "(none)"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "local_properties": [
> {
> "name": "Order",
> "value": "(none)"
> },
> {
> "name": "Grouping",
> "value": "not grouped"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "estimates": [
> {
> "name": "Est. Output Size",
> "value": "(unknown)"
> },
> {
> "name": "Est. Cardinality",
> "value": "(unknown)"
> }
> ],
> "costs": [
> {
> "name": "Network",
> "value": "0.0"
> },
> {
> "name": "Disk I/O",
> "value": "0.0"
> },
> {
> "name": "CPU",
> "value": "0.0"
> },
> {
> "name": "Cumulative Network",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative Disk I/O",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative CPU",
> "value": "(unknown)"
> }
> ],
> "compiler_hints": [
> {
> "name": "Output Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Output Cardinality",
> "value": "(none)"
> },
> {
> "name": "Avg. Output Record Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Filter Factor",
> "value": "(none)"
> }
> ]
> }
> },
> {
> "id": "f6ef3a5be1f89eca633122d825dde2e4",
> "parallelism": 1,
> "operator": "MapPartition -> FlatMap",
> "operator_strategy": "Map Partition<br/> -> FlatMap",
> "description": "MapPartition (MapPartition at 112GenerateStatistics/RunStatsGenerators/MergeDatasetFeatureStatisticsProtos/CombinePerKey/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0)<br/> -> FlatMap (FlatMap at 112GenerateStatistics/RunStatsGenerators/MergeDatasetFeatureStatisticsProtos/CombinePerKey/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0/out.0)",
> "inputs": [
> {
> "num": 0,
> "id": "1e4111a2c49845717e0275467fc1f09f",
> "ship_strategy": "Forward",
> "exchange": "pipelined"
> }
> ],
> "optimizer_properties": {
> "global_properties": [
> {
> "name": "Partitioning",
> "value": "RANDOM_PARTITIONED"
> },
> {
> "name": "Partitioning Order",
> "value": "(none)"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "local_properties": [
> {
> "name": "Order",
> "value": "(none)"
> },
> {
> "name": "Grouping",
> "value": "not grouped"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "estimates": [
> {
> "name": "Est. Output Size",
> "value": "(unknown)"
> },
> {
> "name": "Est. Cardinality",
> "value": "(unknown)"
> }
> ],
> "costs": [
> {
> "name": "Network",
> "value": "0.0"
> },
> {
> "name": "Disk I/O",
> "value": "0.0"
> },
> {
> "name": "CPU",
> "value": "0.0"
> },
> {
> "name": "Cumulative Network",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative Disk I/O",
> "value": "(unknown)"
> },
> {
> "name": "Cumulative CPU",
> "value": "(unknown)"
> }
> ],
> "compiler_hints": [
> {
> "name": "Output Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Output Cardinality",
> "value": "(none)"
> },
> {
> "name": "Avg. Output Record Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Filter Factor",
> "value": "(none)"
> }
> ]
> }
> },
> {
> "id": "12012f2c6a9d768ad5651fd2d01f4276",
> "parallelism": 1,
> "operator": "MapPartition -> FlatMap",
> "operator_strategy": "Map Partition<br/> -> FlatMap",
> "description": "MapPartition (MapPartition at 51WriteStatsOutput/Write/WriteImpl/DoOnce/Map(decode).None/beam:env:docker:v1:0)<br/> -> FlatMap (FlatMap at 51WriteStatsOutput/Write/WriteImpl/DoOnce/Map(decode).None/beam:env:docker:v1:0/out.0)",
> "inputs": [
> {
> "num": 0,
> "id": "766e63063a9b847725557fe8646ec382",
> "ship_strategy": "Forward",
> "exchange": "blocking"
> }
> ],
> "optimizer_properties": {
> "global_properties": [
> {
> "name": "Partitioning",
> "value": "RANDOM_PARTITIONED"
> },
> {
> "name": "Partitioning Order",
> "value": "(none)"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "local_properties": [
> {
> "name": "Order",
> "value": "(none)"
> },
> {
> "name": "Grouping",
> "value": "not grouped"
> },
> {
> "name": "Uniqueness",
> "value": "not unique"
> }
> ],
> "estimates": [
> {
> "name": "Est. Output Size",
> "value": "(unknown)"
> },
> {
> "name": "Est. Cardinality",
> "value": "(unknown)"
> }
> ],
> "costs": [
> {
> "name": "Network",
> "value": "0.0"
> },
> {
> "name": "Disk I/O",
> "value": "0.0"
> },
> {
> "name": "CPU",
> "value": "0.0"
> },
> {
> "name": "Cumulative Network",
> "value": "0.0"
> },
> {
> "name": "Cumulative Disk I/O",
> "value": "0.0"
> },
> {
> "name": "Cumulative CPU",
> "value": "0.0"
> }
> ],
> "compiler_hints": [
> {
> "name": "Output Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Output Cardinality",
> "value": "(none)"
> },
> {
> "name": "Avg. Output Record Size (bytes)",
> "value": "(none)"
> },
> {
> "name": "Filter Factor",
> "value": "(none)"
> }
> ]
> }
> }
> ]
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)