[GitHub] [flink] godfreyhe opened a new pull request #10390: [FLINK-15004] [table-planner-blink] Choose SortMergeJoin and two-phase Aggregate if the row count is unknown

[GitBox <gi...@apache.org>]

godfreyhe opened a new pull request #10390: [FLINK-15004] [table-planner-blink] Choose SortMergeJoin and two-phase Aggregate if the row count is unknown
URL: https://github.com/apache/flink/pull/10390
 
 
   
   ## What is the purpose of the change
   
   *The blink planner will use default rowCount value (defined in FlinkPreparingTableBase#DEFAULT_ROWCOUNT) when the statistics is unknown, and maybe choose bad plan. It's better let planner choose a stable execution plan (such as using SortMergeJoin rather than HashJoin to avoid wrong build side, or using two-phase Aggregate rather than one-phase Aggregate to avoid data skew) if the statistics is unknown.*
   
   
   ## Brief change log
   
     - *add FlinkRelMdTableRealRowCountDetection to tell whether a RelNode has real row count*
     - *Choose SortMergeJoin if the row count is unknown*
     - *Choose two-phase aggregate when AggregatePhaseStrategy is AUTO and the row count is unknown*
   
   
   ## Verifying this change
   
   This change added tests and can be verified as follows:
   
   *(example:)*
     - *Extended integration test for recovery after master (JobManager) failure*
     - *Added FlinkRelMdTableRealRowCountDetectionTest that validates the result on different RelNode*
    - *Added JoinTest that validates the plan of join with or without row count*
    - *Extended AggregateTestBase validates the plan of aggregate with or without row count*
   
   ## Does this pull request potentially affect one of the following parts:
   
     - Dependencies (does it add or upgrade a dependency): (yes / **no)**
     - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (yes / **no)**
     - The serializers: (yes / **no** / don't know)
     - The runtime per-record code paths (performance sensitive): (yes / **no** / don't know)
     - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (yes / **no** / don't know)
     - The S3 file system connector: (yes / **no** / don't know)
   
   ## Documentation
   
     - Does this pull request introduce a new feature? (yes / **no)**
     - If yes, how is the feature documented? (not applicable / docs / JavaDocs / **not documented**)
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services