[jira] [Closed] (FLINK-31062) Release Testing: Verify FLINK-30365 New dynamic partition pruning strategy to support more dpp patterns

["Weijie Guo (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/FLINK-31062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Weijie Guo closed FLINK-31062.
------------------------------
    Resolution: Done

> Release Testing: Verify FLINK-30365 New dynamic partition pruning strategy to support more dpp patterns
> -------------------------------------------------------------------------------------------------------
>
>                 Key: FLINK-31062
>                 URL: https://issues.apache.org/jira/browse/FLINK-31062
>             Project: Flink
>          Issue Type: Sub-task
>          Components: Table SQL / Planner
>    Affects Versions: 1.17.0
>            Reporter: Yunhong Zheng
>            Assignee: Weijie Guo
>            Priority: Major
>             Fix For: 1.17.0
>
>         Attachments: dpp-test-result
>
>
> This issue aims to verify  the FLINK-31062: New dynamic partition pruning strategy to support more dpp patterns . This is an extension work of FLIP-248 : [https://cwiki.apache.org/confluence/display/FLINK/FLIP-248%3A+Introduce+dynamic+partition+pruning] to enable more query patterns support dynamic partition pruning, like subQuery, Union etc.
> This work can be verified in SQL client after we build the flink-disk package.
>  # create a partition table and a non-partition table (only hive connector is supported now), and then insert some data.
>  # you can verify this work by checking the query plan by executing 'explain'.  Most basic dpp patterns have been verified in Flink-1.16 [1]. So we only need to focus on verifying the newly supported patterns:
>  *  Test the case that the dim table and the fact table are not adjacent in the join order: disable join reorder by setting config: 'table.optimizer.join-reorder-enabled = false',  and write a query like the test 'DynamicPartitionPruningProgramTest.testDppWithoutJoinReorder()'. Dpp will success even if the fact table and dim table are not adjacent.
>  * Test the case that the dim side have one dim table but the dim side is an subQuery:  we can write the query like the test 'DynamicPartitionPruningProgramTest.testDppWithSubQuery()'. Dpp will success even if the dim side is a sub query (Notes: the dim side only can contain one same table with filter conddiition).
>  * Test the case that  there are 'Union' in fact side: we can write the query like the test 'DynamicPartitionPruningProgramTest.testDppWithUnionInFactSide()'. Dpp will success.
>  * Test the case that there are 'Agg' in fact side: we can write query as the tests in 'DynamicPartitionPruningProgramTest'. For this case, in order to ensure the semantic accuracy, we must require that the partition key cannot be in the group key of 'agg' operator. This case will not success.
>       3. execute the above plan and verify the execution result. (the execution result should be same with the execution plan which disable dynamic filtering via setting 'table.optimizer.dynamic-filtering.enabled = false'
>  
> [1] [https://issues.apache.org/jira/browse/FLINK-28940|http://example.com/]
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)