[jira] [Commented] (FLINK-26603) [Umbrella] Decouple Hive with Flink itself

["luoyuxia (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-26603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17504883#comment-17504883 ] 

luoyuxia commented on FLINK-26603:
----------------------------------

Just give a quick summary:

> there won't be any specific logic to Hive in planner module

1. Now,  there exists HiveAggSqlFunction/ HiveTableSqlFunction in planner module, they can be removed directly after resolve FLINK-26365, FLINK-26364.

2. In PlannerContext, there are two methods getSqlConformance, getCalciteSqlDialect contains the logic releated to Hive.  We may need an elegant to replace it.

3. There's LogicalDistribution/FlinkLogicalDistribution / BatchPhysicalDistributionRule.INSTANCE  that is specificed to Hive with the Hive's synax "distributed by".  We may need find a way that can make external dialect can define specific LogicNode/PhysicalNode.

 

> remove  flink-sql-parser-hive from flink-table module 

We can do this without much effort for all the logic can be covered in flink-connector-hive.

 

> remove the planner dependency in flink-connector-hive

There are some classes in flink-connector-hive reply on  planner, but fortunately, not too many.

It mainly rely on ParserImpl, PlannerContext, PlannerQueryOperation and so on.  The dependency is mainly required to create RelNode.

To resolve this problem,  we need more abstraction for planner and provides public API for external dialects.

 

I'll still work on it.

 

> [Umbrella] Decouple Hive with Flink itself
> ------------------------------------------
>
>                 Key: FLINK-26603
>                 URL: https://issues.apache.org/jira/browse/FLINK-26603
>             Project: Flink
>          Issue Type: Improvement
>          Components: Connectors / Hive, Table SQL / Planner
>            Reporter: luoyuxia
>            Priority: Major
>             Fix For: 1.16.0
>
>
> To support Hive dialect with Flink, we have implemented FLIP-123, FLIP-152.
> But it also brings much maintenance burden and complexity for it mixes some logic specific to Hive with Flink iteself. We should remove such logic from Flink iteself and make it totally decouple with Flink.
> With this ticket, we expect:
> 1:  there won't be any specific logic to Hive in planner module
> 2:  remove  flink-sql-parser-hive from flink-table module 
> 3:  remove the planner dependency in flink-connector-hive
> I'll update more details after investigation.
>  



--
This message was sent by Atlassian Jira
(v8.20.1#820001)