[jira] [Commented] (FLINK-31575) Don't swap table-planner-loader and table-planner to use hive dialect

["luoyuxia (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-31575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17703658#comment-17703658 ] 

luoyuxia commented on FLINK-31575:
----------------------------------

To decouple Hive with Planner, in [FLIP-216|[https://cwiki.apache.org/confluence/display/FLINK/FLIP-216%3A++Introduce+pluggable+dialect+and+plan+for+migrating+Hive+dialect]], as part of decoupling, we propose to introduce a slim module called flink-table-calcite-bridge which contains the Calcite dependencies for writing planner plugins (e.g. SQL dialects) that interact with Calcite APIs.

 

Then the Hive connector will depend on the module flink-table-calcite-bridge, the flink-table-planner will also depend on flink-table-calcite-bridge. For decoupling, we can make flink-table-planner.jar pack the module flink-table-calcite-bridge.

But it will still require jar swap. With flink-table-planner-loader, the classes in flink-table-planner.jar will be loaded by a

[submoduleClassLoader|[https://github.com/apache/flink/blob/086f13a9cd625f856309d1d57d80a0b6a66c687b/flink-table/flink-table-planner-loader/src/main/java/org/apache/flink/table/planner/loader/PlannerModule.java#L117]].  So the calcite related classes will also be loaded by this submoduleClassLoader. 

 

But the classes releated to Hive dialect, like HiveParser will be loaded FlinkClassLoader(more exactly is AppClassLoader). HiveParser depends on calcite classes,  and then it'll try to load calcite classes but can't find them as they are not in FlinkClassLoader.

 

So, I would like to propose to make flink-table-calcite-bridge to be a seperate jar which only packs calcite dependencies. So that the calcite classes can be loaded and will loaded by same class loader whenever it's for default dialect and Hive dialect.

Hi, [~twalthr], what do you think of this idea?

 

 

 

 

 

 

> Don't swap table-planner-loader and table-planner to use hive dialect
> ---------------------------------------------------------------------
>
>                 Key: FLINK-31575
>                 URL: https://issues.apache.org/jira/browse/FLINK-31575
>             Project: Flink
>          Issue Type: Sub-task
>          Components: Connectors / Hive
>            Reporter: luoyuxia
>            Priority: Major
>
> From Flink 1.15,  to use Hive dialect, user have to swap the flink-table-planner-loader jar with flink-table-planner.jar.
> It really bothers some users who want to use Hive dialect like FLINK-27020, FLINK-28618
> Althogh we has paid much effort like FLINK-29350, FLINK-29045 to tell users to do the swap, but it'll still not convenient.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)