You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Xin Chen (Jira)" <ji...@apache.org> on 2023/10/21 10:25:00 UTC
[jira] [Comment Edited] (FLINK-27020) use hive dialect in SqlClient would thrown an error based on 1.15 version
[ https://issues.apache.org/jira/browse/FLINK-27020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17778050#comment-17778050 ]
Xin Chen edited comment on FLINK-27020 at 10/21/23 10:24 AM:
-------------------------------------------------------------
Sorry. I have a very confused question about that! Why do we have to implement the introduction of 'flink-table-planner-loader'('loader')? Just to adapt to different scala versions? *What would be the problem if I completely abandoned the loader in 1.16.2 and instead used the planner? Is that also completely feasible if I don't care about the scala version？*
The following is a detailed description of my problem：
I use Flink 1.16.2, when I checked the hive view, it involved the hive dialect because we directly used the Hive SQL parser in this scenario. 'HiveDialectFactory' (which belongs to the flink-connector-hive module) class needs to be loaded, but the new loader uses the componentClassloader and it cannot load this class. Therefore, the same error as above is thrown.
Of course, as stated in the official document, placing antlr runtime 3.5.2. jar in/lib and replace flink-table-planner-loader with flink-table-planner_2.12 can avoid this issue.
But this is not friendly to us. Our task code involves a lot of queries for hive views and hive tables, and querying hive tables is not affected. We can use either the planner or the loader, but querying hive views is affected. It is not reasonable and convenient to switch the jar package every time a use case of the hive view is executed. Is there a better way here?
During source code debugging, it was found that the main issue was that the corresponding 'HiveDialectFactory' could not be discovered here:
!image-2023-10-21-18-24-03-145.png!
was (Author: JIRAUSER298666):
Sorry. I have a very confused question about that! Why do we have to implement the introduction of 'flink-table-planner-loader'('loader')? Just to adapt to different scala versions? *What would be the problem if I completely abandoned the loader in 1.16.2 and instead used the planner? Is that also completely feasible if I don't care about the scala version？*
The following is a detailed description of my problem：
I use Flink 1.16.2, when I checked the hive view, it involved the hive dialect because we directly used the Hive SQL parser in this scenario. 'HiveDialectFactory' (which belongs to the flink-connector-hive module) class needs to be loaded, but the new loader uses the componentClassloader and it cannot load this class. Therefore, the same error as above is thrown.
Of course, as stated in the official document, placing antlr runtime 3.5.2. jar in/lib and replace flink-table-planner-loader with flink-table-planner_2.12 can avoid this issue.
But this is not friendly to us. Our task code involves a lot of queries for hive views and hive tables, and querying hive tables is not affected. We can use either the planner or the loader, but querying hive views is affected. It is not reasonable and convenient to switch the jar package every time a use case of the hive view is executed. Is there a better way here?
During source code debugging, it was found that the main issue was that the corresponding 'HiveDialectFactory' could not be discovered here:
!screenshot-1.png!
> use hive dialect in SqlClient would thrown an error based on 1.15 version
> -------------------------------------------------------------------------
>
> Key: FLINK-27020
> URL: https://issues.apache.org/jira/browse/FLINK-27020
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Client
> Affects Versions: 1.15.0
> Reporter: Jing Zhang
> Priority: Major
> Attachments: image-2022-04-02-20-28-01-335.png, image-2023-10-21-18-24-03-145.png, screenshot-1.png
>
>
> I use 1.15 rc0 and encounter a problem.
> An error would be thrown out if I use hive dialect in SqlClient.
> !image-2022-04-02-20-28-01-335.png!
> And I already add flink-sql-connector-hive-2.3.6_2.12-1.15-SNAPSHOT.jar.
> I note that, load and use hive module could work fine, but use hive dialect would fail.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)