You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Shengkai Fang (Jira)" <ji...@apache.org> on 2023/03/21 02:26:00 UTC
[jira] [Comment Edited] (FLINK-31259) Gateway supports initialization of catalog at startup
[ https://issues.apache.org/jira/browse/FLINK-31259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17702994#comment-17702994 ]
Shengkai Fang edited comment on FLINK-31259 at 3/21/23 2:25 AM:
----------------------------------------------------------------
I apologize for the delay in my response. After discussing with [~jark] and [~lincoln.86xy] offline, we have come to the following agreements:
# The -i parameter in the gateway side lacks a clear meaning. The init script allows CREATE TABLE statements that may create a real table in external storage. However, since the CREATE TABLE statement can only be executed once, it is not suitable for the gateway side. Thus, the init script in the gateway differs significantly from its counterpart in the client side.
# FLIP-295 serves the same purpose as the -i parameter. It enables users to specify a special CatalogManager with built-in catalogs. The Gateway builds the special catalog manager for every session using the new CatalogManager API. For example, a platform developer can specify the following arguments in the flink-conf.yaml:
{code:java}
catalogmanager.type: mysql
catalogmanager.mysql.url: <URL>
catalogmanager.mysql.username: flink
catalogmanager.mysql.password: flink2023! {code}
With these configurations, the Gateway uses the SPI mechanism to load the special catalog manager, which is responsible for loading the built-in catalogs registered in the MySql database. I believe this also works for UDF.
In contrast, the -i parameter only works for SQL Gateway users, whereas FLIP-295 is useful for Table API users as well.
WDYT [~zjureel] [~lintingbin]? Please corret me if I am wrong.
was (Author: fsk119):
I apologize for the delay in my response. After discussing with [~jark] and [~lincoln.86xy] offline, we have come to the following agreements:
# The -i parameter in the gateway side lacks a clear meaning. The init script allows CREATE TABLE statements that may create a real table in external storage. However, since the CREATE TABLE statement can only be executed once, it is not suitable for the gateway side. Thus, the init script in the gateway differs significantly from its counterpart in the client side.
# FLIP-295 serves the same purpose as the -i parameter. It enables users to specify a special CatalogManager with built-in catalogs. The Gateway builds the special catalog manager for every session using the new CatalogManager API. For example, a platform developer can specify the following arguments in the flink-conf.yaml:
{code:java}
catalogmanager.type: mysql
catalogmanager.mysql.url: <URL>
catalogmanager.mysql.username: flink
catalogmanager.mysql.password: flink2023! {code}
With these configurations, the Gateway uses the SPI mechanism to load the special catalog manager, which is responsible for loading the built-in catalogs registered in the MySql database. I believe this also works for UDF.
In contrast, the -i parameter only works for SQL Gateway users, whereas FLIP-295 is useful for Table API users as well.
WDYT [~zjureel] [~lintingbin]? Please corret me if I am wrong.
> Gateway supports initialization of catalog at startup
> -----------------------------------------------------
>
> Key: FLINK-31259
> URL: https://issues.apache.org/jira/browse/FLINK-31259
> Project: Flink
> Issue Type: Improvement
> Components: Table SQL / Gateway
> Affects Versions: 1.18.0
> Reporter: Fang Yong
> Assignee: Fang Yong
> Priority: Major
> Labels: pull-request-available
>
> Support to initializing catalogs in gateway when it starts
--
This message was sent by Atlassian Jira
(v8.20.10#820010)