You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user-zh@flink.apache.org by Asahi Lee <97...@qq.com.INVALID> on 2021/07/26 06:43:35 UTC
flink 1.13.1 sql hive is_generic = false 建表成功后，没有列信息
hi!
我使用flink 1.13.1版本，通过sql创建hive表，程序正常运行，表创建成功，但是没有列信息；我的程序如下：
我连接的是hive 2.3.6版本，使用flink-sql-connector-hive-2.3.6依赖包。
package com.meritdata.cloud.flink.test;
import org.apache.flink.table.api.EnvironmentSettings;
import org.apache.flink.table.api.TableEnvironment;
public class Test {
public static void main(String[] args) {
EnvironmentSettings bbSettings = EnvironmentSettings.newInstance().useBlinkPlanner().inBatchMode().build();
TableEnvironment tableEnvironment = TableEnvironment.create(bbSettings);
tableEnvironment.executeSql("create catalog myhive with (" +
" 'type' = 'hive', " +
" 'default-database' = 'default', " +
")");
tableEnvironment.executeSql("use catalog myhive");
tableEnvironment.executeSql("create table if not exists q1 " +
"( id string ) " +
"with ('is_generic' = 'false')");
/**
* hive上表创建成功，没有列, 信息如下
* desc formatted q1;
*
* col_name data_type comment
*
*
* Table Parameters:
* flink.is_generic false
* flink.schema.0.data-type VARCHAR(2147483647)
* flink.schema.0.name id
* transient_lastDdTime 1627279802
*
*/
}
}
Re: flink 1.13.1 sql hive is_generic = false 建表成功后，没有列信息
Posted by Rui Li <li...@gmail.com>.
1.13不再使用is_generic来标记是不是hive表了 （改成了'connector'='hive'）
，文档需要更新一下。不过还是建议DDL操作hive元数据时用hive dialect。
On Mon, Jul 26, 2021 at 5:00 PM Asahi Lee <97...@qq.com.invalid> wrote:
> 我使用flink 1.12.0版本功能是好的
>
>
>
>
> ------------------ 原始邮件 ------------------
> 发件人:
> "user-zh"
> <
> lirui.fudan@gmail.com>;
> 发送时间: 2021年7月26日(星期一) 下午4:43
> 收件人: "user-zh"<user-zh@flink.apache.org>;
>
> 主题: Re: flink 1.13.1 sql hive is_generic = false 建表成功后，没有列信息
>
>
>
> 你好，
>
> 是否能尝试一下用hive dialect建表呢？
>
> On Mon, Jul 26, 2021 at 2:44 PM Asahi Lee <978466273@qq.com.invalid>
> wrote:
>
> > hi!
> > 我使用flink 1.13.1版本，通过sql创建hive表，程序正常运行，表创建成功，但是没有列信息；我的程序如下：
> > 我连接的是hive 2.3.6版本，使用flink-sql-connector-hive-2.3.6依赖包。
> >
> >
> > package com.meritdata.cloud.flink.test;
> >
> >
> > import org.apache.flink.table.api.EnvironmentSettings;
> > import org.apache.flink.table.api.TableEnvironment;
> >
> >
> > public class Test {
> >
> >
> > &nbsp; &nbsp; public static void main(String[] args) {
> >
> >
> > &nbsp; &nbsp; &nbsp; &nbsp; EnvironmentSettings
> bbSettings =
> >
> EnvironmentSettings.newInstance().useBlinkPlanner().inBatchMode().build();
> > &nbsp; &nbsp; &nbsp; &nbsp; TableEnvironment
> tableEnvironment =
> > TableEnvironment.create(bbSettings);
> >
> >
> > &nbsp; &nbsp; &nbsp; &nbsp;
> tableEnvironment.executeSql("create catalog
> > myhive with (" +
> > &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
> &nbsp; &nbsp; " 'type' = 'hive',
> > " +
> > &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
> &nbsp; &nbsp; "
> > 'default-database' = 'default', " +
> > &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
> &nbsp; &nbsp; ")");
> >
> >
> > &nbsp; &nbsp; &nbsp; &nbsp;
> tableEnvironment.executeSql("use catalog
> > myhive");
> > &nbsp; &nbsp; &nbsp; &nbsp;
> tableEnvironment.executeSql("create table if
> > not exists q1 " +
> > &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
> &nbsp; &nbsp; "( id string ) " +
> > &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
> &nbsp; &nbsp; "with
> > ('is_generic' = 'false')");
> >
> >
> > &nbsp; &nbsp; &nbsp; &nbsp; /**
> > &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;*
> hive上表创建成功，没有列, 信息如下
> > &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;* desc
> formatted q1;
> > &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;*
> > &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;*
> col_name&nbsp; &nbsp; &nbsp; &nbsp;
> > &nbsp; &nbsp; &nbsp; data_type&nbsp; &nbsp;
> &nbsp; &nbsp; &nbsp; &nbsp;
> > &nbsp;comment
> > &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;*
> > &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;*
> > &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;* Table
> Parameters:
> > &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;*
> flink.is_generic&nbsp; &nbsp; false
> > &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;*
> flink.schema.0.data-type&nbsp;
> > VARCHAR(2147483647)
> > &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;*
> flink.schema.0.name&nbsp; &nbsp;id
> > &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;*
> transient_lastDdTime 1627279802
> > &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;*
> > &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;*/
> >
> >
> > &nbsp; &nbsp; }
> >
> >
> >
> >
> > }
>
>
>
> --
> Best regards!
> Rui Li
--
Best regards!
Rui Li
回复： flink 1.13.1 sql hive is_generic = false 建表成功后，没有列信息
Posted by Asahi Lee <97...@qq.com.INVALID>.
我使用flink 1.12.0版本功能是好的
------------------ 原始邮件 ------------------
发件人: "user-zh" <lirui.fudan@gmail.com>;
发送时间: 2021年7月26日(星期一) 下午4:43
收件人: "user-zh"<user-zh@flink.apache.org>;
主题: Re: flink 1.13.1 sql hive is_generic = false 建表成功后，没有列信息
你好，
是否能尝试一下用hive dialect建表呢？
On Mon, Jul 26, 2021 at 2:44 PM Asahi Lee <978466273@qq.com.invalid> wrote:
> hi!
> 我使用flink 1.13.1版本，通过sql创建hive表，程序正常运行，表创建成功，但是没有列信息；我的程序如下：
> 我连接的是hive 2.3.6版本，使用flink-sql-connector-hive-2.3.6依赖包。
>
>
> package com.meritdata.cloud.flink.test;
>
>
> import org.apache.flink.table.api.EnvironmentSettings;
> import org.apache.flink.table.api.TableEnvironment;
>
>
> public class Test {
>
>
> &nbsp; &nbsp; public static void main(String[] args) {
>
>
> &nbsp; &nbsp; &nbsp; &nbsp; EnvironmentSettings bbSettings =
> EnvironmentSettings.newInstance().useBlinkPlanner().inBatchMode().build();
> &nbsp; &nbsp; &nbsp; &nbsp; TableEnvironment tableEnvironment =
> TableEnvironment.create(bbSettings);
>
>
> &nbsp; &nbsp; &nbsp; &nbsp; tableEnvironment.executeSql("create catalog
> myhive with (" +
> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; " 'type' = 'hive',
> " +
> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; "
> 'default-database' = 'default', " +
> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; ")");
>
>
> &nbsp; &nbsp; &nbsp; &nbsp; tableEnvironment.executeSql("use catalog
> myhive");
> &nbsp; &nbsp; &nbsp; &nbsp; tableEnvironment.executeSql("create table if
> not exists q1 " +
> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; "( id string ) " +
> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; "with
> ('is_generic' = 'false')");
>
>
> &nbsp; &nbsp; &nbsp; &nbsp; /**
> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;* hive上表创建成功，没有列, 信息如下
> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;* desc formatted q1;
> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;*
> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;* col_name&nbsp; &nbsp; &nbsp; &nbsp;
> &nbsp; &nbsp; &nbsp; data_type&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
> &nbsp;comment
> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;*
> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;*
> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;* Table Parameters:
> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;* flink.is_generic&nbsp; &nbsp; false
> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;* flink.schema.0.data-type&nbsp;
> VARCHAR(2147483647)
> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;* flink.schema.0.name&nbsp; &nbsp;id
> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;* transient_lastDdTime 1627279802
> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;*
> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;*/
>
>
> &nbsp; &nbsp; }
>
>
>
>
> }
--
Best regards!
Rui Li
Re: flink 1.13.1 sql hive is_generic = false 建表成功后，没有列信息
Posted by Rui Li <li...@gmail.com>.
你好，
是否能尝试一下用hive dialect建表呢？
On Mon, Jul 26, 2021 at 2:44 PM Asahi Lee <97...@qq.com.invalid> wrote:
> hi!
> 我使用flink 1.13.1版本，通过sql创建hive表，程序正常运行，表创建成功，但是没有列信息；我的程序如下：
> 我连接的是hive 2.3.6版本，使用flink-sql-connector-hive-2.3.6依赖包。
>
>
> package com.meritdata.cloud.flink.test;
>
>
> import org.apache.flink.table.api.EnvironmentSettings;
> import org.apache.flink.table.api.TableEnvironment;
>
>
> public class Test {
>
>
> public static void main(String[] args) {
>
>
> EnvironmentSettings bbSettings =
> EnvironmentSettings.newInstance().useBlinkPlanner().inBatchMode().build();
> TableEnvironment tableEnvironment =
> TableEnvironment.create(bbSettings);
>
>
> tableEnvironment.executeSql("create catalog
> myhive with (" +
> " 'type' = 'hive',
> " +
> "
> 'default-database' = 'default', " +
> ")");
>
>
> tableEnvironment.executeSql("use catalog
> myhive");
> tableEnvironment.executeSql("create table if
> not exists q1 " +
> "( id string ) " +
> "with
> ('is_generic' = 'false')");
>
>
> /**
> * hive上表创建成功，没有列, 信息如下
> * desc formatted q1;
> *
> * col_name
> data_type
> comment
> *
> *
> * Table Parameters:
> * flink.is_generic false
> * flink.schema.0.data-type
> VARCHAR(2147483647)
> * flink.schema.0.name id
> * transient_lastDdTime 1627279802
> *
> */
>
>
> }
>
>
>
>
> }
--
Best regards!
Rui Li