You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user-zh@flink.apache.org by Asahi Lee <97...@qq.com.INVALID> on 2021/07/26 08:54:15 UTC

回复： flink 1.13.1 sql hive is_generic = false 建表成功后，没有列信息

我使用flink 1.12.0版本功能是好的




------------------&nbsp;原始邮件&nbsp;------------------
发件人:                                                                                                                        "user-zh"                                                                                    <lirui.fudan@gmail.com&gt;;
发送时间:&nbsp;2021年7月26日(星期一) 下午4:43
收件人:&nbsp;"user-zh"<user-zh@flink.apache.org&gt;;

主题:&nbsp;Re: flink 1.13.1 sql hive is_generic = false 建表成功后，没有列信息



你好，

是否能尝试一下用hive dialect建表呢？

On Mon, Jul 26, 2021 at 2:44 PM Asahi Lee <978466273@qq.com.invalid&gt; wrote:

&gt; hi!
&gt; 我使用flink 1.13.1版本，通过sql创建hive表，程序正常运行，表创建成功，但是没有列信息；我的程序如下：
&gt; 我连接的是hive 2.3.6版本，使用flink-sql-connector-hive-2.3.6依赖包。
&gt;
&gt;
&gt; package com.meritdata.cloud.flink.test;
&gt;
&gt;
&gt; import org.apache.flink.table.api.EnvironmentSettings;
&gt; import org.apache.flink.table.api.TableEnvironment;
&gt;
&gt;
&gt; public class Test {
&gt;
&gt;
&gt; &amp;nbsp; &amp;nbsp; public static void main(String[] args) {
&gt;
&gt;
&gt; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; EnvironmentSettings bbSettings =
&gt; EnvironmentSettings.newInstance().useBlinkPlanner().inBatchMode().build();
&gt; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; TableEnvironment tableEnvironment =
&gt; TableEnvironment.create(bbSettings);
&gt;
&gt;
&gt; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; tableEnvironment.executeSql("create catalog
&gt; myhive with (" +
&gt; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; " 'type' = 'hive',
&gt; " +
&gt; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; "
&gt; 'default-database' = 'default', " +
&gt; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; ")");
&gt;
&gt;
&gt; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; tableEnvironment.executeSql("use catalog
&gt; myhive");
&gt; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; tableEnvironment.executeSql("create table if
&gt; not exists q1 " +
&gt; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; "( id string ) " +
&gt; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; "with
&gt; ('is_generic' = 'false')");
&gt;
&gt;
&gt; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; /**
&gt; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp;* hive上表创建成功，没有列, 信息如下
&gt; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp;* desc formatted q1;
&gt; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp;*
&gt; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp;* col_name&amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp;
&gt; &amp;nbsp; &amp;nbsp; &amp;nbsp; data_type&amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp;
&gt; &amp;nbsp;comment
&gt; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp;*
&gt; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp;*
&gt; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp;* Table Parameters:
&gt; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp;* flink.is_generic&amp;nbsp; &amp;nbsp; false
&gt; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp;* flink.schema.0.data-type&amp;nbsp;
&gt; VARCHAR(2147483647)
&gt; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp;* flink.schema.0.name&amp;nbsp; &amp;nbsp;id
&gt; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp;* transient_lastDdTime 1627279802
&gt; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp;*
&gt; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp;*/
&gt;
&gt;
&gt; &amp;nbsp; &amp;nbsp; }
&gt;
&gt;
&gt;
&gt;
&gt; }



-- 
Best regards!
Rui Li

Re: flink 1.13.1 sql hive is_generic = false 建表成功后，没有列信息

Posted by Rui Li <li...@gmail.com>.
1.13不再使用is_generic来标记是不是hive表了 （改成了'connector'='hive'）
，文档需要更新一下。不过还是建议DDL操作hive元数据时用hive dialect。

On Mon, Jul 26, 2021 at 5:00 PM Asahi Lee <97...@qq.com.invalid> wrote:

> 我使用flink 1.12.0版本功能是好的
>
>
>
>
> ------------------&nbsp;原始邮件&nbsp;------------------
> 发件人:
>                                                   "user-zh"
>                                                                     <
> lirui.fudan@gmail.com&gt;;
> 发送时间:&nbsp;2021年7月26日(星期一) 下午4:43
> 收件人:&nbsp;"user-zh"<user-zh@flink.apache.org&gt;;
>
> 主题:&nbsp;Re: flink 1.13.1 sql hive is_generic = false 建表成功后，没有列信息
>
>
>
> 你好，
>
> 是否能尝试一下用hive dialect建表呢？
>
> On Mon, Jul 26, 2021 at 2:44 PM Asahi Lee <978466273@qq.com.invalid&gt;
> wrote:
>
> &gt; hi!
> &gt; 我使用flink 1.13.1版本，通过sql创建hive表，程序正常运行，表创建成功，但是没有列信息；我的程序如下：
> &gt; 我连接的是hive 2.3.6版本，使用flink-sql-connector-hive-2.3.6依赖包。
> &gt;
> &gt;
> &gt; package com.meritdata.cloud.flink.test;
> &gt;
> &gt;
> &gt; import org.apache.flink.table.api.EnvironmentSettings;
> &gt; import org.apache.flink.table.api.TableEnvironment;
> &gt;
> &gt;
> &gt; public class Test {
> &gt;
> &gt;
> &gt; &amp;nbsp; &amp;nbsp; public static void main(String[] args) {
> &gt;
> &gt;
> &gt; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; EnvironmentSettings
> bbSettings =
> &gt;
> EnvironmentSettings.newInstance().useBlinkPlanner().inBatchMode().build();
> &gt; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; TableEnvironment
> tableEnvironment =
> &gt; TableEnvironment.create(bbSettings);
> &gt;
> &gt;
> &gt; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp;
> tableEnvironment.executeSql("create catalog
> &gt; myhive with (" +
> &gt; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp;
> &amp;nbsp; &amp;nbsp; " 'type' = 'hive',
> &gt; " +
> &gt; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp;
> &amp;nbsp; &amp;nbsp; "
> &gt; 'default-database' = 'default', " +
> &gt; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp;
> &amp;nbsp; &amp;nbsp; ")");
> &gt;
> &gt;
> &gt; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp;
> tableEnvironment.executeSql("use catalog
> &gt; myhive");
> &gt; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp;
> tableEnvironment.executeSql("create table if
> &gt; not exists q1 " +
> &gt; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp;
> &amp;nbsp; &amp;nbsp; "( id string ) " +
> &gt; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp;
> &amp;nbsp; &amp;nbsp; "with
> &gt; ('is_generic' = 'false')");
> &gt;
> &gt;
> &gt; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; /**
> &gt; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp;*
> hive上表创建成功，没有列, 信息如下
> &gt; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp;* desc
> formatted q1;
> &gt; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp;*
> &gt; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp;*
> col_name&amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp;
> &gt; &amp;nbsp; &amp;nbsp; &amp;nbsp; data_type&amp;nbsp; &amp;nbsp;
> &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp;
> &gt; &amp;nbsp;comment
> &gt; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp;*
> &gt; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp;*
> &gt; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp;* Table
> Parameters:
> &gt; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp;*
> flink.is_generic&amp;nbsp; &amp;nbsp; false
> &gt; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp;*
> flink.schema.0.data-type&amp;nbsp;
> &gt; VARCHAR(2147483647)
> &gt; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp;*
> flink.schema.0.name&amp;nbsp; &amp;nbsp;id
> &gt; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp;*
> transient_lastDdTime 1627279802
> &gt; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp;*
> &gt; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp;*/
> &gt;
> &gt;
> &gt; &amp;nbsp; &amp;nbsp; }
> &gt;
> &gt;
> &gt;
> &gt;
> &gt; }
>
>
>
> --
> Best regards!
> Rui Li



-- 
Best regards!
Rui Li