You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user-zh@flink.apache.org by 张锴 <zk...@gmail.com> on 2020/05/21 07:23:51 UTC
按照官网进行flink-shell操作，出现无法解决的错误：Only BatchTableSource and InputFormatTableSource are supported in BatchTableEnvironment.
具体操作及错误信息我贴到下面，各位大佬帮忙看下如何解决，不知道是不是BUG。
scala> import org.apache.flink.table.catalog.hive.HiveCatalog
import org.apache.flink.table.catalog.hive.HiveCatalog
scala> val hiveCatalog = new HiveCatalog("hive", "mydatabase",
"/opt/hive2.3.3/conf", "2.3.3");
hiveCatalog: org.apache.flink.table.catalog.hive.HiveCatalog =
org.apache.flink.table.catalog.hive.HiveCatalog@193f3306
scala> btenv.registerCatalog("hive", hiveCatalog)
Thu May 21 15:10:05 CST 2020 WARN: Establishing SSL connection without
server's identity verification is not recommended. According to MySQL
5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established
by default if explicit option isn't set. For compliance with existing
applications not using SSL the verifyServerCertificate property is set to
'false'. You need either to explicitly disable SSL by setting useSSL=false,
or set useSSL=true and provide truststore for server certificate
verification.
Thu May 21 15:10:05 CST 2020 WARN: Establishing SSL connection without
server's identity verification is not recommended. According to MySQL
5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established
by default if explicit option isn't set. For compliance with existing
applications not using SSL the verifyServerCertificate property is set to
'false'. You need either to explicitly disable SSL by setting useSSL=false,
or set useSSL=true and provide truststore for server certificate
verification.
scala> btenv.useCatalog("hive")
scala> btenv.listTables
res2: Array[String] = Array(mytable)
scala> btenv.sqlQuery("select * from mytable").toDataSet[Row].print()
org.apache.flink.table.api.TableException: Only BatchTableSource and
InputFormatTableSource are supported in BatchTableEnvironment.
at
org.apache.flink.table.plan.nodes.dataset.BatchTableSourceScan.translateToPlan(BatchTableSourceScan.scala:118)
at
org.apache.flink.table.api.internal.BatchTableEnvImpl.translate(BatchTableEnvImpl.scala:306)
at
org.apache.flink.table.api.internal.BatchTableEnvImpl.translate(BatchTableEnvImpl.scala:281)
at
org.apache.flink.table.api.scala.internal.BatchTableEnvironmentImpl.toDataSet(BatchTableEnvironmentImpl.scala:69)
at
org.apache.flink.table.api.scala.TableConversions.toDataSet(TableConversions.scala:53)
... 30 elided
Re: 按照官网进行flink-shell操作，出现无法解决的错误：Only BatchTableSource and InputFormatTableSource are supported in BatchTableEnvironment.
Posted by 张锴 <zk...@gmail.com>.
我想请问，在blink planner中，通过连接hive的方式将数据转成DataSet[Row]呢
Jingsong Li <ji...@gmail.com> 于2020年5月21日周四 下午4:43写道：
> Hi,
>
> 不好意思，现在版本hive connector已经不支持old planner了，
> 但是scala shell还是默认old planner。
>
> Best,
> Jingsong Lee
>
> On Thu, May 21, 2020 at 3:24 PM 张锴 <zk...@gmail.com> wrote:
>
> > 具体操作及错误信息我贴到下面，各位大佬帮忙看下如何解决，不知道是不是BUG。
> >
> > scala> import org.apache.flink.table.catalog.hive.HiveCatalog
> > import org.apache.flink.table.catalog.hive.HiveCatalog
> >
> > scala> val hiveCatalog = new HiveCatalog("hive", "mydatabase",
> > "/opt/hive2.3.3/conf", "2.3.3");
> > hiveCatalog: org.apache.flink.table.catalog.hive.HiveCatalog =
> > org.apache.flink.table.catalog.hive.HiveCatalog@193f3306
> >
> > scala> btenv.registerCatalog("hive", hiveCatalog)
> > Thu May 21 15:10:05 CST 2020 WARN: Establishing SSL connection without
> > server's identity verification is not recommended. According to MySQL
> > 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be
> established
> > by default if explicit option isn't set. For compliance with existing
> > applications not using SSL the verifyServerCertificate property is set to
> > 'false'. You need either to explicitly disable SSL by setting
> useSSL=false,
> > or set useSSL=true and provide truststore for server certificate
> > verification.
> > Thu May 21 15:10:05 CST 2020 WARN: Establishing SSL connection without
> > server's identity verification is not recommended. According to MySQL
> > 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be
> established
> > by default if explicit option isn't set. For compliance with existing
> > applications not using SSL the verifyServerCertificate property is set to
> > 'false'. You need either to explicitly disable SSL by setting
> useSSL=false,
> > or set useSSL=true and provide truststore for server certificate
> > verification.
> >
> > scala> btenv.useCatalog("hive")
> >
> > scala> btenv.listTables
> > res2: Array[String] = Array(mytable)
> >
> > scala> btenv.sqlQuery("select * from mytable").toDataSet[Row].print()
> > org.apache.flink.table.api.TableException: Only BatchTableSource and
> > InputFormatTableSource are supported in BatchTableEnvironment.
> > at
> >
> >
> org.apache.flink.table.plan.nodes.dataset.BatchTableSourceScan.translateToPlan(BatchTableSourceScan.scala:118)
> > at
> >
> >
> org.apache.flink.table.api.internal.BatchTableEnvImpl.translate(BatchTableEnvImpl.scala:306)
> > at
> >
> >
> org.apache.flink.table.api.internal.BatchTableEnvImpl.translate(BatchTableEnvImpl.scala:281)
> > at
> >
> >
> org.apache.flink.table.api.scala.internal.BatchTableEnvironmentImpl.toDataSet(BatchTableEnvironmentImpl.scala:69)
> > at
> >
> >
> org.apache.flink.table.api.scala.TableConversions.toDataSet(TableConversions.scala:53)
> > ... 30 elided
> >
>
>
> --
> Best, Jingsong Lee
>
Re: 按照官网进行flink-shell操作，出现无法解决的错误：Only BatchTableSource and InputFormatTableSource are supported in BatchTableEnvironment.
Posted by Jingsong Li <ji...@gmail.com>.
> 在blink planner中，通过连接hive的方式将数据转成DataSet[Row]呢
不能，社区正在做BoundedStream，在stream上完成DataSet的功能。
Best,
Jingsong Lee
On Thu, May 21, 2020 at 6:45 PM 张锴 <zk...@gmail.com> wrote:
> 我去看看
>
> Jeff Zhang <zj...@gmail.com> 于2020年5月21日周四 下午4:54写道：
>
> > 可以在zeppelin里写scala代码，是支持hive的，参考这个视频，
> > https://www.bilibili.com/video/BV1Te411W73b?p=10
> >
> > 也可以到这个钉钉群讨论: 30022475
> >
> > Jingsong Li <ji...@gmail.com> 于2020年5月21日周四 下午4:43写道：
> >
> > > Hi,
> > >
> > > 不好意思，现在版本hive connector已经不支持old planner了，
> > > 但是scala shell还是默认old planner。
> > >
> > > Best,
> > > Jingsong Lee
> > >
> > > On Thu, May 21, 2020 at 3:24 PM 张锴 <zk...@gmail.com> wrote:
> > >
> > > > 具体操作及错误信息我贴到下面，各位大佬帮忙看下如何解决，不知道是不是BUG。
> > > >
> > > > scala> import org.apache.flink.table.catalog.hive.HiveCatalog
> > > > import org.apache.flink.table.catalog.hive.HiveCatalog
> > > >
> > > > scala> val hiveCatalog = new HiveCatalog("hive", "mydatabase",
> > > > "/opt/hive2.3.3/conf", "2.3.3");
> > > > hiveCatalog: org.apache.flink.table.catalog.hive.HiveCatalog =
> > > > org.apache.flink.table.catalog.hive.HiveCatalog@193f3306
> > > >
> > > > scala> btenv.registerCatalog("hive", hiveCatalog)
> > > > Thu May 21 15:10:05 CST 2020 WARN: Establishing SSL connection
> without
> > > > server's identity verification is not recommended. According to MySQL
> > > > 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be
> > > established
> > > > by default if explicit option isn't set. For compliance with existing
> > > > applications not using SSL the verifyServerCertificate property is
> set
> > to
> > > > 'false'. You need either to explicitly disable SSL by setting
> > > useSSL=false,
> > > > or set useSSL=true and provide truststore for server certificate
> > > > verification.
> > > > Thu May 21 15:10:05 CST 2020 WARN: Establishing SSL connection
> without
> > > > server's identity verification is not recommended. According to MySQL
> > > > 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be
> > > established
> > > > by default if explicit option isn't set. For compliance with existing
> > > > applications not using SSL the verifyServerCertificate property is
> set
> > to
> > > > 'false'. You need either to explicitly disable SSL by setting
> > > useSSL=false,
> > > > or set useSSL=true and provide truststore for server certificate
> > > > verification.
> > > >
> > > > scala> btenv.useCatalog("hive")
> > > >
> > > > scala> btenv.listTables
> > > > res2: Array[String] = Array(mytable)
> > > >
> > > > scala> btenv.sqlQuery("select * from mytable").toDataSet[Row].print()
> > > > org.apache.flink.table.api.TableException: Only BatchTableSource and
> > > > InputFormatTableSource are supported in BatchTableEnvironment.
> > > > at
> > > >
> > > >
> > >
> >
> org.apache.flink.table.plan.nodes.dataset.BatchTableSourceScan.translateToPlan(BatchTableSourceScan.scala:118)
> > > > at
> > > >
> > > >
> > >
> >
> org.apache.flink.table.api.internal.BatchTableEnvImpl.translate(BatchTableEnvImpl.scala:306)
> > > > at
> > > >
> > > >
> > >
> >
> org.apache.flink.table.api.internal.BatchTableEnvImpl.translate(BatchTableEnvImpl.scala:281)
> > > > at
> > > >
> > > >
> > >
> >
> org.apache.flink.table.api.scala.internal.BatchTableEnvironmentImpl.toDataSet(BatchTableEnvironmentImpl.scala:69)
> > > > at
> > > >
> > > >
> > >
> >
> org.apache.flink.table.api.scala.TableConversions.toDataSet(TableConversions.scala:53)
> > > > ... 30 elided
> > > >
> > >
> > >
> > > --
> > > Best, Jingsong Lee
> > >
> >
> >
> > --
> > Best Regards
> >
> > Jeff Zhang
> >
>
--
Best, Jingsong Lee
Re: 按照官网进行flink-shell操作，出现无法解决的错误：Only BatchTableSource and InputFormatTableSource are supported in BatchTableEnvironment.
Posted by 张锴 <zk...@gmail.com>.
我去看看
Jeff Zhang <zj...@gmail.com> 于2020年5月21日周四 下午4:54写道：
> 可以在zeppelin里写scala代码，是支持hive的，参考这个视频，
> https://www.bilibili.com/video/BV1Te411W73b?p=10
>
> 也可以到这个钉钉群讨论: 30022475
>
> Jingsong Li <ji...@gmail.com> 于2020年5月21日周四 下午4:43写道：
>
> > Hi,
> >
> > 不好意思，现在版本hive connector已经不支持old planner了，
> > 但是scala shell还是默认old planner。
> >
> > Best,
> > Jingsong Lee
> >
> > On Thu, May 21, 2020 at 3:24 PM 张锴 <zk...@gmail.com> wrote:
> >
> > > 具体操作及错误信息我贴到下面，各位大佬帮忙看下如何解决，不知道是不是BUG。
> > >
> > > scala> import org.apache.flink.table.catalog.hive.HiveCatalog
> > > import org.apache.flink.table.catalog.hive.HiveCatalog
> > >
> > > scala> val hiveCatalog = new HiveCatalog("hive", "mydatabase",
> > > "/opt/hive2.3.3/conf", "2.3.3");
> > > hiveCatalog: org.apache.flink.table.catalog.hive.HiveCatalog =
> > > org.apache.flink.table.catalog.hive.HiveCatalog@193f3306
> > >
> > > scala> btenv.registerCatalog("hive", hiveCatalog)
> > > Thu May 21 15:10:05 CST 2020 WARN: Establishing SSL connection without
> > > server's identity verification is not recommended. According to MySQL
> > > 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be
> > established
> > > by default if explicit option isn't set. For compliance with existing
> > > applications not using SSL the verifyServerCertificate property is set
> to
> > > 'false'. You need either to explicitly disable SSL by setting
> > useSSL=false,
> > > or set useSSL=true and provide truststore for server certificate
> > > verification.
> > > Thu May 21 15:10:05 CST 2020 WARN: Establishing SSL connection without
> > > server's identity verification is not recommended. According to MySQL
> > > 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be
> > established
> > > by default if explicit option isn't set. For compliance with existing
> > > applications not using SSL the verifyServerCertificate property is set
> to
> > > 'false'. You need either to explicitly disable SSL by setting
> > useSSL=false,
> > > or set useSSL=true and provide truststore for server certificate
> > > verification.
> > >
> > > scala> btenv.useCatalog("hive")
> > >
> > > scala> btenv.listTables
> > > res2: Array[String] = Array(mytable)
> > >
> > > scala> btenv.sqlQuery("select * from mytable").toDataSet[Row].print()
> > > org.apache.flink.table.api.TableException: Only BatchTableSource and
> > > InputFormatTableSource are supported in BatchTableEnvironment.
> > > at
> > >
> > >
> >
> org.apache.flink.table.plan.nodes.dataset.BatchTableSourceScan.translateToPlan(BatchTableSourceScan.scala:118)
> > > at
> > >
> > >
> >
> org.apache.flink.table.api.internal.BatchTableEnvImpl.translate(BatchTableEnvImpl.scala:306)
> > > at
> > >
> > >
> >
> org.apache.flink.table.api.internal.BatchTableEnvImpl.translate(BatchTableEnvImpl.scala:281)
> > > at
> > >
> > >
> >
> org.apache.flink.table.api.scala.internal.BatchTableEnvironmentImpl.toDataSet(BatchTableEnvironmentImpl.scala:69)
> > > at
> > >
> > >
> >
> org.apache.flink.table.api.scala.TableConversions.toDataSet(TableConversions.scala:53)
> > > ... 30 elided
> > >
> >
> >
> > --
> > Best, Jingsong Lee
> >
>
>
> --
> Best Regards
>
> Jeff Zhang
>
Re: 按照官网进行flink-shell操作，出现无法解决的错误：Only BatchTableSource and InputFormatTableSource are supported in BatchTableEnvironment.
Posted by Jeff Zhang <zj...@gmail.com>.
可以在zeppelin里写scala代码，是支持hive的，参考这个视频，
https://www.bilibili.com/video/BV1Te411W73b?p=10
也可以到这个钉钉群讨论: 30022475
Jingsong Li <ji...@gmail.com> 于2020年5月21日周四 下午4:43写道：
> Hi,
>
> 不好意思，现在版本hive connector已经不支持old planner了，
> 但是scala shell还是默认old planner。
>
> Best,
> Jingsong Lee
>
> On Thu, May 21, 2020 at 3:24 PM 张锴 <zk...@gmail.com> wrote:
>
> > 具体操作及错误信息我贴到下面，各位大佬帮忙看下如何解决，不知道是不是BUG。
> >
> > scala> import org.apache.flink.table.catalog.hive.HiveCatalog
> > import org.apache.flink.table.catalog.hive.HiveCatalog
> >
> > scala> val hiveCatalog = new HiveCatalog("hive", "mydatabase",
> > "/opt/hive2.3.3/conf", "2.3.3");
> > hiveCatalog: org.apache.flink.table.catalog.hive.HiveCatalog =
> > org.apache.flink.table.catalog.hive.HiveCatalog@193f3306
> >
> > scala> btenv.registerCatalog("hive", hiveCatalog)
> > Thu May 21 15:10:05 CST 2020 WARN: Establishing SSL connection without
> > server's identity verification is not recommended. According to MySQL
> > 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be
> established
> > by default if explicit option isn't set. For compliance with existing
> > applications not using SSL the verifyServerCertificate property is set to
> > 'false'. You need either to explicitly disable SSL by setting
> useSSL=false,
> > or set useSSL=true and provide truststore for server certificate
> > verification.
> > Thu May 21 15:10:05 CST 2020 WARN: Establishing SSL connection without
> > server's identity verification is not recommended. According to MySQL
> > 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be
> established
> > by default if explicit option isn't set. For compliance with existing
> > applications not using SSL the verifyServerCertificate property is set to
> > 'false'. You need either to explicitly disable SSL by setting
> useSSL=false,
> > or set useSSL=true and provide truststore for server certificate
> > verification.
> >
> > scala> btenv.useCatalog("hive")
> >
> > scala> btenv.listTables
> > res2: Array[String] = Array(mytable)
> >
> > scala> btenv.sqlQuery("select * from mytable").toDataSet[Row].print()
> > org.apache.flink.table.api.TableException: Only BatchTableSource and
> > InputFormatTableSource are supported in BatchTableEnvironment.
> > at
> >
> >
> org.apache.flink.table.plan.nodes.dataset.BatchTableSourceScan.translateToPlan(BatchTableSourceScan.scala:118)
> > at
> >
> >
> org.apache.flink.table.api.internal.BatchTableEnvImpl.translate(BatchTableEnvImpl.scala:306)
> > at
> >
> >
> org.apache.flink.table.api.internal.BatchTableEnvImpl.translate(BatchTableEnvImpl.scala:281)
> > at
> >
> >
> org.apache.flink.table.api.scala.internal.BatchTableEnvironmentImpl.toDataSet(BatchTableEnvironmentImpl.scala:69)
> > at
> >
> >
> org.apache.flink.table.api.scala.TableConversions.toDataSet(TableConversions.scala:53)
> > ... 30 elided
> >
>
>
> --
> Best, Jingsong Lee
>
--
Best Regards
Jeff Zhang
Re: 按照官网进行flink-shell操作，出现无法解决的错误：Only BatchTableSource and InputFormatTableSource are supported in BatchTableEnvironment.
Posted by Jingsong Li <ji...@gmail.com>.
Hi,
不好意思，现在版本hive connector已经不支持old planner了，
但是scala shell还是默认old planner。
Best,
Jingsong Lee
On Thu, May 21, 2020 at 3:24 PM 张锴 <zk...@gmail.com> wrote:
> 具体操作及错误信息我贴到下面，各位大佬帮忙看下如何解决，不知道是不是BUG。
>
> scala> import org.apache.flink.table.catalog.hive.HiveCatalog
> import org.apache.flink.table.catalog.hive.HiveCatalog
>
> scala> val hiveCatalog = new HiveCatalog("hive", "mydatabase",
> "/opt/hive2.3.3/conf", "2.3.3");
> hiveCatalog: org.apache.flink.table.catalog.hive.HiveCatalog =
> org.apache.flink.table.catalog.hive.HiveCatalog@193f3306
>
> scala> btenv.registerCatalog("hive", hiveCatalog)
> Thu May 21 15:10:05 CST 2020 WARN: Establishing SSL connection without
> server's identity verification is not recommended. According to MySQL
> 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established
> by default if explicit option isn't set. For compliance with existing
> applications not using SSL the verifyServerCertificate property is set to
> 'false'. You need either to explicitly disable SSL by setting useSSL=false,
> or set useSSL=true and provide truststore for server certificate
> verification.
> Thu May 21 15:10:05 CST 2020 WARN: Establishing SSL connection without
> server's identity verification is not recommended. According to MySQL
> 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established
> by default if explicit option isn't set. For compliance with existing
> applications not using SSL the verifyServerCertificate property is set to
> 'false'. You need either to explicitly disable SSL by setting useSSL=false,
> or set useSSL=true and provide truststore for server certificate
> verification.
>
> scala> btenv.useCatalog("hive")
>
> scala> btenv.listTables
> res2: Array[String] = Array(mytable)
>
> scala> btenv.sqlQuery("select * from mytable").toDataSet[Row].print()
> org.apache.flink.table.api.TableException: Only BatchTableSource and
> InputFormatTableSource are supported in BatchTableEnvironment.
> at
>
> org.apache.flink.table.plan.nodes.dataset.BatchTableSourceScan.translateToPlan(BatchTableSourceScan.scala:118)
> at
>
> org.apache.flink.table.api.internal.BatchTableEnvImpl.translate(BatchTableEnvImpl.scala:306)
> at
>
> org.apache.flink.table.api.internal.BatchTableEnvImpl.translate(BatchTableEnvImpl.scala:281)
> at
>
> org.apache.flink.table.api.scala.internal.BatchTableEnvironmentImpl.toDataSet(BatchTableEnvironmentImpl.scala:69)
> at
>
> org.apache.flink.table.api.scala.TableConversions.toDataSet(TableConversions.scala:53)
> ... 30 elided
>
--
Best, Jingsong Lee