You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by kant kodali <ka...@gmail.com> on 2020/01/19 02:24:57 UTC
Does Flink 1.9 support create or replace views syntax in raw sql?
Hi All,
Does Flink 1.9 support create or replace views syntax in raw SQL? like
spark streaming does?
Thanks!
Re: Does Flink 1.9 support create or replace views syntax in raw sql?
Posted by godfrey he <go...@gmail.com>.
HI kant,
> 1) Where does the hive catalog persist view definitions? in mysql? or
HDFS?
hive catalog stores all metadata in derby or mysql [1]
> 2) If the views are not persisted what happens if the application crashes
and restarted? will it create the view again and safely read the data from
where it left off?
as Jark said, until to latest release version, the views are stored in
memory. So if the application crashes or is restarted, you must create the
view again.
[1] https://data-flair.training/blogs/apache-hive-metastore/
Bests,
Godfrey
kant kodali <ka...@gmail.com> 于2020年2月27日周四 下午9:25写道：
>
> https://ci.apache.org/projects/flink/flink-docs-release-1.10/dev/table/hive/hive_catalog.html#hivecatalog
>
> Can I use the hive catalog to store view definitions in HDFS? I am
> assuming the metastore can be anything or does it have to be have MySQL?
>
> On Thu, Feb 27, 2020 at 4:46 AM kant kodali <ka...@gmail.com> wrote:
>
>> Hi,
>>
>> 1) Where does the hive catalog persist view definitions? in mysql? or
>> HDFS?
>>
>> 2) If the views are not persisted what happens if the application crashes
>> and restarted? will it create the view again and safely read the data from
>> where it left off?
>>
>> Thanks!
>>
>> On Wed, Feb 26, 2020 at 6:47 AM godfrey he <go...@gmail.com> wrote:
>>
>>> Hi Kant, if you want the store the catalog data in Local
>>> Filesystem/HDFS, you can implement a user defined catalog (just need to
>>> implement Catalog interface)
>>>
>>> Bests,
>>> Godfrey
>>>
>>> kant kodali <ka...@gmail.com> 于2020年2月26日周三 下午12:28写道：
>>>
>>>> Hi Jingsong,
>>>>
>>>> Can I store it in Local Filesystem/HDFS?
>>>>
>>>> Thanks!
>>>>
>>>> On Mon, Jan 20, 2020 at 6:59 PM Jingsong Li <ji...@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi Kant,
>>>>>
>>>>> If you want your view persisted, you must to dock a catalog like hive
>>>>> catalog, it stores views in the metastore with mysql.
>>>>> - In 1.10, you can store views in catalog through
>>>>> "Catalog.createTable", you can create a "CatalogViewImpl". This is an
>>>>> internal API, which is not easy to use.
>>>>> - In 1.11, we will introduce create view DDL for "TableEnv.sqlUpdate"
>>>>> and "TableEnv.createView". It will be easy to use.
>>>>>
>>>>> Best,
>>>>> Jingsong Lee
>>>>>
>>>>> On Tue, Jan 21, 2020 at 10:03 AM Jark Wu <im...@gmail.com> wrote:
>>>>>
>>>>>> Hi Kant,
>>>>>>
>>>>>> The TableEnv#createTemporaryView and CREATE VIEW in SQL Cli both
>>>>>> creates temporary views which is not persisted and will lost after session
>>>>>> close.
>>>>>> I think the persisted views will be supported in 1.11.
>>>>>>
>>>>>> Best,
>>>>>> Jark
>>>>>>
>>>>>> 2020年1月20日 18:46，kant kodali <ka...@gmail.com> 写道：
>>>>>>
>>>>>> Hi Jingsong,
>>>>>>
>>>>>> Thanks a lot, I think I can live with
>>>>>> TableEnvironment.createTemporaryView in Flink 1.10 (which I am expecting to
>>>>>> be released this month) but are these views persisted somewhere? for
>>>>>> example across sessions? or say I stop my application and start again will
>>>>>> it work as expected?
>>>>>>
>>>>>> Thanks!
>>>>>>
>>>>>>
>>>>>> On Mon, Jan 20, 2020 at 1:12 AM Jingsong Li <ji...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Kant,
>>>>>>>
>>>>>>> Sorry, 1.10 not support "CREATE VIEW" in raw SQL too. Workaround is:
>>>>>>> - Using TableEnvironment.createTemporaryView...
>>>>>>> - Or using "create view" and "drop view" in the sql-client.
>>>>>>> - Or using hive catalog, in 1.10, we support query catalog views.
>>>>>>>
>>>>>>> FLIP-71 will be finished in 1.11 soon.
>>>>>>>
>>>>>>> Best,
>>>>>>> Jingsong Lee
>>>>>>>
>>>>>>> On Sun, Jan 19, 2020 at 4:10 PM kant kodali <ka...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> I tried the following.
>>>>>>>>
>>>>>>>> bsTableEnv.sqlUpdate("CREATE VIEW my_view AS SELECT * FROM sample1 FULL OUTER JOIN sample2 on sample1.f0=sample2.f0");
>>>>>>>>
>>>>>>>> Table result = bsTableEnv.sqlQuery("select * from my_view");
>>>>>>>>
>>>>>>>> It looks like
>>>>>>>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-71+-+E2E+View+support+in+FLINK+SQL Views
>>>>>>>> are not supported. Can I expect them to be supported in Flink 1.10?
>>>>>>>>
>>>>>>>> Currently, with Spark SQL when the query gets big I break it down
>>>>>>>> into views and this is one of the most important features my application
>>>>>>>> relies on. is there any workaround for this at the moment?
>>>>>>>>
>>>>>>>> Thanks!
>>>>>>>>
>>>>>>>> On Sat, Jan 18, 2020 at 6:24 PM kant kodali <ka...@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi All,
>>>>>>>>>
>>>>>>>>> Does Flink 1.9 support create or replace views syntax in raw SQL?
>>>>>>>>> like spark streaming does?
>>>>>>>>>
>>>>>>>>> Thanks!
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Best, Jingsong Lee
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>> --
>>>>> Best, Jingsong Lee
>>>>>
>>>>
Re: Does Flink 1.9 support create or replace views syntax in raw sql?
Posted by kant kodali <ka...@gmail.com>.
https://ci.apache.org/projects/flink/flink-docs-release-1.10/dev/table/hive/hive_catalog.html#hivecatalog
Can I use the hive catalog to store view definitions in HDFS? I am assuming
the metastore can be anything or does it have to be have MySQL?
On Thu, Feb 27, 2020 at 4:46 AM kant kodali <ka...@gmail.com> wrote:
> Hi,
>
> 1) Where does the hive catalog persist view definitions? in mysql? or HDFS?
>
> 2) If the views are not persisted what happens if the application crashes
> and restarted? will it create the view again and safely read the data from
> where it left off?
>
> Thanks!
>
> On Wed, Feb 26, 2020 at 6:47 AM godfrey he <go...@gmail.com> wrote:
>
>> Hi Kant, if you want the store the catalog data in Local Filesystem/HDFS,
>> you can implement a user defined catalog (just need to implement Catalog
>> interface)
>>
>> Bests,
>> Godfrey
>>
>> kant kodali <ka...@gmail.com> 于2020年2月26日周三 下午12:28写道：
>>
>>> Hi Jingsong,
>>>
>>> Can I store it in Local Filesystem/HDFS?
>>>
>>> Thanks!
>>>
>>> On Mon, Jan 20, 2020 at 6:59 PM Jingsong Li <ji...@gmail.com>
>>> wrote:
>>>
>>>> Hi Kant,
>>>>
>>>> If you want your view persisted, you must to dock a catalog like hive
>>>> catalog, it stores views in the metastore with mysql.
>>>> - In 1.10, you can store views in catalog through
>>>> "Catalog.createTable", you can create a "CatalogViewImpl". This is an
>>>> internal API, which is not easy to use.
>>>> - In 1.11, we will introduce create view DDL for "TableEnv.sqlUpdate"
>>>> and "TableEnv.createView". It will be easy to use.
>>>>
>>>> Best,
>>>> Jingsong Lee
>>>>
>>>> On Tue, Jan 21, 2020 at 10:03 AM Jark Wu <im...@gmail.com> wrote:
>>>>
>>>>> Hi Kant,
>>>>>
>>>>> The TableEnv#createTemporaryView and CREATE VIEW in SQL Cli both
>>>>> creates temporary views which is not persisted and will lost after session
>>>>> close.
>>>>> I think the persisted views will be supported in 1.11.
>>>>>
>>>>> Best,
>>>>> Jark
>>>>>
>>>>> 2020年1月20日 18:46，kant kodali <ka...@gmail.com> 写道：
>>>>>
>>>>> Hi Jingsong,
>>>>>
>>>>> Thanks a lot, I think I can live with
>>>>> TableEnvironment.createTemporaryView in Flink 1.10 (which I am expecting to
>>>>> be released this month) but are these views persisted somewhere? for
>>>>> example across sessions? or say I stop my application and start again will
>>>>> it work as expected?
>>>>>
>>>>> Thanks!
>>>>>
>>>>>
>>>>> On Mon, Jan 20, 2020 at 1:12 AM Jingsong Li <ji...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hi Kant,
>>>>>>
>>>>>> Sorry, 1.10 not support "CREATE VIEW" in raw SQL too. Workaround is:
>>>>>> - Using TableEnvironment.createTemporaryView...
>>>>>> - Or using "create view" and "drop view" in the sql-client.
>>>>>> - Or using hive catalog, in 1.10, we support query catalog views.
>>>>>>
>>>>>> FLIP-71 will be finished in 1.11 soon.
>>>>>>
>>>>>> Best,
>>>>>> Jingsong Lee
>>>>>>
>>>>>> On Sun, Jan 19, 2020 at 4:10 PM kant kodali <ka...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> I tried the following.
>>>>>>>
>>>>>>> bsTableEnv.sqlUpdate("CREATE VIEW my_view AS SELECT * FROM sample1 FULL OUTER JOIN sample2 on sample1.f0=sample2.f0");
>>>>>>>
>>>>>>> Table result = bsTableEnv.sqlQuery("select * from my_view");
>>>>>>>
>>>>>>> It looks like
>>>>>>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-71+-+E2E+View+support+in+FLINK+SQL Views
>>>>>>> are not supported. Can I expect them to be supported in Flink 1.10?
>>>>>>>
>>>>>>> Currently, with Spark SQL when the query gets big I break it down
>>>>>>> into views and this is one of the most important features my application
>>>>>>> relies on. is there any workaround for this at the moment?
>>>>>>>
>>>>>>> Thanks!
>>>>>>>
>>>>>>> On Sat, Jan 18, 2020 at 6:24 PM kant kodali <ka...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi All,
>>>>>>>>
>>>>>>>> Does Flink 1.9 support create or replace views syntax in raw SQL?
>>>>>>>> like spark streaming does?
>>>>>>>>
>>>>>>>> Thanks!
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>> --
>>>>>> Best, Jingsong Lee
>>>>>>
>>>>>
>>>>>
>>>>
>>>> --
>>>> Best, Jingsong Lee
>>>>
>>>
Re: Does Flink 1.9 support create or replace views syntax in raw sql?
Posted by kant kodali <ka...@gmail.com>.
Hi,
1) Where does the hive catalog persist view definitions? in mysql? or HDFS?
2) If the views are not persisted what happens if the application crashes
and restarted? will it create the view again and safely read the data from
where it left off?
Thanks!
On Wed, Feb 26, 2020 at 6:47 AM godfrey he <go...@gmail.com> wrote:
> Hi Kant, if you want the store the catalog data in Local Filesystem/HDFS,
> you can implement a user defined catalog (just need to implement Catalog
> interface)
>
> Bests,
> Godfrey
>
> kant kodali <ka...@gmail.com> 于2020年2月26日周三 下午12:28写道：
>
>> Hi Jingsong,
>>
>> Can I store it in Local Filesystem/HDFS?
>>
>> Thanks!
>>
>> On Mon, Jan 20, 2020 at 6:59 PM Jingsong Li <ji...@gmail.com>
>> wrote:
>>
>>> Hi Kant,
>>>
>>> If you want your view persisted, you must to dock a catalog like hive
>>> catalog, it stores views in the metastore with mysql.
>>> - In 1.10, you can store views in catalog through "Catalog.createTable",
>>> you can create a "CatalogViewImpl". This is an internal API, which is not
>>> easy to use.
>>> - In 1.11, we will introduce create view DDL for "TableEnv.sqlUpdate"
>>> and "TableEnv.createView". It will be easy to use.
>>>
>>> Best,
>>> Jingsong Lee
>>>
>>> On Tue, Jan 21, 2020 at 10:03 AM Jark Wu <im...@gmail.com> wrote:
>>>
>>>> Hi Kant,
>>>>
>>>> The TableEnv#createTemporaryView and CREATE VIEW in SQL Cli both
>>>> creates temporary views which is not persisted and will lost after session
>>>> close.
>>>> I think the persisted views will be supported in 1.11.
>>>>
>>>> Best,
>>>> Jark
>>>>
>>>> 2020年1月20日 18:46，kant kodali <ka...@gmail.com> 写道：
>>>>
>>>> Hi Jingsong,
>>>>
>>>> Thanks a lot, I think I can live with
>>>> TableEnvironment.createTemporaryView in Flink 1.10 (which I am expecting to
>>>> be released this month) but are these views persisted somewhere? for
>>>> example across sessions? or say I stop my application and start again will
>>>> it work as expected?
>>>>
>>>> Thanks!
>>>>
>>>>
>>>> On Mon, Jan 20, 2020 at 1:12 AM Jingsong Li <ji...@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi Kant,
>>>>>
>>>>> Sorry, 1.10 not support "CREATE VIEW" in raw SQL too. Workaround is:
>>>>> - Using TableEnvironment.createTemporaryView...
>>>>> - Or using "create view" and "drop view" in the sql-client.
>>>>> - Or using hive catalog, in 1.10, we support query catalog views.
>>>>>
>>>>> FLIP-71 will be finished in 1.11 soon.
>>>>>
>>>>> Best,
>>>>> Jingsong Lee
>>>>>
>>>>> On Sun, Jan 19, 2020 at 4:10 PM kant kodali <ka...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> I tried the following.
>>>>>>
>>>>>> bsTableEnv.sqlUpdate("CREATE VIEW my_view AS SELECT * FROM sample1 FULL OUTER JOIN sample2 on sample1.f0=sample2.f0");
>>>>>>
>>>>>> Table result = bsTableEnv.sqlQuery("select * from my_view");
>>>>>>
>>>>>> It looks like
>>>>>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-71+-+E2E+View+support+in+FLINK+SQL Views
>>>>>> are not supported. Can I expect them to be supported in Flink 1.10?
>>>>>>
>>>>>> Currently, with Spark SQL when the query gets big I break it down
>>>>>> into views and this is one of the most important features my application
>>>>>> relies on. is there any workaround for this at the moment?
>>>>>>
>>>>>> Thanks!
>>>>>>
>>>>>> On Sat, Jan 18, 2020 at 6:24 PM kant kodali <ka...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi All,
>>>>>>>
>>>>>>> Does Flink 1.9 support create or replace views syntax in raw SQL?
>>>>>>> like spark streaming does?
>>>>>>>
>>>>>>> Thanks!
>>>>>>>
>>>>>>
>>>>>
>>>>> --
>>>>> Best, Jingsong Lee
>>>>>
>>>>
>>>>
>>>
>>> --
>>> Best, Jingsong Lee
>>>
>>
Re: Does Flink 1.9 support create or replace views syntax in raw sql?
Posted by godfrey he <go...@gmail.com>.
Hi Kant, if you want the store the catalog data in Local Filesystem/HDFS,
you can implement a user defined catalog (just need to implement Catalog
interface)
Bests,
Godfrey
kant kodali <ka...@gmail.com> 于2020年2月26日周三 下午12:28写道：
> Hi Jingsong,
>
> Can I store it in Local Filesystem/HDFS?
>
> Thanks!
>
> On Mon, Jan 20, 2020 at 6:59 PM Jingsong Li <ji...@gmail.com>
> wrote:
>
>> Hi Kant,
>>
>> If you want your view persisted, you must to dock a catalog like hive
>> catalog, it stores views in the metastore with mysql.
>> - In 1.10, you can store views in catalog through "Catalog.createTable",
>> you can create a "CatalogViewImpl". This is an internal API, which is not
>> easy to use.
>> - In 1.11, we will introduce create view DDL for "TableEnv.sqlUpdate"
>> and "TableEnv.createView". It will be easy to use.
>>
>> Best,
>> Jingsong Lee
>>
>> On Tue, Jan 21, 2020 at 10:03 AM Jark Wu <im...@gmail.com> wrote:
>>
>>> Hi Kant,
>>>
>>> The TableEnv#createTemporaryView and CREATE VIEW in SQL Cli both creates
>>> temporary views which is not persisted and will lost after session close.
>>> I think the persisted views will be supported in 1.11.
>>>
>>> Best,
>>> Jark
>>>
>>> 2020年1月20日 18:46，kant kodali <ka...@gmail.com> 写道：
>>>
>>> Hi Jingsong,
>>>
>>> Thanks a lot, I think I can live with
>>> TableEnvironment.createTemporaryView in Flink 1.10 (which I am expecting to
>>> be released this month) but are these views persisted somewhere? for
>>> example across sessions? or say I stop my application and start again will
>>> it work as expected?
>>>
>>> Thanks!
>>>
>>>
>>> On Mon, Jan 20, 2020 at 1:12 AM Jingsong Li <ji...@gmail.com>
>>> wrote:
>>>
>>>> Hi Kant,
>>>>
>>>> Sorry, 1.10 not support "CREATE VIEW" in raw SQL too. Workaround is:
>>>> - Using TableEnvironment.createTemporaryView...
>>>> - Or using "create view" and "drop view" in the sql-client.
>>>> - Or using hive catalog, in 1.10, we support query catalog views.
>>>>
>>>> FLIP-71 will be finished in 1.11 soon.
>>>>
>>>> Best,
>>>> Jingsong Lee
>>>>
>>>> On Sun, Jan 19, 2020 at 4:10 PM kant kodali <ka...@gmail.com> wrote:
>>>>
>>>>> I tried the following.
>>>>>
>>>>> bsTableEnv.sqlUpdate("CREATE VIEW my_view AS SELECT * FROM sample1 FULL OUTER JOIN sample2 on sample1.f0=sample2.f0");
>>>>>
>>>>> Table result = bsTableEnv.sqlQuery("select * from my_view");
>>>>>
>>>>> It looks like
>>>>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-71+-+E2E+View+support+in+FLINK+SQL Views
>>>>> are not supported. Can I expect them to be supported in Flink 1.10?
>>>>>
>>>>> Currently, with Spark SQL when the query gets big I break it down into
>>>>> views and this is one of the most important features my application relies
>>>>> on. is there any workaround for this at the moment?
>>>>>
>>>>> Thanks!
>>>>>
>>>>> On Sat, Jan 18, 2020 at 6:24 PM kant kodali <ka...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hi All,
>>>>>>
>>>>>> Does Flink 1.9 support create or replace views syntax in raw SQL?
>>>>>> like spark streaming does?
>>>>>>
>>>>>> Thanks!
>>>>>>
>>>>>
>>>>
>>>> --
>>>> Best, Jingsong Lee
>>>>
>>>
>>>
>>
>> --
>> Best, Jingsong Lee
>>
>
Re: Does Flink 1.9 support create or replace views syntax in raw sql?
Posted by kant kodali <ka...@gmail.com>.
Hi Jingsong,
Can I store it in Local Filesystem/HDFS?
Thanks!
On Mon, Jan 20, 2020 at 6:59 PM Jingsong Li <ji...@gmail.com> wrote:
> Hi Kant,
>
> If you want your view persisted, you must to dock a catalog like hive
> catalog, it stores views in the metastore with mysql.
> - In 1.10, you can store views in catalog through "Catalog.createTable",
> you can create a "CatalogViewImpl". This is an internal API, which is not
> easy to use.
> - In 1.11, we will introduce create view DDL for "TableEnv.sqlUpdate"
> and "TableEnv.createView". It will be easy to use.
>
> Best,
> Jingsong Lee
>
> On Tue, Jan 21, 2020 at 10:03 AM Jark Wu <im...@gmail.com> wrote:
>
>> Hi Kant,
>>
>> The TableEnv#createTemporaryView and CREATE VIEW in SQL Cli both creates
>> temporary views which is not persisted and will lost after session close.
>> I think the persisted views will be supported in 1.11.
>>
>> Best,
>> Jark
>>
>> 2020年1月20日 18:46，kant kodali <ka...@gmail.com> 写道：
>>
>> Hi Jingsong,
>>
>> Thanks a lot, I think I can live with
>> TableEnvironment.createTemporaryView in Flink 1.10 (which I am expecting to
>> be released this month) but are these views persisted somewhere? for
>> example across sessions? or say I stop my application and start again will
>> it work as expected?
>>
>> Thanks!
>>
>>
>> On Mon, Jan 20, 2020 at 1:12 AM Jingsong Li <ji...@gmail.com>
>> wrote:
>>
>>> Hi Kant,
>>>
>>> Sorry, 1.10 not support "CREATE VIEW" in raw SQL too. Workaround is:
>>> - Using TableEnvironment.createTemporaryView...
>>> - Or using "create view" and "drop view" in the sql-client.
>>> - Or using hive catalog, in 1.10, we support query catalog views.
>>>
>>> FLIP-71 will be finished in 1.11 soon.
>>>
>>> Best,
>>> Jingsong Lee
>>>
>>> On Sun, Jan 19, 2020 at 4:10 PM kant kodali <ka...@gmail.com> wrote:
>>>
>>>> I tried the following.
>>>>
>>>> bsTableEnv.sqlUpdate("CREATE VIEW my_view AS SELECT * FROM sample1 FULL OUTER JOIN sample2 on sample1.f0=sample2.f0");
>>>>
>>>> Table result = bsTableEnv.sqlQuery("select * from my_view");
>>>>
>>>> It looks like
>>>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-71+-+E2E+View+support+in+FLINK+SQL Views
>>>> are not supported. Can I expect them to be supported in Flink 1.10?
>>>>
>>>> Currently, with Spark SQL when the query gets big I break it down into
>>>> views and this is one of the most important features my application relies
>>>> on. is there any workaround for this at the moment?
>>>>
>>>> Thanks!
>>>>
>>>> On Sat, Jan 18, 2020 at 6:24 PM kant kodali <ka...@gmail.com> wrote:
>>>>
>>>>> Hi All,
>>>>>
>>>>> Does Flink 1.9 support create or replace views syntax in raw SQL? like
>>>>> spark streaming does?
>>>>>
>>>>> Thanks!
>>>>>
>>>>
>>>
>>> --
>>> Best, Jingsong Lee
>>>
>>
>>
>
> --
> Best, Jingsong Lee
>
Re: Does Flink 1.9 support create or replace views syntax in raw sql?
Posted by Jingsong Li <ji...@gmail.com>.
Hi Kant,
If you want your view persisted, you must to dock a catalog like hive
catalog, it stores views in the metastore with mysql.
- In 1.10, you can store views in catalog through "Catalog.createTable",
you can create a "CatalogViewImpl". This is an internal API, which is not
easy to use.
- In 1.11, we will introduce create view DDL for "TableEnv.sqlUpdate"
and "TableEnv.createView". It will be easy to use.
Best,
Jingsong Lee
On Tue, Jan 21, 2020 at 10:03 AM Jark Wu <im...@gmail.com> wrote:
> Hi Kant,
>
> The TableEnv#createTemporaryView and CREATE VIEW in SQL Cli both creates
> temporary views which is not persisted and will lost after session close.
> I think the persisted views will be supported in 1.11.
>
> Best,
> Jark
>
> 2020年1月20日 18:46，kant kodali <ka...@gmail.com> 写道：
>
> Hi Jingsong,
>
> Thanks a lot, I think I can live with TableEnvironment.createTemporaryView
> in Flink 1.10 (which I am expecting to be released this month) but are
> these views persisted somewhere? for example across sessions? or say I stop
> my application and start again will it work as expected?
>
> Thanks!
>
>
> On Mon, Jan 20, 2020 at 1:12 AM Jingsong Li <ji...@gmail.com>
> wrote:
>
>> Hi Kant,
>>
>> Sorry, 1.10 not support "CREATE VIEW" in raw SQL too. Workaround is:
>> - Using TableEnvironment.createTemporaryView...
>> - Or using "create view" and "drop view" in the sql-client.
>> - Or using hive catalog, in 1.10, we support query catalog views.
>>
>> FLIP-71 will be finished in 1.11 soon.
>>
>> Best,
>> Jingsong Lee
>>
>> On Sun, Jan 19, 2020 at 4:10 PM kant kodali <ka...@gmail.com> wrote:
>>
>>> I tried the following.
>>>
>>> bsTableEnv.sqlUpdate("CREATE VIEW my_view AS SELECT * FROM sample1 FULL OUTER JOIN sample2 on sample1.f0=sample2.f0");
>>>
>>> Table result = bsTableEnv.sqlQuery("select * from my_view");
>>>
>>> It looks like
>>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-71+-+E2E+View+support+in+FLINK+SQL Views
>>> are not supported. Can I expect them to be supported in Flink 1.10?
>>>
>>> Currently, with Spark SQL when the query gets big I break it down into
>>> views and this is one of the most important features my application relies
>>> on. is there any workaround for this at the moment?
>>>
>>> Thanks!
>>>
>>> On Sat, Jan 18, 2020 at 6:24 PM kant kodali <ka...@gmail.com> wrote:
>>>
>>>> Hi All,
>>>>
>>>> Does Flink 1.9 support create or replace views syntax in raw SQL? like
>>>> spark streaming does?
>>>>
>>>> Thanks!
>>>>
>>>
>>
>> --
>> Best, Jingsong Lee
>>
>
>
--
Best, Jingsong Lee
Re: Does Flink 1.9 support create or replace views syntax in raw sql?
Posted by Jark Wu <im...@gmail.com>.
Hi Kant,
The TableEnv#createTemporaryView and CREATE VIEW in SQL Cli both creates temporary views which is not persisted and will lost after session close.
I think the persisted views will be supported in 1.11.
Best,
Jark
> 2020年1月20日 18:46，kant kodali <ka...@gmail.com> 写道：
>
> Hi Jingsong,
>
> Thanks a lot, I think I can live with TableEnvironment.createTemporaryView in Flink 1.10 (which I am expecting to be released this month) but are these views persisted somewhere? for example across sessions? or say I stop my application and start again will it work as expected?
>
> Thanks!
>
>
> On Mon, Jan 20, 2020 at 1:12 AM Jingsong Li <jingsonglee0@gmail.com <ma...@gmail.com>> wrote:
> Hi Kant,
>
> Sorry, 1.10 not support "CREATE VIEW" in raw SQL too. Workaround is:
> - Using TableEnvironment.createTemporaryView...
> - Or using "create view" and "drop view" in the sql-client.
> - Or using hive catalog, in 1.10, we support query catalog views.
>
> FLIP-71 will be finished in 1.11 soon.
>
> Best,
> Jingsong Lee
>
> On Sun, Jan 19, 2020 at 4:10 PM kant kodali <kanth909@gmail.com <ma...@gmail.com>> wrote:
> I tried the following.
> bsTableEnv.sqlUpdate("CREATE VIEW my_view AS SELECT * FROM sample1 FULL OUTER JOIN sample2 on sample1.f0=sample2.f0");
>
> Table result = bsTableEnv.sqlQuery("select * from my_view");
> It looks like https://cwiki.apache.org/confluence/display/FLINK/FLIP-71+-+E2E+View+support+in+FLINK+SQL <https://cwiki.apache.org/confluence/display/FLINK/FLIP-71+-+E2E+View+support+in+FLINK+SQL> Views are not supported. Can I expect them to be supported in Flink 1.10?
>
> Currently, with Spark SQL when the query gets big I break it down into views and this is one of the most important features my application relies on. is there any workaround for this at the moment?
>
> Thanks!
>
> On Sat, Jan 18, 2020 at 6:24 PM kant kodali <kanth909@gmail.com <ma...@gmail.com>> wrote:
> Hi All,
>
> Does Flink 1.9 support create or replace views syntax in raw SQL? like spark streaming does?
>
> Thanks!
>
>
> --
> Best, Jingsong Lee
Re: Does Flink 1.9 support create or replace views syntax in raw sql?
Posted by kant kodali <ka...@gmail.com>.
Hi Jingsong,
Thanks a lot, I think I can live with TableEnvironment.createTemporaryView
in Flink 1.10 (which I am expecting to be released this month) but are
these views persisted somewhere? for example across sessions? or say I stop
my application and start again will it work as expected?
Thanks!
On Mon, Jan 20, 2020 at 1:12 AM Jingsong Li <ji...@gmail.com> wrote:
> Hi Kant,
>
> Sorry, 1.10 not support "CREATE VIEW" in raw SQL too. Workaround is:
> - Using TableEnvironment.createTemporaryView...
> - Or using "create view" and "drop view" in the sql-client.
> - Or using hive catalog, in 1.10, we support query catalog views.
>
> FLIP-71 will be finished in 1.11 soon.
>
> Best,
> Jingsong Lee
>
> On Sun, Jan 19, 2020 at 4:10 PM kant kodali <ka...@gmail.com> wrote:
>
>> I tried the following.
>>
>> bsTableEnv.sqlUpdate("CREATE VIEW my_view AS SELECT * FROM sample1 FULL OUTER JOIN sample2 on sample1.f0=sample2.f0");
>>
>> Table result = bsTableEnv.sqlQuery("select * from my_view");
>>
>> It looks like
>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-71+-+E2E+View+support+in+FLINK+SQL Views
>> are not supported. Can I expect them to be supported in Flink 1.10?
>>
>> Currently, with Spark SQL when the query gets big I break it down into
>> views and this is one of the most important features my application relies
>> on. is there any workaround for this at the moment?
>>
>> Thanks!
>>
>> On Sat, Jan 18, 2020 at 6:24 PM kant kodali <ka...@gmail.com> wrote:
>>
>>> Hi All,
>>>
>>> Does Flink 1.9 support create or replace views syntax in raw SQL? like
>>> spark streaming does?
>>>
>>> Thanks!
>>>
>>
>
> --
> Best, Jingsong Lee
>
Re: Does Flink 1.9 support create or replace views syntax in raw sql?
Posted by Jingsong Li <ji...@gmail.com>.
Hi Kant,
Sorry, 1.10 not support "CREATE VIEW" in raw SQL too. Workaround is:
- Using TableEnvironment.createTemporaryView...
- Or using "create view" and "drop view" in the sql-client.
- Or using hive catalog, in 1.10, we support query catalog views.
FLIP-71 will be finished in 1.11 soon.
Best,
Jingsong Lee
On Sun, Jan 19, 2020 at 4:10 PM kant kodali <ka...@gmail.com> wrote:
> I tried the following.
>
> bsTableEnv.sqlUpdate("CREATE VIEW my_view AS SELECT * FROM sample1 FULL OUTER JOIN sample2 on sample1.f0=sample2.f0");
>
> Table result = bsTableEnv.sqlQuery("select * from my_view");
>
> It looks like
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-71+-+E2E+View+support+in+FLINK+SQL Views
> are not supported. Can I expect them to be supported in Flink 1.10?
>
> Currently, with Spark SQL when the query gets big I break it down into
> views and this is one of the most important features my application relies
> on. is there any workaround for this at the moment?
>
> Thanks!
>
> On Sat, Jan 18, 2020 at 6:24 PM kant kodali <ka...@gmail.com> wrote:
>
>> Hi All,
>>
>> Does Flink 1.9 support create or replace views syntax in raw SQL? like
>> spark streaming does?
>>
>> Thanks!
>>
>
--
Best, Jingsong Lee
Re: Does Flink 1.9 support create or replace views syntax in raw sql?
Posted by kant kodali <ka...@gmail.com>.
I tried the following.
bsTableEnv.sqlUpdate("CREATE VIEW my_view AS SELECT * FROM sample1
FULL OUTER JOIN sample2 on sample1.f0=sample2.f0");
Table result = bsTableEnv.sqlQuery("select * from my_view");
It looks like
https://cwiki.apache.org/confluence/display/FLINK/FLIP-71+-+E2E+View+support+in+FLINK+SQL
Views
are not supported. Can I expect them to be supported in Flink 1.10?
Currently, with Spark SQL when the query gets big I break it down into
views and this is one of the most important features my application relies
on. is there any workaround for this at the moment?
Thanks!
On Sat, Jan 18, 2020 at 6:24 PM kant kodali <ka...@gmail.com> wrote:
> Hi All,
>
> Does Flink 1.9 support create or replace views syntax in raw SQL? like
> spark streaming does?
>
> Thanks!
>