You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flink.apache.org by Gyula Fóra <gy...@gmail.com> on 2020/03/03 13:16:33 UTC
Creating TemporalTable based on Catalog table in SQL Client
Hi all!
I was testing the TemporalTable functionality in the SQL client while using
the Hive Catalog and I ran into the following problem.
I have a table created in the Hive catalog and I want to create a temporal
table over it.
As we cannot create temporal tables in SQL directly I have to define it in
the environment yaml file. Unfortunately it seems to be impossible to
reference a table only present in the catalog (not in the yaml) as catalogs
are loaded only after creating the temporal table (see
https://github.com/apache/flink/blob/master/flink-table/flink-sql-client/src/main/java/org/apache/flink/table/client/gateway/local/ExecutionContext.java#L622
)
I am wondering if it would make sense to set the catalogs before all else
or if that would cause some other problems.
What do you think?
Gyula
Re: Creating TemporalTable based on Catalog table in SQL Client
Posted by Bowen Li <bo...@gmail.com>.
you would need to reference the table with fully qualified name with
catalog and database
On Wed, Mar 4, 2020 at 02:17 Gyula Fóra <gy...@gmail.com> wrote:
> I guess it will only work now if you specify the catalog name too when
> referencing the table.
>
>
> On Wed, Mar 4, 2020 at 11:15 AM Gyula Fóra <gy...@gmail.com> wrote:
>
> > You are right but still if the default catalog is something else and
> > that's the one containing the table then it still wont work currently.
> >
> > Gyula
> >
> > On Wed, Mar 4, 2020 at 5:08 AM Bowen Li <bo...@gmail.com> wrote:
> >
> >> Hi Gyula,
> >>
> >> What line 622 (the link you shared) does is not registering catalogs,
> but
> >> setting an already registered catalog as the current one. As you can see
> >> from the method and its comment, catalogs are loaded first before any
> >> tables in yaml are registered, so you should be able to achieve what you
> >> described.
> >>
> >> Bowen
> >>
> >> On Tue, Mar 3, 2020 at 5:16 AM Gyula Fóra <gy...@gmail.com> wrote:
> >>
> >> > Hi all!
> >> >
> >> > I was testing the TemporalTable functionality in the SQL client while
> >> using
> >> > the Hive Catalog and I ran into the following problem.
> >> >
> >> > I have a table created in the Hive catalog and I want to create a
> >> temporal
> >> > table over it.
> >> >
> >> > As we cannot create temporal tables in SQL directly I have to define
> it
> >> in
> >> > the environment yaml file. Unfortunately it seems to be impossible to
> >> > reference a table only present in the catalog (not in the yaml) as
> >> catalogs
> >> > are loaded only after creating the temporal table (see
> >> >
> >> >
> >>
> https://github.com/apache/flink/blob/master/flink-table/flink-sql-client/src/main/java/org/apache/flink/table/client/gateway/local/ExecutionContext.java#L622
> >> > )
> >> >
> >> > I am wondering if it would make sense to set the catalogs before all
> >> else
> >> > or if that would cause some other problems.
> >> >
> >> > What do you think?
> >> > Gyula
> >> >
> >>
> >
>
Re: Creating TemporalTable based on Catalog table in SQL Client
Posted by Gyula Fóra <gy...@gmail.com>.
I guess it will only work now if you specify the catalog name too when
referencing the table.
On Wed, Mar 4, 2020 at 11:15 AM Gyula Fóra <gy...@gmail.com> wrote:
> You are right but still if the default catalog is something else and
> that's the one containing the table then it still wont work currently.
>
> Gyula
>
> On Wed, Mar 4, 2020 at 5:08 AM Bowen Li <bo...@gmail.com> wrote:
>
>> Hi Gyula,
>>
>> What line 622 (the link you shared) does is not registering catalogs, but
>> setting an already registered catalog as the current one. As you can see
>> from the method and its comment, catalogs are loaded first before any
>> tables in yaml are registered, so you should be able to achieve what you
>> described.
>>
>> Bowen
>>
>> On Tue, Mar 3, 2020 at 5:16 AM Gyula Fóra <gy...@gmail.com> wrote:
>>
>> > Hi all!
>> >
>> > I was testing the TemporalTable functionality in the SQL client while
>> using
>> > the Hive Catalog and I ran into the following problem.
>> >
>> > I have a table created in the Hive catalog and I want to create a
>> temporal
>> > table over it.
>> >
>> > As we cannot create temporal tables in SQL directly I have to define it
>> in
>> > the environment yaml file. Unfortunately it seems to be impossible to
>> > reference a table only present in the catalog (not in the yaml) as
>> catalogs
>> > are loaded only after creating the temporal table (see
>> >
>> >
>> https://github.com/apache/flink/blob/master/flink-table/flink-sql-client/src/main/java/org/apache/flink/table/client/gateway/local/ExecutionContext.java#L622
>> > )
>> >
>> > I am wondering if it would make sense to set the catalogs before all
>> else
>> > or if that would cause some other problems.
>> >
>> > What do you think?
>> > Gyula
>> >
>>
>
Re: Creating TemporalTable based on Catalog table in SQL Client
Posted by Gyula Fóra <gy...@gmail.com>.
You are right but still if the default catalog is something else and that's
the one containing the table then it still wont work currently.
Gyula
On Wed, Mar 4, 2020 at 5:08 AM Bowen Li <bo...@gmail.com> wrote:
> Hi Gyula,
>
> What line 622 (the link you shared) does is not registering catalogs, but
> setting an already registered catalog as the current one. As you can see
> from the method and its comment, catalogs are loaded first before any
> tables in yaml are registered, so you should be able to achieve what you
> described.
>
> Bowen
>
> On Tue, Mar 3, 2020 at 5:16 AM Gyula Fóra <gy...@gmail.com> wrote:
>
> > Hi all!
> >
> > I was testing the TemporalTable functionality in the SQL client while
> using
> > the Hive Catalog and I ran into the following problem.
> >
> > I have a table created in the Hive catalog and I want to create a
> temporal
> > table over it.
> >
> > As we cannot create temporal tables in SQL directly I have to define it
> in
> > the environment yaml file. Unfortunately it seems to be impossible to
> > reference a table only present in the catalog (not in the yaml) as
> catalogs
> > are loaded only after creating the temporal table (see
> >
> >
> https://github.com/apache/flink/blob/master/flink-table/flink-sql-client/src/main/java/org/apache/flink/table/client/gateway/local/ExecutionContext.java#L622
> > )
> >
> > I am wondering if it would make sense to set the catalogs before all else
> > or if that would cause some other problems.
> >
> > What do you think?
> > Gyula
> >
>
Re: Creating TemporalTable based on Catalog table in SQL Client
Posted by Bowen Li <bo...@gmail.com>.
Hi Gyula,
What line 622 (the link you shared) does is not registering catalogs, but
setting an already registered catalog as the current one. As you can see
from the method and its comment, catalogs are loaded first before any
tables in yaml are registered, so you should be able to achieve what you
described.
Bowen
On Tue, Mar 3, 2020 at 5:16 AM Gyula Fóra <gy...@gmail.com> wrote:
> Hi all!
>
> I was testing the TemporalTable functionality in the SQL client while using
> the Hive Catalog and I ran into the following problem.
>
> I have a table created in the Hive catalog and I want to create a temporal
> table over it.
>
> As we cannot create temporal tables in SQL directly I have to define it in
> the environment yaml file. Unfortunately it seems to be impossible to
> reference a table only present in the catalog (not in the yaml) as catalogs
> are loaded only after creating the temporal table (see
>
> https://github.com/apache/flink/blob/master/flink-table/flink-sql-client/src/main/java/org/apache/flink/table/client/gateway/local/ExecutionContext.java#L622
> )
>
> I am wondering if it would make sense to set the catalogs before all else
> or if that would cause some other problems.
>
> What do you think?
> Gyula
>