You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Krzysztof Chmielewski <kr...@gmail.com> on 2021/11/09 14:58:32 UTC
Access to GlobalJobParameters From DynamicTableSourceFactory
Hi,
Is there a way to access GlobalJobParameters registered as
env.getConfig().setGlobalJobParameters(parameters);
from DynamicTableSourceFactory implementation?
To be more specific
from DynamicTableSourceFactory::createDynamicTableSource method.
The Context parameter of createDynamicTableSource has access only
to ReadableConfig object which does not have GlobalParameters.
Cheers,
Krzysztof Chmielewski
Re: Access to GlobalJobParameters From DynamicTableSourceFactory
Posted by Krzysztof Chmielewski <kr...@gmail.com>.
Well,
the use case is rather simple, just pass a JVM parameter to the Table
Source's underlying code and the property should not be exposed to Table
Definition.
Like something that end user should not change but its driven via
application properties. For example you are running SQL that should use a
some kind of "mock" implementation on test env.
Its just an example
Ideally you would not want to change the code of the SQL or table
definition or even expose such parameter to the end user.
Best,
Krzysztof Chmielewski
śr., 10 lis 2021 o 02:24 Caizhi Weng <ts...@gmail.com> napisał(a):
> Hi!
>
> You can't get GLOBAL_JOB_PARAMETERS
> from DynamicTableSourceFactory#createDynamicTableSource as far as I know,
> because configuration in that context is from table config, which only
> contains configuration on table API level.
>
> Could you tell us more about your use case? There are many other ways to
> configure a table source for example setting the table configs or passing
> table options (the options after the WITH clause in DDL).
>
> Krzysztof Chmielewski <kr...@gmail.com> 于2021年11月10日周三
> 上午6:10写道：
>
>> Hi,
>> Well no, because ReadableContext does not have getConfiguration method
>> ;)
>>
>> So like I wrote, I would like to get the GLOBAL_JOB_PARAMETERS from
>> DynamicTableSourceFactory interface implementations.
>> I'm talking about DynamicTableSourceFactory:: createDynamicTableSource
>> method. It seems that I have access only to Table definition fields.
>>
>> Regards,
>> Krzysztof Chmielewski
>>
>> wt., 9 lis 2021 o 19:02 Francesco Guardiani <fr...@ververica.com>
>> napisał(a):
>>
>>> Have you tried this?
>>>
>>>
>>> context.getConfiguration().get(org.apache.flink.configuration.PipelineOptions.GLOBAL_JOB_PARAMETERS)
>>>
>>>
>>>
>>> On Tue, Nov 9, 2021 at 3:59 PM Krzysztof Chmielewski <
>>> krzysiek.chmielewski@gmail.com> wrote:
>>>
>>>> Hi,
>>>> Is there a way to access GlobalJobParameters registered as
>>>> env.getConfig().setGlobalJobParameters(parameters);
>>>>
>>>> from DynamicTableSourceFactory implementation?
>>>> To be more specific
>>>> from DynamicTableSourceFactory::createDynamicTableSource method.
>>>>
>>>> The Context parameter of createDynamicTableSource has access only
>>>> to ReadableConfig object which does not have GlobalParameters.
>>>>
>>>> Cheers,
>>>> Krzysztof Chmielewski
>>>>
>>>
Re: Access to GlobalJobParameters From DynamicTableSourceFactory
Posted by Caizhi Weng <ts...@gmail.com>.
Hi!
You can't get GLOBAL_JOB_PARAMETERS
from DynamicTableSourceFactory#createDynamicTableSource as far as I know,
because configuration in that context is from table config, which only
contains configuration on table API level.
Could you tell us more about your use case? There are many other ways to
configure a table source for example setting the table configs or passing
table options (the options after the WITH clause in DDL).
Krzysztof Chmielewski <kr...@gmail.com> 于2021年11月10日周三
上午6:10写道：
> Hi,
> Well no, because ReadableContext does not have getConfiguration method ;)
>
> So like I wrote, I would like to get the GLOBAL_JOB_PARAMETERS from
> DynamicTableSourceFactory interface implementations.
> I'm talking about DynamicTableSourceFactory:: createDynamicTableSource
> method. It seems that I have access only to Table definition fields.
>
> Regards,
> Krzysztof Chmielewski
>
> wt., 9 lis 2021 o 19:02 Francesco Guardiani <fr...@ververica.com>
> napisał(a):
>
>> Have you tried this?
>>
>>
>> context.getConfiguration().get(org.apache.flink.configuration.PipelineOptions.GLOBAL_JOB_PARAMETERS)
>>
>>
>>
>> On Tue, Nov 9, 2021 at 3:59 PM Krzysztof Chmielewski <
>> krzysiek.chmielewski@gmail.com> wrote:
>>
>>> Hi,
>>> Is there a way to access GlobalJobParameters registered as
>>> env.getConfig().setGlobalJobParameters(parameters);
>>>
>>> from DynamicTableSourceFactory implementation?
>>> To be more specific
>>> from DynamicTableSourceFactory::createDynamicTableSource method.
>>>
>>> The Context parameter of createDynamicTableSource has access only
>>> to ReadableConfig object which does not have GlobalParameters.
>>>
>>> Cheers,
>>> Krzysztof Chmielewski
>>>
>>
Re: Access to GlobalJobParameters From DynamicTableSourceFactory
Posted by Krzysztof Chmielewski <kr...@gmail.com>.
Hi,
Well no, because ReadableContext does not have getConfiguration method ;)
So like I wrote, I would like to get the GLOBAL_JOB_PARAMETERS from
DynamicTableSourceFactory interface implementations.
I'm talking about DynamicTableSourceFactory:: createDynamicTableSource
method. It seems that I have access only to Table definition fields.
Regards,
Krzysztof Chmielewski
wt., 9 lis 2021 o 19:02 Francesco Guardiani <fr...@ververica.com>
napisał(a):
> Have you tried this?
>
>
> context.getConfiguration().get(org.apache.flink.configuration.PipelineOptions.GLOBAL_JOB_PARAMETERS)
>
>
>
> On Tue, Nov 9, 2021 at 3:59 PM Krzysztof Chmielewski <
> krzysiek.chmielewski@gmail.com> wrote:
>
>> Hi,
>> Is there a way to access GlobalJobParameters registered as
>> env.getConfig().setGlobalJobParameters(parameters);
>>
>> from DynamicTableSourceFactory implementation?
>> To be more specific
>> from DynamicTableSourceFactory::createDynamicTableSource method.
>>
>> The Context parameter of createDynamicTableSource has access only
>> to ReadableConfig object which does not have GlobalParameters.
>>
>> Cheers,
>> Krzysztof Chmielewski
>>
>
Re: Access to GlobalJobParameters From DynamicTableSourceFactory
Posted by Francesco Guardiani <fr...@ververica.com>.
Have you tried this?
context.getConfiguration().get(org.apache.flink.configuration.PipelineOptions.GLOBAL_JOB_PARAMETERS)
On Tue, Nov 9, 2021 at 3:59 PM Krzysztof Chmielewski <
krzysiek.chmielewski@gmail.com> wrote:
> Hi,
> Is there a way to access GlobalJobParameters registered as
> env.getConfig().setGlobalJobParameters(parameters);
>
> from DynamicTableSourceFactory implementation?
> To be more specific
> from DynamicTableSourceFactory::createDynamicTableSource method.
>
> The Context parameter of createDynamicTableSource has access only
> to ReadableConfig object which does not have GlobalParameters.
>
> Cheers,
> Krzysztof Chmielewski
>