You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Vishwas Siravara <vs...@gmail.com> on 2019/07/17 17:23:28 UTC
Re: Providing external files to flink classpath
Does the -yt option work for standalone cluster without dedicated resource
manager ? So this property file is read by one of the dependencies inside
my application as a file, so I can't really use Parameter tool to parse the
config file.
Thanks,
Vishwas
On Fri, Jun 28, 2019 at 11:08 PM Yun Tang <my...@live.com> wrote:
> Hi Vishwas
>
>
> 1. You could use '-yt' to ship specified files to the class path,
> please refer to [1] for more details.
> 2. If the properties are only loaded on client side before executing
> the application, you could let your application to just read from local
> property data. Flink support to load properties within the
> ParameterTool [2].
>
> [1]
> https://ci.apache.org/projects/flink/flink-docs-release-1.8/ops/cli.html#usage
> [2]
> https://github.com/apache/flink/blob/f1721293b0701d584d42bd68671181e332d2ad04/flink-java/src/main/java/org/apache/flink/api/java/utils/ParameterTool.java#L120
>
> Best
> Yun Tang
>
> ------------------------------
> *From:* Vishwas Siravara <vs...@gmail.com>
> *Sent:* Saturday, June 29, 2019 0:43
> *To:* user
> *Subject:* Providing external files to flink classpath
>
> Hi ,
> I am trying to add external property files to the flink classpath for
> my application. These files are not a part of the fat jar. I put them
> under the lib folder but flink cant find them? How can I manage
> external property files that needs to be read by flink ?
>
> Thanks,
> Vishwas
>
Re: Providing external files to flink classpath
Posted by Maxim Parkachov <la...@gmail.com>.
Hi Vishwas,
took me some time to find out as well. If you have your properties file
under lib following will work:
val kafkaPropertiesInputStream =
getClass.getClassLoader.getResourceAsStream("lib/config/kafka.properties")
Hope this helps,
Maxim.
On Wed, Jul 17, 2019 at 7:23 PM Vishwas Siravara <vs...@gmail.com>
wrote:
> Does the -yt option work for standalone cluster without dedicated resource
> manager ? So this property file is read by one of the dependencies inside
> my application as a file, so I can't really use Parameter tool to parse the
> config file.
>
> Thanks,
> Vishwas
>
> On Fri, Jun 28, 2019 at 11:08 PM Yun Tang <my...@live.com> wrote:
>
>> Hi Vishwas
>>
>>
>> 1. You could use '-yt' to ship specified files to the class path,
>> please refer to [1] for more details.
>> 2. If the properties are only loaded on client side before executing
>> the application, you could let your application to just read from local
>> property data. Flink support to load properties within the
>> ParameterTool [2].
>>
>> [1]
>> https://ci.apache.org/projects/flink/flink-docs-release-1.8/ops/cli.html#usage
>> [2]
>> https://github.com/apache/flink/blob/f1721293b0701d584d42bd68671181e332d2ad04/flink-java/src/main/java/org/apache/flink/api/java/utils/ParameterTool.java#L120
>>
>> Best
>> Yun Tang
>>
>> ------------------------------
>> *From:* Vishwas Siravara <vs...@gmail.com>
>> *Sent:* Saturday, June 29, 2019 0:43
>> *To:* user
>> *Subject:* Providing external files to flink classpath
>>
>> Hi ,
>> I am trying to add external property files to the flink classpath for
>> my application. These files are not a part of the fat jar. I put them
>> under the lib folder but flink cant find them? How can I manage
>> external property files that needs to be read by flink ?
>>
>> Thanks,
>> Vishwas
>>
>