You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Gwenhael Pasquiers <gw...@ericsson.com> on 2015/12/21 15:34:11 UTC
Configure log4j with XML files
Hi everybody,
Could it be possible to have a way to configure log4j with xml files ?
I've looked into the code and it looks like the properties files names are hardcoded. However we have the need to use xml :
- We log everything into ELK (Elasticsearch / Logstash / Kibana) using SocketAppender
- Socket appender is synchronous by default and slow whole app if anything goes wrong with the ELK
- We usually add an AsyncAppender on top of the SocketAppender, but this sort of configuration is only possible using an XML config file...
We've already ran into the issue. Everything was almost paused because the ELK was overloaded and extremely slow.
B.R.
Gwenhaël PASQUIERS
Re: Configure log4j with XML files
Posted by Robert Metzger <rm...@apache.org>.
as an additional note: Flink is sending all files in the /lib folder to all
YARN containers. So you could place the XML file in "/lib" and override the
properties.
I think you need to delete the log4j properties from the conf/ directory,
then at least on YARN, we'll not set the -Dlog4j.configuration property
On Mon, Dec 21, 2015 at 3:58 PM, Till Rohrmann <tr...@apache.org> wrote:
> Hi Gwenhaël,
>
> as far as I know, there is no direct way to do so. You can either adapt
> the flink-daemon.sh script in line 68 to use a different configuration or
> you can test whether the dynamic property
> -Dlog4j.configurationFile:CONFIG_FILE overrides the -Dlog4j.confguration
> property. You can set the dynamic property using Flink’s env.java.opts
> configuration parameter.
>
> Cheers,
> Till
>
>
> On Mon, Dec 21, 2015 at 3:34 PM, Gwenhael Pasquiers <
> gwenhael.pasquiers@ericsson.com> wrote:
>
>> Hi everybody,
>>
>>
>>
>> Could it be possible to have a way to configure log4j with xml files ?
>>
>>
>>
>> I’ve looked into the code and it looks like the properties files names
>> are hardcoded. However we have the need to use xml :
>>
>> - We log everything into ELK (Elasticsearch / Logstash /
>> Kibana) using SocketAppender
>>
>> - Socket appender is synchronous by default and slow whole app
>> if anything goes wrong with the ELK
>>
>> - We usually add an AsyncAppender on top of the SocketAppender,
>> but this sort of configuration is only possible using an XML config file…
>>
>>
>>
>> We’ve already ran into the issue. Everything was almost paused because
>> the ELK was overloaded and extremely slow.
>>
>>
>>
>> B.R.
>>
>>
>>
>> Gwenhaël PASQUIERS
>>
>
>
Re: Configure log4j with XML files
Posted by Till Rohrmann <tr...@apache.org>.
Hi Gwenhaël,
as far as I know, there is no direct way to do so. You can either adapt the
flink-daemon.sh script in line 68 to use a different configuration or you
can test whether the dynamic property -Dlog4j.configurationFile:CONFIG_FILE
overrides the -Dlog4j.confguration property. You can set the dynamic
property using Flink’s env.java.opts configuration parameter.
Cheers,
Till
On Mon, Dec 21, 2015 at 3:34 PM, Gwenhael Pasquiers <
gwenhael.pasquiers@ericsson.com> wrote:
> Hi everybody,
>
>
>
> Could it be possible to have a way to configure log4j with xml files ?
>
>
>
> I’ve looked into the code and it looks like the properties files names are
> hardcoded. However we have the need to use xml :
>
> - We log everything into ELK (Elasticsearch / Logstash / Kibana)
> using SocketAppender
>
> - Socket appender is synchronous by default and slow whole app
> if anything goes wrong with the ELK
>
> - We usually add an AsyncAppender on top of the SocketAppender,
> but this sort of configuration is only possible using an XML config file…
>
>
>
> We’ve already ran into the issue. Everything was almost paused because the
> ELK was overloaded and extremely slow.
>
>
>
> B.R.
>
>
>
> Gwenhaël PASQUIERS
>