Log4j configuration on YARN

[Nick Dimiduk <nd...@gmail.com>]

Can anyone tell me where I must place my application-specific
log4j.properties to have them honored when running on a YARN cluster? In my
application jar doesn't work. In the log4j files under flink/conf doesn't
work.

My goal is to set the log level for 'com.mycompany' classes used in my
flink application to DEBUG.

Thanks,
Nick

Re: Log4j configuration on YARN

[Robert Metzger <rm...@apache.org>]

Hi Nick,

the name of the "log4j-yarn-session.properties" file might be a bit
misleading. The file is just used for the YARN session client, running
locally.
The Job- and TaskManager are going to use the log4j.properties on the
cluster.

On Fri, Mar 11, 2016 at 7:20 PM, Ufuk Celebi <uc...@apache.org> wrote:

> Hey Nick!
>
> I just checked and the conf/log4j.properties file is copied and is
> given as an argument to the JVM.
>
> You should see the following:
> - client logs that the conf/log4j.properties file is copied
> - JobManager logs show log4j.configuration being passed to the JVM.
>
> Can you confirm that these shows up? If yes, but you still don't get
> the expected logging, I would check via -Dlog4j.debug what is
> configured (prints to stdout I think). Does this help?
>
> – Ufuk
>
>
> On Fri, Mar 11, 2016 at 6:02 PM, Nick Dimiduk <nd...@gmail.com> wrote:
> > Can anyone tell me where I must place my application-specific
> > log4j.properties to have them honored when running on a YARN cluster? In
> my
> > application jar doesn't work. In the log4j files under flink/conf doesn't
> > work.
> >
> > My goal is to set the log level for 'com.mycompany' classes used in my
> flink
> > application to DEBUG.
> >
> > Thanks,
> > Nick
> >
>

Re: Log4j configuration on YARN

[Ufuk Celebi <uc...@apache.org>]

Hey Nick!

I just checked and the conf/log4j.properties file is copied and is
given as an argument to the JVM.

You should see the following:
- client logs that the conf/log4j.properties file is copied
- JobManager logs show log4j.configuration being passed to the JVM.

Can you confirm that these shows up? If yes, but you still don't get
the expected logging, I would check via -Dlog4j.debug what is
configured (prints to stdout I think). Does this help?

– Ufuk


On Fri, Mar 11, 2016 at 6:02 PM, Nick Dimiduk <nd...@gmail.com> wrote:
> Can anyone tell me where I must place my application-specific
> log4j.properties to have them honored when running on a YARN cluster? In my
> application jar doesn't work. In the log4j files under flink/conf doesn't
> work.
>
> My goal is to set the log level for 'com.mycompany' classes used in my flink
> application to DEBUG.
>
> Thanks,
> Nick
>