Dynamically change logging levels for loggers

[Kunal Khatua <kk...@mapr.com>]

I figured this is a rarely modified piece of code but most frequently used across all components. Hoping that someone who might have worked on logging can share some insight from their experience in general, if not within Drill.

I was wondering if changes to Drill's logback.xml can be picked up dynamically.

i.e. without restarting the Drillbit, change the logging level of specific classes within the Drillbit.

I ask this because sometimes, a Drillbit needs to go through a warmup phase where the JVM optimizes the functions frequently in use. Changing the logging from something like an INFO to a DEBUG level would then allow me to correctly capture specific log messages without having to lose all those optimizations due to a restart (for the DEBUG to take effect).

Is it something worth having ?

~ Kunal

Re: Dynamically change logging levels for loggers

[Paul Rogers <pr...@mapr.com>]

See the “LogFixture” class for how I got this to work (for tests) using LogBack. If we combine the LogBack details, with the prior Apex framework, we might have most of what we need.

BTW: another huge help would be a way to “poke” Drill and have it dump the state of some of its internal data structures, such as the memory allocator, the list of running fragments, etc. We’ve seen cases where, after Drill runs for a long time, “something bad” happens. But, it is hard to see that internal state to figure out what’s what.

Thanks,

- Paul

> On Aug 25, 2017, at 11:22 PM, Vlad Rozov <vr...@apache.org> wrote:
> 
> +1. Even though it was done for log4j in Apache Apex, I am pretty sure that the same can be done for logback. The only thing to consider is that all such functionality is specific to a logging provider in use. I am quite familiar with how it was done in Apex and can help if necessary.
> 
> Thank you,
> 
> Vlad
> 
> On 8/25/17 14:22, Timothy Farkas wrote:
>> +1 for exploring adding this feature. We had a feature to dynamically change log levels at runtime through the rest API in Apache Apex and it was very helpful with debugging things.
>> 
>> ________________________________
>> From: Paul Rogers <pr...@mapr.com>
>> Sent: Friday, August 25, 2017 11:01:29 AM
>> To: dev@drill.apache.org
>> Subject: Re: Dynamically change logging levels for loggers
>> 
>> Hi Kunal,
>> 
>> Don’t know about rereading the config file, but I have had luck in the unit test framework with adjusting log levels programmatically. (Tests turn on interesting log levels for the duration of a single tests.) We might be able to use that capability (provided by Logback) to make adjustments at run time.
>> 
>> - Paul
>> 
>>> On Aug 25, 2017, at 10:55 AM, Kunal Khatua <kk...@mapr.com> wrote:
>>> 
>>> I figured this is a rarely modified piece of code but most frequently used across all components. Hoping that someone who might have worked on logging can share some insight from their experience in general, if not within Drill.
>>> 
>>> I was wondering if changes to Drill's logback.xml can be picked up dynamically.
>>> 
>>> i.e. without restarting the Drillbit, change the logging level of specific classes within the Drillbit.
>>> 
>>> I ask this because sometimes, a Drillbit needs to go through a warmup phase where the JVM optimizes the functions frequently in use. Changing the logging from something like an INFO to a DEBUG level would then allow me to correctly capture specific log messages without having to lose all those optimizations due to a restart (for the DEBUG to take effect).
>>> 
>>> Is it something worth having ?
>>> 
>>> ~ Kunal
>> 
> 
> 
> Thank you,
> 
> Vlad

Re: Dynamically change logging levels for loggers

[Vlad Rozov <vr...@apache.org>]

+1. Even though it was done for log4j in Apache Apex, I am pretty sure 
that the same can be done for logback. The only thing to consider is 
that all such functionality is specific to a logging provider in use. I 
am quite familiar with how it was done in Apex and can help if necessary.

Thank you,

Vlad

On 8/25/17 14:22, Timothy Farkas wrote:
> +1 for exploring adding this feature. We had a feature to dynamically change log levels at runtime through the rest API in Apache Apex and it was very helpful with debugging things.
>
> ________________________________
> From: Paul Rogers <pr...@mapr.com>
> Sent: Friday, August 25, 2017 11:01:29 AM
> To: dev@drill.apache.org
> Subject: Re: Dynamically change logging levels for loggers
>
> Hi Kunal,
>
> Don’t know about rereading the config file, but I have had luck in the unit test framework with adjusting log levels programmatically. (Tests turn on interesting log levels for the duration of a single tests.) We might be able to use that capability (provided by Logback) to make adjustments at run time.
>
> - Paul
>
>> On Aug 25, 2017, at 10:55 AM, Kunal Khatua <kk...@mapr.com> wrote:
>>
>> I figured this is a rarely modified piece of code but most frequently used across all components. Hoping that someone who might have worked on logging can share some insight from their experience in general, if not within Drill.
>>
>> I was wondering if changes to Drill's logback.xml can be picked up dynamically.
>>
>> i.e. without restarting the Drillbit, change the logging level of specific classes within the Drillbit.
>>
>> I ask this because sometimes, a Drillbit needs to go through a warmup phase where the JVM optimizes the functions frequently in use. Changing the logging from something like an INFO to a DEBUG level would then allow me to correctly capture specific log messages without having to lose all those optimizations due to a restart (for the DEBUG to take effect).
>>
>> Is it something worth having ?
>>
>> ~ Kunal
>


Thank you,

Vlad

Re: Dynamically change logging levels for loggers

[Timothy Farkas <tf...@mapr.com>]

+1 for exploring adding this feature. We had a feature to dynamically change log levels at runtime through the rest API in Apache Apex and it was very helpful with debugging things.

________________________________
From: Paul Rogers <pr...@mapr.com>
Sent: Friday, August 25, 2017 11:01:29 AM
To: dev@drill.apache.org
Subject: Re: Dynamically change logging levels for loggers

Hi Kunal,

Don’t know about rereading the config file, but I have had luck in the unit test framework with adjusting log levels programmatically. (Tests turn on interesting log levels for the duration of a single tests.) We might be able to use that capability (provided by Logback) to make adjustments at run time.

- Paul

> On Aug 25, 2017, at 10:55 AM, Kunal Khatua <kk...@mapr.com> wrote:
>
> I figured this is a rarely modified piece of code but most frequently used across all components. Hoping that someone who might have worked on logging can share some insight from their experience in general, if not within Drill.
>
> I was wondering if changes to Drill's logback.xml can be picked up dynamically.
>
> i.e. without restarting the Drillbit, change the logging level of specific classes within the Drillbit.
>
> I ask this because sometimes, a Drillbit needs to go through a warmup phase where the JVM optimizes the functions frequently in use. Changing the logging from something like an INFO to a DEBUG level would then allow me to correctly capture specific log messages without having to lose all those optimizations due to a restart (for the DEBUG to take effect).
>
> Is it something worth having ?
>
> ~ Kunal

Re: Dynamically change logging levels for loggers

[Paul Rogers <pr...@mapr.com>]

Hi Kunal,

Don’t know about rereading the config file, but I have had luck in the unit test framework with adjusting log levels programmatically. (Tests turn on interesting log levels for the duration of a single tests.) We might be able to use that capability (provided by Logback) to make adjustments at run time.

- Paul

> On Aug 25, 2017, at 10:55 AM, Kunal Khatua <kk...@mapr.com> wrote:
> 
> I figured this is a rarely modified piece of code but most frequently used across all components. Hoping that someone who might have worked on logging can share some insight from their experience in general, if not within Drill.
> 
> I was wondering if changes to Drill's logback.xml can be picked up dynamically.
> 
> i.e. without restarting the Drillbit, change the logging level of specific classes within the Drillbit.
> 
> I ask this because sometimes, a Drillbit needs to go through a warmup phase where the JVM optimizes the functions frequently in use. Changing the logging from something like an INFO to a DEBUG level would then allow me to correctly capture specific log messages without having to lose all those optimizations due to a restart (for the DEBUG to take effect).
> 
> Is it something worth having ?
> 
> ~ Kunal