You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@nifi.apache.org by Huagen peng <hu...@gmail.com> on 2016/06/05 01:27:06 UTC

Excessive logging

Hi,

I got excessive logging from my NiFi instance.  I suddenly see logging like the following going into nidi-bootstrap.log very fast, like 10g/hour, filling up my disk quickly. What is the cause of this? How to stop it going into the log?  I tried to look into the logback.xml file and even commented out the logger for org.apache.nifi.StdOut, this did not help at all. 

Thanks,

Huagen

2016-06-04 12:56:09,416 INFO [NiFi logging handler] org.apache.nifi.StdOut 12:56:08.696 [Timer-Driven Process Thread-4] DEBUG org.apache.nifi.engine.FlowEngine - A flow controller execution task 'java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@7753bd5e' has been cancelled.

2016-06-04 05:22:58,034 INFO [NiFi logging handler] org.apache.nifi.StdOut 05:22:58.034 [Timer-Driven Process Thread-3] DEBUG org.apache.nifi.engine.FlowEngine - A flow controller execution task 'java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@546d70b0' has been cancelled.

Re: Excessive logging

Posted by Huagen peng <hu...@gmail.com>.
Oleg,

Like I said, I made no change to the logback.xml file at all when the excessive logging happened; it is the default one.  I am able to stop the excessive logging by set the following logger from INFO to ERROR level

<logger name="org.apache.nifi" level="ERROR"/>

But this fix is not good enough as I do need to see the INFO entries of classes other than org.apache.nifi.StdOut.  The only thing I did before the excessive logging occurred was to modify the canvas, changing the data flow.  I guess I just have to find the cause myself, because others may not be able to reproduce the issue.

Thanks anyway.

Huagen

> 在 2016年6月5日，上午10:26，Oleg Zhurakousky <oz...@hortonworks.com> 写道：
> 
> I can't say without looking at the whole file. Could you post it?
> 
> Oleg 
> 
> On Jun 5, 2016, at 10:23, Huagen peng <huagen.peng@gmail.com <ma...@gmail.com>> wrote:
> 
>> When the excessive logging happened suddenly, I made no change to the logback.xml file.  And I look into the logback.xml file, the org.apache.nifi.StdOut logger is set at the INFO level.  This didn’t explain how the DEBUG log entries came about.  It didn’t help at all to comment out the org.apache.nifi.StdOut logger and set the default logger to ERROR level.
>> 
>> <root level="ERROR">
>>         <appender-ref ref="APP_FILE"/>
>>     </root>
>> 
>> Any other idea?
>> 
>> Huagen
>> 
>>> 在 2016年6月5日，上午9:11，Oleg Zhurakousky <ozhurakousky@hortonworks.com <ma...@hortonworks.com>> 写道：
>>> 
>>> Huagen, it appears you have DEBUG level logging enabled. 
>>> 
>>> Oleg
>>> 
>>> On Jun 4, 2016, at 21:27, Huagen peng <huagen.peng@gmail.com <ma...@gmail.com>> wrote:
>>> 
>>>> Hi,
>>>> 
>>>> I got excessive logging from my NiFi instance.  I suddenly see logging like the following going into nidi-bootstrap.log very fast, like 10g/hour, filling up my disk quickly. What is the cause of this? How to stop it going into the log?  I tried to look into the logback.xml file and even commented out the logger for org.apache.nifi.StdOut, this did not help at all. 
>>>> 
>>>> Thanks,
>>>> 
>>>> Huagen
>>>> 
>>>> 2016-06-04 12:56:09,416 INFO [NiFi logging handler] org.apache.nifi.StdOut 12:56:08.696 [Timer-Driven Process Thread-4] DEBUG org.apache.nifi.engine.FlowEngine - A flow controller execution task 'java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@7753bd5e' has been cancelled.
>>>> 
>>>> 2016-06-04 05:22:58,034 INFO [NiFi logging handler] org.apache.nifi.StdOut 05:22:58.034 [Timer-Driven Process Thread-3] DEBUG org.apache.nifi.engine.FlowEngine - A flow controller execution task 'java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@546d70b0' has been cancelled.    
>>>> 
>>>> 
>>>> 
>>>> 
>>

Re: Excessive logging

Posted by Oleg Zhurakousky <oz...@hortonworks.com>.
I can't say without looking at the whole file. Could you post it?

Oleg

On Jun 5, 2016, at 10:23, Huagen peng <hu...@gmail.com>> wrote:

When the excessive logging happened suddenly, I made no change to the logback.xml file.  And I look into the logback.xml file, the org.apache.nifi.StdOut logger is set at the INFO level.  This didn't explain how the DEBUG log entries came about.  It didn't help at all to comment out the org.apache.nifi.StdOut logger and set the default logger to ERROR level.

<root level="ERROR">
        <appender-ref ref="APP_FILE"/>
    </root>

Any other idea?

Huagen

? 2016?6?5????9:11?Oleg Zhurakousky <oz...@hortonworks.com>> ???

Huagen, it appears you have DEBUG level logging enabled.

Oleg

On Jun 4, 2016, at 21:27, Huagen peng <hu...@gmail.com>> wrote:

Hi,

I got excessive logging from my NiFi instance.  I suddenly see logging like the following going into nidi-bootstrap.log very fast, like 10g/hour, filling up my disk quickly. What is the cause of this? How to stop it going into the log?  I tried to look into the logback.xml file and even commented out the logger for org.apache.nifi.StdOut, this did not help at all.

Thanks,

Huagen

2016-06-04 12:56:09,416 INFO [NiFi logging handler] org.apache.nifi.StdOut 12:56:08.696 [Timer-Driven Process Thread-4] DEBUG org.apache.nifi.engine.FlowEngine - A flow controller execution task 'java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@7753bd5e' has been cancelled.

2016-06-04 05:22:58,034 INFO [NiFi logging handler] org.apache.nifi.StdOut 05:22:58.034 [Timer-Driven Process Thread-3] DEBUG org.apache.nifi.engine.FlowEngine - A flow controller execution task 'java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@546d70b0' has been cancelled.

Re: Excessive logging

Posted by Huagen peng <hu...@gmail.com>.
When the excessive logging happened suddenly, I made no change to the logback.xml file.  And I look into the logback.xml file, the org.apache.nifi.StdOut logger is set at the INFO level.  This didn’t explain how the DEBUG log entries came about.  It didn’t help at all to comment out the org.apache.nifi.StdOut logger and set the default logger to ERROR level.

<root level="ERROR">
        <appender-ref ref="APP_FILE"/>
    </root>

Any other idea?

Huagen

> 在 2016年6月5日，上午9:11，Oleg Zhurakousky <oz...@hortonworks.com> 写道：
> 
> Huagen, it appears you have DEBUG level logging enabled. 
> 
> Oleg
> 
> On Jun 4, 2016, at 21:27, Huagen peng <huagen.peng@gmail.com <ma...@gmail.com>> wrote:
> 
>> Hi,
>> 
>> I got excessive logging from my NiFi instance.  I suddenly see logging like the following going into nidi-bootstrap.log very fast, like 10g/hour, filling up my disk quickly. What is the cause of this? How to stop it going into the log?  I tried to look into the logback.xml file and even commented out the logger for org.apache.nifi.StdOut, this did not help at all. 
>> 
>> Thanks,
>> 
>> Huagen
>> 
>> 2016-06-04 12:56:09,416 INFO [NiFi logging handler] org.apache.nifi.StdOut 12:56:08.696 [Timer-Driven Process Thread-4] DEBUG org.apache.nifi.engine.FlowEngine - A flow controller execution task 'java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@7753bd5e' has been cancelled.
>> 
>> 2016-06-04 05:22:58,034 INFO [NiFi logging handler] org.apache.nifi.StdOut 05:22:58.034 [Timer-Driven Process Thread-3] DEBUG org.apache.nifi.engine.FlowEngine - A flow controller execution task 'java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@546d70b0' has been cancelled.    
>> 
>> 
>> 
>>

Re: Excessive logging

Posted by Oleg Zhurakousky <oz...@hortonworks.com>.
Huagen, it appears you have DEBUG level logging enabled.

Oleg

On Jun 4, 2016, at 21:27, Huagen peng <hu...@gmail.com>> wrote:

Hi,

I got excessive logging from my NiFi instance.  I suddenly see logging like the following going into nidi-bootstrap.log very fast, like 10g/hour, filling up my disk quickly. What is the cause of this? How to stop it going into the log?  I tried to look into the logback.xml file and even commented out the logger for org.apache.nifi.StdOut, this did not help at all.

Thanks,

Huagen

2016-06-04 12:56:09,416 INFO [NiFi logging handler] org.apache.nifi.StdOut 12:56:08.696 [Timer-Driven Process Thread-4] DEBUG org.apache.nifi.engine.FlowEngine - A flow controller execution task 'java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@7753bd5e' has been cancelled.

2016-06-04 05:22:58,034 INFO [NiFi logging handler] org.apache.nifi.StdOut 05:22:58.034 [Timer-Driven Process Thread-3] DEBUG org.apache.nifi.engine.FlowEngine - A flow controller execution task 'java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@546d70b0' has been cancelled.