[jira] [Resolved] (LOG4J2-354) log4j2 + FastFileAppender + Tomcat logrotate problem

["Remko Popma (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/LOG4J2-354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Remko Popma resolved LOG4J2-354.
--------------------------------

    Resolution: Won't Fix

Resolving this issue as "Won't Fix".

The issue is that the RandomAccessFileAppender is not aware that an external application (logrotate) moved the file, and it will continue to write bytes to its current file position. I don't believe this is a Log4J problem; RandomAccessFileAppender is not designed to work with external applications like logrotate. FileAppender may work well with logrotate, but that is more by accident than by design. :-) 

If I knew how to fix this I would, but currently I don't have a clue...
Please feel free to re-open this issue if anyone has ideas on how to fix this (preferably without impacting performance).

Log4J provides its own solution for rolling over log files with RollingFileAppender and RollingRandomAccessFileAppender, and this is the supported solution. I would recommend to anyone to use the supported solution instead of logrotate.

> log4j2 + FastFileAppender + Tomcat logrotate problem
> ----------------------------------------------------
>
>                 Key: LOG4J2-354
>                 URL: https://issues.apache.org/jira/browse/LOG4J2-354
>             Project: Log4j 2
>          Issue Type: Bug
>    Affects Versions: 2.0-beta8
>         Environment: log4j2-beta8
> disruptor 3.1.1
> tomcat 7 with Java 1.6
> We use the SLF4J library as well.
>            Reporter: Remko Popma
>
> (from log4j-user mailing list)
> Kamil Mroczek wrote:
> Hello,
> We decided to try out log4j2-beta8 to see if we could improve our logging performance.  We were testing with the disruptor 3.1.1. library to make all our appenders async.
> {{-DLog4jContextSelector=org.apache.logging.log4j.core.async.AsyncLoggerContextSelector}}
> We are running tomcat 7 with Java 1.6.  We use the SLF4J library as well.
> The appender that we were using in this case was the Fast File Appender with a definition like:
> {code}
> <FastFile name="RequestLog" fileName="requests.log"
>           immediateFlush="false" append="true">
>   <PatternLayout>
>     <pattern>%m%n</pattern>
>   </PatternLayout>
> </FastFile>
> {code}
> And logger was..
> {code}
> <logger name="a.namespace" level="info" additivity="false">
>   <appender-ref ref="RequestLog"/>
> </logger>
> {code}
> So the system was designed to allow log4j to do the logging and then have logrotate rotate the log files from the host to an external destination.
> We rotate the logs every 5 minutes with these params (with LZO compression).
> compress
> compresscmd /bin/lzop
> compressoptions "-U"
> compressext .lzo
> What we were seeing was that after a log rotation happened the new file would start with a massive chunk of binary data at the start. Many times on the order of 100-200MB.  This would turn the logs from being on the order of 50-100MB to 200-350MB.
> My guess was that it had something to do with the byte buffer flushing mid-rotate since these chunks always come at the start of the file. But I also saw LOG4J2-295 (Fast(Rolling)FileAppender now correctly handles messages exceeding the buffer size. ) which was fixed in beta8 which my discredit that idea.
> We were able to fix the issue by using the regular FileAppender like this:
> {code}
> <File name="RequestLog" fileName="requests.log" immediateFlush="true"
> append="true" bufferedIO="false">
>       <PatternLayout>
>         <pattern>%m%n</pattern>
>       </PatternLayout>
> </File>
> {code}
> I can't remember for certain, but I am pretty sure that even if we had bufferedIO="true" on the FileAppender everything worked okay as well.
> We could reproduce it pretty consistently.  I wanted to post to the group to see if anyone has seen anything like this before.  Any ideas on what the issue could be?



--
This message was sent by Atlassian JIRA
(v6.1#6144)

---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-dev-unsubscribe@logging.apache.org
For additional commands, e-mail: log4j-dev-help@logging.apache.org