[jira] [Updated] (MINIFI-390) Logback archiving can cause issues with rollover when compression is enabled

["Aldrin Piri (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/MINIFI-390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Aldrin Piri updated MINIFI-390:
-------------------------------
    Description: 
At high rates of logging, the compression of archived log files configured in logback can cause an unbounded growth of file size despite totalSizeCap.

The issue can be seen in the following APP appender and triggered with a root-level of DEBUG.

{code}
    <appender name="APP_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
        <file>${org.apache.nifi.minifi.bootstrap.config.log.dir}/minifi-app.log</file>
        <rollingPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">
            <!--
              For daily rollover, use 'app_%d.log'.
              For hourly rollover, use 'app_%d{yyyy-MM-dd_HH}.log'.
              To GZIP rolled files, replace '.log' with '.log.gz'.
              To ZIP rolled files, replace '.log' with '.log.zip'.
            -->
            <fileNamePattern>${org.apache.nifi.minifi.bootstrap.config.log.dir}/minifi-app_%d{yyyy-MM-dd_HH}.%i.log</fileNamePattern>
            <!-- Keep 10 rolling periods worth of log files-->
            <maxHistory>1</maxHistory>
            <!-- Max size each log file will be-->
            <maxFileSize>10MB</maxFileSize>
            <!-- Provide a cap of 10 MB across all archive files -->
            <totalSizeCap>1000MB</totalSizeCap>
        </rollingPolicy>
        <immediateFlush>true</immediateFlush>
        <encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
            <pattern>%date %level [%thread] %logger{40} %msg%n</pattern>
        </encoder>
    </appender>
{code}

This seems to be related to https://jira.qos.ch/browse/LOGBACK-1162 which is currently in progress.  We should monitor this progression and incorporate the updated dependency when released.  The workaround for those environments experiencing this is to disable archiving (by removing the .gz from the filename pattern).  

> Logback archiving can cause issues with rollover when compression is enabled
> ----------------------------------------------------------------------------
>
>                 Key: MINIFI-390
>                 URL: https://issues.apache.org/jira/browse/MINIFI-390
>             Project: Apache NiFi MiNiFi
>          Issue Type: Bug
>          Components: Core Framework
>    Affects Versions: 0.2.0
>            Reporter: Aldrin Piri
>
> At high rates of logging, the compression of archived log files configured in logback can cause an unbounded growth of file size despite totalSizeCap.
> The issue can be seen in the following APP appender and triggered with a root-level of DEBUG.
> {code}
>     <appender name="APP_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
>         <file>${org.apache.nifi.minifi.bootstrap.config.log.dir}/minifi-app.log</file>
>         <rollingPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">
>             <!--
>               For daily rollover, use 'app_%d.log'.
>               For hourly rollover, use 'app_%d{yyyy-MM-dd_HH}.log'.
>               To GZIP rolled files, replace '.log' with '.log.gz'.
>               To ZIP rolled files, replace '.log' with '.log.zip'.
>             -->
>             <fileNamePattern>${org.apache.nifi.minifi.bootstrap.config.log.dir}/minifi-app_%d{yyyy-MM-dd_HH}.%i.log</fileNamePattern>
>             <!-- Keep 10 rolling periods worth of log files-->
>             <maxHistory>1</maxHistory>
>             <!-- Max size each log file will be-->
>             <maxFileSize>10MB</maxFileSize>
>             <!-- Provide a cap of 10 MB across all archive files -->
>             <totalSizeCap>1000MB</totalSizeCap>
>         </rollingPolicy>
>         <immediateFlush>true</immediateFlush>
>         <encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
>             <pattern>%date %level [%thread] %logger{40} %msg%n</pattern>
>         </encoder>
>     </appender>
> {code}
> This seems to be related to https://jira.qos.ch/browse/LOGBACK-1162 which is currently in progress.  We should monitor this progression and incorporate the updated dependency when released.  The workaround for those environments experiencing this is to disable archiving (by removing the .gz from the filename pattern).  



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)