Why is totalSizeCap not a default parameter in the logback.xml file

["Jens M. Kofoed" <jm...@gmail.com>]

Hi

We have unfortunately had an incident where NiFi doing a weekend is filling
up the disk with logs because a process is failing and produce hundreds of
error messages per seconds.
We have changed the rollingPolicy to use daily rollover with a maxHistory
of 30 and maxFileSize at 100MB.
When the daily logfile is going to be bigger than maxFileSize the
rollingpolicy will create incrementing "subfiles" as .# for that day. But
the maxHistory does not count for subfiles.
So with a process producing hundreds of error messages per seconds you can
have a situation where you will end up with thousands of subfiles for each
day filling up the disk.

There are an attribute called "totalSizeCap" which has been asked for in
JIRA:
https://issues.apache.org/jira/browse/NIFI-2203
https://issues.apache.org/jira/browse/NIFI-4315

This attribute is already working, but is by default not included in the
logback.xml file nor in the new stateless-logback.xml file.

Example:
    <appender name="APP_FILE"
class="ch.qos.logback.core.rolling.RollingFileAppender">

<file>${org.apache.nifi.bootstrap.config.log.dir}/nifi-stateless.log</file>
        <rollingPolicy
class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">
            <!--
              For daily rollover, use 'app_%d.log'.
              For hourly rollover, use 'app_%d{yyyy-MM-dd_HH}.log'.
              To GZIP rolled files, replace '.log' with '.log.gz'.
              To ZIP rolled files, replace '.log' with '.log.zip'.
            -->

<fileNamePattern>${org.apache.nifi.bootstrap.config.log.dir}/nifi-stateless_%d{yyyy-MM-dd_HH}.%i.log</fileNamePattern>
            <maxFileSize>100MB</maxFileSize>
            <!-- keep 30 log files worth of history -->
            <maxHistory>30</maxHistory>
        </rollingPolicy>

*        <totalSizeCap>10GB</totalSizeCap>*
<immediateFlush>true</immediateFlush>
        <encoder
class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
            <pattern>%date %level [%thread] %logger{40} %msg%n</pattern>
        </encoder>
    </appender>

Please add this as a new default parameter

kind regards
Jens M. Kofoed