You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Hema Bhatia <He...@apollo.edu> on 2015/09/25 20:04:26 UTC
log clean up
Is there a way to delete kafka server, controller and state-change logs. They just keep growing over time and not purged.
-Hema
RE: log clean up
Posted by Karthikeyan Annamalai <ka...@owler.com>.
If its a daily rolling appender then next day when you get an another log
data will be rolled.
On Sep 26, 2015 3:20 AM, "Hema Bhatia" <He...@apollo.edu> wrote:
> Thanks Gwen!
>
> I made the changes and restarted kafka nodes. Looks like all log files are
> still present. Does it take some time to kick in the changes?
>
> Here is the sample of changes:
>
> log4j.appender.stateChangeAppender=org.apache.log4j.DailyRollingFileAppender
> log4j.appender.stateChangeAppender.DatePattern='.'yyyy-MM-dd-HH
> log4j.appender.stateChangeAppender.File=${kafka.logs.dir}/state-change.log
> log4j.appender.stateChangeAppender.layout=org.apache.log4j.PatternLayout
> log4j.appender.stateChangeAppender.layout.ConversionPattern=[%d] %p %m
> (%c)%n
> log4j.appender.stateChangeAppender.MaxFileSize=100KB
> log4j.appender.stateChangeAppender.MaxBackupIndex=5
> .
> .
> log4j.appender.kafkaAppender.MaxFileSize=100KB
> log4j.appender.kafkaAppender.MaxBackupIndex=5
> .
> .
> log4j.appender.controllerAppender.MaxFileSize=100KB
> log4j.appender.controllerAppender.MaxBackupIndex=5
>
>
> -----Original Message-----
> From: Gwen Shapira [mailto:gwen@confluent.io]
> Sent: Friday, September 25, 2015 1:13 PM
> To: users@kafka.apache.org
> Subject: Re: log clean up
>
> Absolutely.
>
> You can go into config/log4j.properties and configure the appenders to
> roll the logs.
>
> For example:
>
>
> log4j.appender.stateChangeAppender=org.apache.log4j.DailyRollingFileAppender
> log4j.appender.stateChangeAppender.DatePattern='.'yyyy-MM-dd-HH
> log4j.appender.stateChangeAppender.File=${kafka.logs.dir}/state-change.log
> log4j.appender.stateChangeAppender.layout=org.apache.log4j.PatternLayout
> log4j.appender.stateChangeAppender.layout.ConversionPattern=[%d] %p %m
> (%c)%n
>
> log4j.appender.stateChangeAppender.MaxFileSize=100KB
> log4j.appender.stateChangeAppender.MaxBackupIndex=5
>
> Adding the last two lines will make sure you have 5 state change log files
> each 100KB.
>
> Gwen
>
> On Fri, Sep 25, 2015 at 11:04 AM, Hema Bhatia <He...@apollo.edu>
> wrote:
>
> > Is there a way to delete kafka server, controller and state-change logs.
> > They just keep growing over time and not purged.
> >
> > -Hema
> >
> >
> >
>
Re: log clean up
Posted by Tulio Ballari <tu...@gmail.com>.
Hi Kafka Team!
Just today I was checking this in a cluster, and I was wondering...
As I search in the log4j docs, DailyRollingFileAppender doesn't support
MaxFileSize or MaxBackupIndex, by default only RollingFileAppender supports
it, but there is a patch out there with this changes
http://wiki.apache.org/logging-log4j/DailyRollingFileAppender
Did you include this patch in the distribution with kafka?, or it was just
a typo and you meant to use RollingFileAppender?
Thank you!
On Fri, Sep 25, 2015 at 6:48 PM, Hema Bhatia <He...@apollo.edu> wrote:
> Thanks Gwen!
>
> I made the changes and restarted kafka nodes. Looks like all log files are
> still present. Does it take some time to kick in the changes?
>
> Here is the sample of changes:
>
> log4j.appender.stateChangeAppender=org.apache.log4j.DailyRollingFileAppender
> log4j.appender.stateChangeAppender.DatePattern='.'yyyy-MM-dd-HH
> log4j.appender.stateChangeAppender.File=${kafka.logs.dir}/state-change.log
> log4j.appender.stateChangeAppender.layout=org.apache.log4j.PatternLayout
> log4j.appender.stateChangeAppender.layout.ConversionPattern=[%d] %p %m
> (%c)%n
> log4j.appender.stateChangeAppender.MaxFileSize=100KB
> log4j.appender.stateChangeAppender.MaxBackupIndex=5
> .
> .
> log4j.appender.kafkaAppender.MaxFileSize=100KB
> log4j.appender.kafkaAppender.MaxBackupIndex=5
> .
> .
> log4j.appender.controllerAppender.MaxFileSize=100KB
> log4j.appender.controllerAppender.MaxBackupIndex=5
>
>
> -----Original Message-----
> From: Gwen Shapira [mailto:gwen@confluent.io]
> Sent: Friday, September 25, 2015 1:13 PM
> To: users@kafka.apache.org
> Subject: Re: log clean up
>
> Absolutely.
>
> You can go into config/log4j.properties and configure the appenders to
> roll the logs.
>
> For example:
>
>
> log4j.appender.stateChangeAppender=org.apache.log4j.DailyRollingFileAppender
> log4j.appender.stateChangeAppender.DatePattern='.'yyyy-MM-dd-HH
> log4j.appender.stateChangeAppender.File=${kafka.logs.dir}/state-change.log
> log4j.appender.stateChangeAppender.layout=org.apache.log4j.PatternLayout
> log4j.appender.stateChangeAppender.layout.ConversionPattern=[%d] %p %m
> (%c)%n
>
> log4j.appender.stateChangeAppender.MaxFileSize=100KB
> log4j.appender.stateChangeAppender.MaxBackupIndex=5
>
> Adding the last two lines will make sure you have 5 state change log files
> each 100KB.
>
> Gwen
>
> On Fri, Sep 25, 2015 at 11:04 AM, Hema Bhatia <He...@apollo.edu>
> wrote:
>
> > Is there a way to delete kafka server, controller and state-change logs.
> > They just keep growing over time and not purged.
> >
> > -Hema
> >
> >
> >
>
RE: log clean up
Posted by Hema Bhatia <He...@apollo.edu>.
Thanks Gwen!
I made the changes and restarted kafka nodes. Looks like all log files are still present. Does it take some time to kick in the changes?
Here is the sample of changes:
log4j.appender.stateChangeAppender=org.apache.log4j.DailyRollingFileAppender
log4j.appender.stateChangeAppender.DatePattern='.'yyyy-MM-dd-HH
log4j.appender.stateChangeAppender.File=${kafka.logs.dir}/state-change.log
log4j.appender.stateChangeAppender.layout=org.apache.log4j.PatternLayout
log4j.appender.stateChangeAppender.layout.ConversionPattern=[%d] %p %m (%c)%n
log4j.appender.stateChangeAppender.MaxFileSize=100KB
log4j.appender.stateChangeAppender.MaxBackupIndex=5
.
.
log4j.appender.kafkaAppender.MaxFileSize=100KB
log4j.appender.kafkaAppender.MaxBackupIndex=5
.
.
log4j.appender.controllerAppender.MaxFileSize=100KB
log4j.appender.controllerAppender.MaxBackupIndex=5
-----Original Message-----
From: Gwen Shapira [mailto:gwen@confluent.io]
Sent: Friday, September 25, 2015 1:13 PM
To: users@kafka.apache.org
Subject: Re: log clean up
Absolutely.
You can go into config/log4j.properties and configure the appenders to roll the logs.
For example:
log4j.appender.stateChangeAppender=org.apache.log4j.DailyRollingFileAppender
log4j.appender.stateChangeAppender.DatePattern='.'yyyy-MM-dd-HH
log4j.appender.stateChangeAppender.File=${kafka.logs.dir}/state-change.log
log4j.appender.stateChangeAppender.layout=org.apache.log4j.PatternLayout
log4j.appender.stateChangeAppender.layout.ConversionPattern=[%d] %p %m (%c)%n
log4j.appender.stateChangeAppender.MaxFileSize=100KB
log4j.appender.stateChangeAppender.MaxBackupIndex=5
Adding the last two lines will make sure you have 5 state change log files each 100KB.
Gwen
On Fri, Sep 25, 2015 at 11:04 AM, Hema Bhatia <He...@apollo.edu>
wrote:
> Is there a way to delete kafka server, controller and state-change logs.
> They just keep growing over time and not purged.
>
> -Hema
>
>
>
Re: log clean up
Posted by Gwen Shapira <gw...@confluent.io>.
Absolutely.
You can go into config/log4j.properties and configure the appenders to roll
the logs.
For example:
log4j.appender.stateChangeAppender=org.apache.log4j.DailyRollingFileAppender
log4j.appender.stateChangeAppender.DatePattern='.'yyyy-MM-dd-HH
log4j.appender.stateChangeAppender.File=${kafka.logs.dir}/state-change.log
log4j.appender.stateChangeAppender.layout=org.apache.log4j.PatternLayout
log4j.appender.stateChangeAppender.layout.ConversionPattern=[%d] %p %m (%c)%n
log4j.appender.stateChangeAppender.MaxFileSize=100KB
log4j.appender.stateChangeAppender.MaxBackupIndex=5
Adding the last two lines will make sure you have 5 state change log files
each 100KB.
Gwen
On Fri, Sep 25, 2015 at 11:04 AM, Hema Bhatia <He...@apollo.edu>
wrote:
> Is there a way to delete kafka server, controller and state-change logs.
> They just keep growing over time and not purged.
>
> -Hema
>
>
>