You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@zookeeper.apache.org by Ryan Fox <ry...@mediabeacon.com> on 2013/11/04 16:46:07 UTC
Re: auto-purge not purging
>
> What version of ZooKeeper are you running? (solr 4.1.0 is pretty old,
> 4.5.1 is out now, perhaps try upgrading?)
>
> Zookeeper version is 3.4.5. This is part of a large software package, so
upgrading the solr library would be painful, but possible.
"do not get autopurged" - could you provide more insight. 1) are you
> sure the configuration options are being applied? 2) what does the
> datadir look like? (snaps/logs, perhaps do an "ls -la" and provide in
> this thread) 3) anything autopurge related in the zk server logs?
>
> Sure, the datadir contains only logs and snapshots. I would expect the
autoPurge function to delete logs and snapshots older than the number I
have set in zoo.cfg, but nothing gets deleted, and they just pile up,
eating up my hard drive.
1) The zookeeper instance is picking up my settings for clientPort, so I
assume I have the purge settings in the correct place as well. The
relevant section of zoo.cfg:
autopurge.snapRetainCount=3
autopurge.purgeInterval=1
2) ls -lah
-rw-r--r-- 1 ryanfox staff 64M Oct 30 13:09 log.1
-rw-r--r-- 1 ryanfox staff 64M Nov 4 09:36 log.48
-rw-r--r-- 1 ryanfox staff 64M Nov 4 09:37 log.7b
-rw-r--r-- 1 ryanfox staff 296B Oct 30 12:53 snapshot.0
-rw-r--r-- 1 ryanfox staff 60K Nov 4 09:34 snapshot.47
-rw-r--r-- 1 ryanfox staff 60K Nov 4 09:36 snapshot.7a
A new log and snapshot gets created every time I stop and start solr. As
an aside, every log file appears to be 64MB, no matter how much/little is
in my index. Is that correct? Seems like a lot for an empty solr instance.
3) I haven't seen any smoking gun in the logs, but haven't specifically
looked, either. I'll let it go until it should have triggered the
auto-purge and report back.
Thanks for your help in this.
Ryan
Re: auto-purge not purging
Posted by Ryan Fox <ry...@mediabeacon.com>.
Ah, sorry I wasn't clear. There's currently 3 log/snapshot pairs in that
dir, because I cleared it out recently. If I keep starting and stopping
solr/zookeeper, they'll continue to pile up.
$ ls -lah
drwxr-xr-x 14 ryanfox staff 476B Nov 4 17:25 .
drwxr-xr-x 4 ryanfox staff 136B Nov 4 09:30 ..
-rw-r--r-- 1 ryanfox staff 64M Oct 30 13:09 log.1
-rw-r--r-- 1 ryanfox staff 64M Nov 4 17:25 log.112
-rw-r--r-- 1 ryanfox staff 64M Nov 4 09:36 log.48
-rw-r--r-- 1 ryanfox staff 64M Nov 4 17:22 log.7b
-rw-r--r-- 1 ryanfox staff 64M Nov 4 17:22 log.ad
-rw-r--r-- 1 ryanfox staff 64M Nov 4 17:25 log.e1
-rw-r--r-- 1 ryanfox staff 296B Oct 30 12:53 snapshot.0
-rw-r--r-- 1 ryanfox staff 60K Nov 4 17:25 snapshot.111
-rw-r--r-- 1 ryanfox staff 60K Nov 4 09:34 snapshot.47
-rw-r--r-- 1 ryanfox staff 60K Nov 4 09:36 snapshot.7a
-rw-r--r-- 1 ryanfox staff 60K Nov 4 17:22 snapshot.ac
-rw-r--r-- 1 ryanfox staff 60K Nov 4 17:23 snapshot.e0
Thanks for the info on preAllocSize, that may prove useful as well.
Nothing showed up in the logs today despite running for ~8 hours.
Best,
Ryan
On Mon, Nov 4, 2013 at 5:13 PM, Patrick Hunt <ph...@apache.org> wrote:
> On Mon, Nov 4, 2013 at 7:46 AM, Ryan Fox <ry...@mediabeacon.com> wrote:
> >>
> >> What version of ZooKeeper are you running? (solr 4.1.0 is pretty old,
> >> 4.5.1 is out now, perhaps try upgrading?)
> >>
> >>
>
> > Zookeeper version is 3.4.5. This is part of a large software package, so
> > upgrading the solr library would be painful, but possible.
> >
>
> 3.4.5 is fine, I just wasn't sure what version solr was using, if it
> was old it might be related but I don't think that's the case here.
>
> >> "do not get autopurged" - could you provide more insight. 1) are you
> >> sure the configuration options are being applied? 2) what does the
> >> datadir look like? (snaps/logs, perhaps do an "ls -la" and provide in
> >> this thread) 3) anything autopurge related in the zk server logs?
> >>
>
> > Sure, the datadir contains only logs and snapshots. I would expect the
> > autoPurge function to delete logs and snapshots older than the number I
> > have set in zoo.cfg, but nothing gets deleted, and they just pile up,
> > eating up my hard drive.
> > 1) The zookeeper instance is picking up my settings for clientPort, so I
> > assume I have the purge settings in the correct place as well. The
> > relevant section of zoo.cfg:
> > autopurge.snapRetainCount=3
> > autopurge.purgeInterval=1
> >
> > 2) ls -lah
> > -rw-r--r-- 1 ryanfox staff 64M Oct 30 13:09 log.1
> > -rw-r--r-- 1 ryanfox staff 64M Nov 4 09:36 log.48
> > -rw-r--r-- 1 ryanfox staff 64M Nov 4 09:37 log.7b
> > -rw-r--r-- 1 ryanfox staff 296B Oct 30 12:53 snapshot.0
> > -rw-r--r-- 1 ryanfox staff 60K Nov 4 09:34 snapshot.47
> > -rw-r--r-- 1 ryanfox staff 60K Nov 4 09:36 snapshot.7a
> >
>
> You have our snapRetainCount set to 3. That's why three snaps (and
> their related logs) are saved.
>
> Try setting it to 2.
>
> > A new log and snapshot gets created every time I stop and start solr. As
> > an aside, every log file appears to be 64MB, no matter how much/little is
> > in my index. Is that correct? Seems like a lot for an empty solr
> instance.
> >
>
> We pre-allocate the log files for performance. See "preAllocSize" here:
> http://zookeeper.apache.org/doc/r3.4.5/zookeeperAdmin.html
>
> If you want smaller files try setting that to something like 10k or
> somesuch.
>
> > 3) I haven't seen any smoking gun in the logs, but haven't specifically
> > looked, either. I'll let it go until it should have triggered the
> > auto-purge and report back.
>
> See if the above suggestions/info help.
>
> Obviously don't carry over these changes to production. ;-)
>
> Regards,
>
> Patrick
>
Re: auto-purge not purging
Posted by Patrick Hunt <ph...@apache.org>.
On Mon, Nov 4, 2013 at 7:46 AM, Ryan Fox <ry...@mediabeacon.com> wrote:
>>
>> What version of ZooKeeper are you running? (solr 4.1.0 is pretty old,
>> 4.5.1 is out now, perhaps try upgrading?)
>>
>>
> Zookeeper version is 3.4.5. This is part of a large software package, so
> upgrading the solr library would be painful, but possible.
>
3.4.5 is fine, I just wasn't sure what version solr was using, if it
was old it might be related but I don't think that's the case here.
>> "do not get autopurged" - could you provide more insight. 1) are you
>> sure the configuration options are being applied? 2) what does the
>> datadir look like? (snaps/logs, perhaps do an "ls -la" and provide in
>> this thread) 3) anything autopurge related in the zk server logs?
>>
> Sure, the datadir contains only logs and snapshots. I would expect the
> autoPurge function to delete logs and snapshots older than the number I
> have set in zoo.cfg, but nothing gets deleted, and they just pile up,
> eating up my hard drive.
> 1) The zookeeper instance is picking up my settings for clientPort, so I
> assume I have the purge settings in the correct place as well. The
> relevant section of zoo.cfg:
> autopurge.snapRetainCount=3
> autopurge.purgeInterval=1
>
> 2) ls -lah
> -rw-r--r-- 1 ryanfox staff 64M Oct 30 13:09 log.1
> -rw-r--r-- 1 ryanfox staff 64M Nov 4 09:36 log.48
> -rw-r--r-- 1 ryanfox staff 64M Nov 4 09:37 log.7b
> -rw-r--r-- 1 ryanfox staff 296B Oct 30 12:53 snapshot.0
> -rw-r--r-- 1 ryanfox staff 60K Nov 4 09:34 snapshot.47
> -rw-r--r-- 1 ryanfox staff 60K Nov 4 09:36 snapshot.7a
>
You have our snapRetainCount set to 3. That's why three snaps (and
their related logs) are saved.
Try setting it to 2.
> A new log and snapshot gets created every time I stop and start solr. As
> an aside, every log file appears to be 64MB, no matter how much/little is
> in my index. Is that correct? Seems like a lot for an empty solr instance.
>
We pre-allocate the log files for performance. See "preAllocSize" here:
http://zookeeper.apache.org/doc/r3.4.5/zookeeperAdmin.html
If you want smaller files try setting that to something like 10k or somesuch.
> 3) I haven't seen any smoking gun in the logs, but haven't specifically
> looked, either. I'll let it go until it should have triggered the
> auto-purge and report back.
See if the above suggestions/info help.
Obviously don't carry over these changes to production. ;-)
Regards,
Patrick