High disk space usage after replication

[mechravi25 <me...@yahoo.co.in>]

Hi,
 
I'm using one master and slave in SOLR. When I try to replicate from master
to slave, the data is getting replicated properly and the changes are
getting implemented rightly in the SOLR UI. But, the indexing size is
doubled in the slave when compared to the master. (i.e.) for eg:
If,
Master Indexed Data Size is 270MB,
The Slave's Indexed Data Size is 600MB
 
It seems like its retaining the old data. we had referred to the following
links regarding this,
http://wiki.apache.org/solr/CollectionDistribution#snapcleaner
http://markmail.org/thread/yw5n4dk2t5zbt5z5#query:+page:1+mid:43cqxnjkecfnotiz+state:results
 
Previously, It was working fine; but, this kind of issue has started to
appear only now. The Version Specifications that we are using is as follows
 
SOLR Master

Solr Specification Version: 1.4.0.2010.01.13.08.09.44 
Solr Implementation Version: 1.5-dev exported - yonik - 2010-01-13 08:09:44 
Lucene Specification Version: 2.9.1-dev 
Lucene Implementation Version: 2.9.1-dev 888785 - 2009-12-09 18:03:31 
 
SOLR Slave

Solr Specification Version: 1.4.0.2010.01.13.08.09.44 
Solr Implementation Version: 1.5-dev exported - yonik - 2010-01-13 08:09:44 
Lucene Specification Version: 2.9.1-dev 
Lucene Implementation Version: 2.9.1-dev 888785 - 2009-12-09 18:03:31 

Can someone please tell me where I'm going wrong and guide me on this?

Thanks.


--
View this message in context: http://lucene.472066.n3.nabble.com/High-disk-space-usage-after-replication-tp3806417p3806417.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: High disk space usage after replication

[Erick Erickson <er...@gmail.com>]

Well, I'd upgrade to a newer Solr <G>...

But seriously, first there is an expected temporary spike during
replication, you can expect the  size on disk to occasionally
up to double during replication, that's just how replication is
designed to work...

But if the files are *staying*, then that is, indeed, odd. Do they remain
after you say, bounce the slave? Under *nix op systems, files
remain until all processes are done using them, so it's possible
that you're looking at them before, say, warmups are done.

If you bounce the slave and they disappear, then you might
be able to chalk it up to a one-time issue (hand waving here).

Best
Erick

On Wed, Mar 7, 2012 at 6:20 AM, mechravi25 <me...@yahoo.co.in> wrote:
> Hi,
>
> I'm using one master and slave in SOLR. When I try to replicate from master
> to slave, the data is getting replicated properly and the changes are
> getting implemented rightly in the SOLR UI. But, the indexing size is
> doubled in the slave when compared to the master. (i.e.) for eg:
> If,
> Master Indexed Data Size is 270MB,
> The Slave's Indexed Data Size is 600MB
>
> It seems like its retaining the old data. we had referred to the following
> links regarding this,
> http://wiki.apache.org/solr/CollectionDistribution#snapcleaner
> http://markmail.org/thread/yw5n4dk2t5zbt5z5#query:+page:1+mid:43cqxnjkecfnotiz+state:results
>
> Previously, It was working fine; but, this kind of issue has started to
> appear only now. The Version Specifications that we are using is as follows
>
> SOLR Master
>
> Solr Specification Version: 1.4.0.2010.01.13.08.09.44
> Solr Implementation Version: 1.5-dev exported - yonik - 2010-01-13 08:09:44
> Lucene Specification Version: 2.9.1-dev
> Lucene Implementation Version: 2.9.1-dev 888785 - 2009-12-09 18:03:31
>
> SOLR Slave
>
> Solr Specification Version: 1.4.0.2010.01.13.08.09.44
> Solr Implementation Version: 1.5-dev exported - yonik - 2010-01-13 08:09:44
> Lucene Specification Version: 2.9.1-dev
> Lucene Implementation Version: 2.9.1-dev 888785 - 2009-12-09 18:03:31
>
> Can someone please tell me where I'm going wrong and guide me on this?
>
> Thanks.
>
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/High-disk-space-usage-after-replication-tp3806417p3806417.html
> Sent from the Solr - User mailing list archive at Nabble.com.