You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Jason <hi...@gmail.com> on 2012/12/28 01:25:49 UTC
old index not cleaned up on the slave
Hi,
I'm using master/slave replication on Solr 4.0.
Replication is successfully run.
But old index not cleaned up.
Is that bug or not?
My slave index directory is below...
$ ls -l solr_kr/krg01/data/index/
total 23472512
-rw-r--r--. 1 tomcat tomcat 563722625 Dec 24 21:48 _15.fdt
-rw-r--r--. 1 tomcat tomcat 4855210 Dec 24 21:48 _15.fdx
-rw-r--r--. 1 tomcat tomcat 4155 Dec 24 22:01 _15.fnm
-rw-r--r--. 1 tomcat tomcat 3367203143 Dec 24 22:01 _15_Lucene40_0.frq
-rw-r--r--. 1 tomcat tomcat 6951612380 Dec 24 22:01 _15_Lucene40_0.prx
-rw-r--r--. 1 tomcat tomcat 1096591353 Dec 24 22:01 _15_Lucene40_0.tim
-rw-r--r--. 1 tomcat tomcat 26026916 Dec 24 22:01 _15_Lucene40_0.tip
-rw-r--r--. 1 tomcat tomcat 388 Dec 24 22:01 _15.si
-rw-r--r--. 1 tomcat tomcat 98 Nov 30 13:43 segments_3
-rw-r--r--. 1 tomcat tomcat 99 Dec 24 22:01 segments_4
-rw-r--r--. 1 tomcat tomcat 20 Aug 12 07:21 segments.gen
-rw-r--r--. 1 tomcat tomcat 563742324 Nov 30 13:32 _t.fdt
-rw-r--r--. 1 tomcat tomcat 4855210 Nov 30 13:32 _t.fdx
-rw-r--r--. 1 tomcat tomcat 4155 Nov 30 13:43 _t.fnm
-rw-r--r--. 1 tomcat tomcat 3382846438 Nov 30 13:43 _t_Lucene40_0.frq
-rw-r--r--. 1 tomcat tomcat 6951620034 Nov 30 13:43 _t_Lucene40_0.prx
-rw-r--r--. 1 tomcat tomcat 1096654275 Nov 30 13:43 _t_Lucene40_0.tim
-rw-r--r--. 1 tomcat tomcat 26027222 Nov 30 13:43 _t_Lucene40_0.tip
-rw-r--r--. 1 tomcat tomcat 379 Nov 30 13:43 _t.si
--
View this message in context: http://lucene.472066.n3.nabble.com/old-index-not-cleaned-up-on-the-slave-tp4029370.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: old index not cleaned up on the slave
Posted by Jason <hi...@gmail.com>.
Hi, Upayavira
I know multiple segments are not problem.
But I always optimize index on master server before replicate.
So just single segment file is on master.
File lists of the master server directory are below.
Additionally, segments_1 and segments_2 on slave server are deleted by hand.
[root@solr06 4_0]# ls -l solr_kr/krg01/data/index/
total 11728576
-rw-r--r--. 1 tomcat tomcat 563722625 Dec 24 21:48 _15.fdt
-rw-r--r--. 1 tomcat tomcat 4855210 Dec 24 21:48 _15.fdx
-rw-r--r--. 1 tomcat tomcat 4155 Dec 24 22:01 _15.fnm
-rw-r--r--. 1 tomcat tomcat 3367203143 Dec 24 22:01 _15_Lucene40_0.frq
-rw-r--r--. 1 tomcat tomcat 6951612380 Dec 24 22:01 _15_Lucene40_0.prx
-rw-r--r--. 1 tomcat tomcat 1096591353 Dec 24 22:01 _15_Lucene40_0.tim
-rw-r--r--. 1 tomcat tomcat 26026916 Dec 24 22:01 _15_Lucene40_0.tip
-rw-r--r--. 1 tomcat tomcat 388 Dec 24 22:01 _15.si
-rw-r--r--. 1 tomcat tomcat 99 Dec 24 22:01 segments_4
-rw-r--r--. 1 tomcat tomcat 20 Dec 24 22:01 segments.gen
-rw-r--r--. 1 tomcat tomcat 0 Dec 26 17:25 write.lock
--
View this message in context: http://lucene.472066.n3.nabble.com/old-index-not-cleaned-up-on-the-slave-tp4029370p4029981.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: old index not cleaned up on the slave
Posted by Upayavira <uv...@odoko.co.uk>.
You've shown us the directory on the slave but not the master, so it is
hard to see how you have decided that stuff should have been deleted. I
see two segments_* files which shouldn't be the case, but they are
small. The other files are segment files and multiple segments is
legitimate.
Upayavira
On Mon, Dec 31, 2012, at 12:55 AM, Jason wrote:
> Hi, Erick
> I didn't configure anything for index backup.
> My ReplicationHandler configuration is below.
> Other setting in solrconfig.xml is almost default.
> Is there a deletion policy for replication?
> I know "maxNumberOfBackups" parameter, but this is for master server.
> Are there any configuration for index backup on slave server?
>
> <requestHandler name="/replication" class="solr.ReplicationHandler">
> <lst name="master">
> <str name="enable">${solr.master.enable:false}</str>
> <str name="replicateAfter">optimize</str>
> <str name="replicateAfter">startup</str>
> <str
> name="confFiles">schema.xml,solrconfig.xml,db-data-config.xml,stopwords.txt,stopwords_en.txt</str>
> </lst>
> <lst name="slave">
> <str name="enable">${solr.slave.enable:false}</str>
> <str
> name="masterUrl">http://${solr.master.url}/${solr.context.name}/${solr.core.name}/replication</str>
> </lst>
> </requestHandler>
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/old-index-not-cleaned-up-on-the-slave-tp4029370p4029736.html
> Sent from the Solr - User mailing list archive at Nabble.com.
Re: old index not cleaned up on the slave
Posted by Jason <hi...@gmail.com>.
Hi, Erick
I didn't configure anything for index backup.
My ReplicationHandler configuration is below.
Other setting in solrconfig.xml is almost default.
Is there a deletion policy for replication?
I know "maxNumberOfBackups" parameter, but this is for master server.
Are there any configuration for index backup on slave server?
<requestHandler name="/replication" class="solr.ReplicationHandler">
<lst name="master">
<str name="enable">${solr.master.enable:false}</str>
<str name="replicateAfter">optimize</str>
<str name="replicateAfter">startup</str>
<str
name="confFiles">schema.xml,solrconfig.xml,db-data-config.xml,stopwords.txt,stopwords_en.txt</str>
</lst>
<lst name="slave">
<str name="enable">${solr.slave.enable:false}</str>
<str
name="masterUrl">http://${solr.master.url}/${solr.context.name}/${solr.core.name}/replication</str>
</lst>
</requestHandler>
--
View this message in context: http://lucene.472066.n3.nabble.com/old-index-not-cleaned-up-on-the-slave-tp4029370p4029736.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: old index not cleaned up on the slave
Posted by Erick Erickson <er...@gmail.com>.
What is your index deletion policy? Depending on how that's configured, N
old indexes may be kept.....
Best
Erick
On Thu, Dec 27, 2012 at 7:25 PM, Jason <hi...@gmail.com> wrote:
> Hi,
> I'm using master/slave replication on Solr 4.0.
> Replication is successfully run.
> But old index not cleaned up.
> Is that bug or not?
>
> My slave index directory is below...
>
> $ ls -l solr_kr/krg01/data/index/
> total 23472512
> -rw-r--r--. 1 tomcat tomcat 563722625 Dec 24 21:48 _15.fdt
> -rw-r--r--. 1 tomcat tomcat 4855210 Dec 24 21:48 _15.fdx
> -rw-r--r--. 1 tomcat tomcat 4155 Dec 24 22:01 _15.fnm
> -rw-r--r--. 1 tomcat tomcat 3367203143 Dec 24 22:01 _15_Lucene40_0.frq
> -rw-r--r--. 1 tomcat tomcat 6951612380 Dec 24 22:01 _15_Lucene40_0.prx
> -rw-r--r--. 1 tomcat tomcat 1096591353 Dec 24 22:01 _15_Lucene40_0.tim
> -rw-r--r--. 1 tomcat tomcat 26026916 Dec 24 22:01 _15_Lucene40_0.tip
> -rw-r--r--. 1 tomcat tomcat 388 Dec 24 22:01 _15.si
> -rw-r--r--. 1 tomcat tomcat 98 Nov 30 13:43 segments_3
> -rw-r--r--. 1 tomcat tomcat 99 Dec 24 22:01 segments_4
> -rw-r--r--. 1 tomcat tomcat 20 Aug 12 07:21 segments.gen
> -rw-r--r--. 1 tomcat tomcat 563742324 Nov 30 13:32 _t.fdt
> -rw-r--r--. 1 tomcat tomcat 4855210 Nov 30 13:32 _t.fdx
> -rw-r--r--. 1 tomcat tomcat 4155 Nov 30 13:43 _t.fnm
> -rw-r--r--. 1 tomcat tomcat 3382846438 Nov 30 13:43 _t_Lucene40_0.frq
> -rw-r--r--. 1 tomcat tomcat 6951620034 Nov 30 13:43 _t_Lucene40_0.prx
> -rw-r--r--. 1 tomcat tomcat 1096654275 Nov 30 13:43 _t_Lucene40_0.tim
> -rw-r--r--. 1 tomcat tomcat 26027222 Nov 30 13:43 _t_Lucene40_0.tip
> -rw-r--r--. 1 tomcat tomcat 379 Nov 30 13:43 _t.si
>
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/old-index-not-cleaned-up-on-the-slave-tp4029370.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>