You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Edwin Khodabakchian (JIRA)" <ji...@apache.org> on 2011/01/23 00:50:43 UTC
[jira] Updated: (SOLR-2329) old index files not deleted on slave
[ https://issues.apache.org/jira/browse/SOLR-2329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edwin Khodabakchian updated SOLR-2329:
--------------------------------------
Attachment: solrconfig_slave.xml
solrconfig.xml
solr config files
> old index files not deleted on slave
> ------------------------------------
>
> Key: SOLR-2329
> URL: https://issues.apache.org/jira/browse/SOLR-2329
> Project: Solr
> Issue Type: Bug
> Components: replication (java)
> Affects Versions: 4.0
> Environment: centos 5.5
> ext3 file system
> Reporter: Edwin Khodabakchian
> Attachments: solrconfig.xml, solrconfig_slave.xml
>
>
> I have set up index replication (triggered on optimize). The problem I
> am having is the old index files are not being deleted on the slave.
> After each replication, I can see the old files still hanging around
> as well as the files that have just been pulled. This causes the data
> directory size to increase by the index size every replication until
> the disk fills up.
> I am running 4.0 rev 993367 with patch SOLR-1316. Otherwise, the setup
> is pretty vanilla. I can reproduce this on multiple slaves.
> Checking the logs, I see the following error:
> SEVERE: SnapPull failed
> org.apache.solr.common.SolrException: Index fetch failed :
> at org.apache.solr.handler.SnapPuller.fetchLatestIndex(SnapPuller.java:329)
> at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:265)
> at org.apache.solr.handler.SnapPuller$1.run(SnapPuller.java:159)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
> at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:619)
> Caused by: org.apache.lucene.store.LockObtainFailedException: Lock
> obtain timed out:
> NativeFSLock@/var/solrhome/data/index/lucene-cdaa80c0fefe1a7dfc7aab89298c614c-write.lock
> at org.apache.lucene.store.Lock.obtain(Lock.java:84)
> at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:1065)
> at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:954)
> at org.apache.solr.update.SolrIndexWriter.<init>(SolrIndexWriter.java:192)
> at org.apache.solr.update.UpdateHandler.createMainIndexWriter(UpdateHandler.java:99)
> at org.apache.solr.update.DirectUpdateHandler2.openWriter(DirectUpdateHandler2.java:173)
> at org.apache.solr.update.DirectUpdateHandler2.forceOpenWriter(DirectUpdateHandler2.java:376)
> at org.apache.solr.handler.SnapPuller.doCommit(SnapPuller.java:471)
> at org.apache.solr.handler.SnapPuller.fetchLatestIndex(SnapPuller.java:319)
> ... 11 more
> lsof reveals that the file is still opened from the java process.
> Contents of the index data dir:
> master:
> -rw-rw-r-- 1 feeddo feeddo 191 Dec 14 01:06 _1lg.fnm
> -rw-rw-r-- 1 feeddo feeddo 26M Dec 14 01:07 _1lg.fdx
> -rw-rw-r-- 1 feeddo feeddo 1.9G Dec 14 01:07 _1lg.fdt
> -rw-rw-r-- 1 feeddo feeddo 474M Dec 14 01:12 _1lg.tis
> -rw-rw-r-- 1 feeddo feeddo 15M Dec 14 01:12 _1lg.tii
> -rw-rw-r-- 1 feeddo feeddo 144M Dec 14 01:12 _1lg.prx
> -rw-rw-r-- 1 feeddo feeddo 277M Dec 14 01:12 _1lg.frq
> -rw-rw-r-- 1 feeddo feeddo 311 Dec 14 01:12 segments_1ji
> -rw-rw-r-- 1 feeddo feeddo 23M Dec 14 01:12 _1lg.nrm
> -rw-rw-r-- 1 feeddo feeddo 191 Dec 18 01:11 _24e.fnm
> -rw-rw-r-- 1 feeddo feeddo 26M Dec 18 01:12 _24e.fdx
> -rw-rw-r-- 1 feeddo feeddo 1.9G Dec 18 01:12 _24e.fdt
> -rw-rw-r-- 1 feeddo feeddo 483M Dec 18 01:23 _24e.tis
> -rw-rw-r-- 1 feeddo feeddo 15M Dec 18 01:23 _24e.tii
> -rw-rw-r-- 1 feeddo feeddo 146M Dec 18 01:23 _24e.prx
> -rw-rw-r-- 1 feeddo feeddo 283M Dec 18 01:23 _24e.frq
> -rw-rw-r-- 1 feeddo feeddo 311 Dec 18 01:24 segments_1xz
> -rw-rw-r-- 1 feeddo feeddo 23M Dec 18 01:24 _24e.nrm
> -rw-rw-r-- 1 feeddo feeddo 191 Dec 18 13:15 _25z.fnm
> -rw-rw-r-- 1 feeddo feeddo 26M Dec 18 13:16 _25z.fdx
> -rw-rw-r-- 1 feeddo feeddo 1.9G Dec 18 13:16 _25z.fdt
> -rw-rw-r-- 1 feeddo feeddo 484M Dec 18 13:35 _25z.tis
> -rw-rw-r-- 1 feeddo feeddo 15M Dec 18 13:35 _25z.tii
> -rw-rw-r-- 1 feeddo feeddo 146M Dec 18 13:35 _25z.prx
> -rw-rw-r-- 1 feeddo feeddo 284M Dec 18 13:35 _25z.frq
> -rw-rw-r-- 1 feeddo feeddo 20 Dec 18 13:35 segments.gen
> -rw-rw-r-- 1 feeddo feeddo 311 Dec 18 13:35 segments_1y1
> -rw-rw-r-- 1 feeddo feeddo 23M Dec 18 13:35 _25z.nrm
> slave:
> -rw-rw-r-- 1 feeddo feeddo 20 Dec 13 17:54 segments.gen
> -rw-rw-r-- 1 feeddo feeddo 191 Dec 15 01:07 _1mk.fnm
> -rw-rw-r-- 1 feeddo feeddo 26M Dec 15 01:08 _1mk.fdx
> -rw-rw-r-- 1 feeddo feeddo 1.9G Dec 15 01:08 _1mk.fdt
> -rw-rw-r-- 1 feeddo feeddo 476M Dec 15 01:18 _1mk.tis
> -rw-rw-r-- 1 feeddo feeddo 15M Dec 15 01:18 _1mk.tii
> -rw-rw-r-- 1 feeddo feeddo 144M Dec 15 01:18 _1mk.prx
> -rw-rw-r-- 1 feeddo feeddo 278M Dec 15 01:18 _1mk.frq
> -rw-rw-r-- 1 feeddo feeddo 312 Dec 15 01:18 segments_1kj
> -rw-rw-r-- 1 feeddo feeddo 23M Dec 15 01:18 _1mk.nrm
> -rw-rw-r-- 1 feeddo feeddo 0 Dec 15 01:19
> lucene-cdaa80c0fefe1a7dfc7aab89298c614c-write.lock
> -rw-rw-r-- 1 feeddo feeddo 191 Dec 15 13:14 _1qu.fnm
> -rw-rw-r-- 1 feeddo feeddo 26M Dec 15 13:16 _1qu.fdx
> -rw-rw-r-- 1 feeddo feeddo 1.9G Dec 15 13:16 _1qu.fdt
> -rw-rw-r-- 1 feeddo feeddo 477M Dec 15 13:28 _1qu.tis
> -rw-rw-r-- 1 feeddo feeddo 15M Dec 15 13:28 _1qu.tii
> -rw-rw-r-- 1 feeddo feeddo 144M Dec 15 13:28 _1qu.prx
> -rw-rw-r-- 1 feeddo feeddo 278M Dec 15 13:28 _1qu.frq
> -rw-rw-r-- 1 feeddo feeddo 311 Dec 15 13:28 segments_1oe
> -rw-rw-r-- 1 feeddo feeddo 23M Dec 15 13:28 _1qu.nrm
> -rw-rw-r-- 1 feeddo feeddo 191 Dec 17 01:12 _222.fnm
> -rw-rw-r-- 1 feeddo feeddo 26M Dec 17 01:15 _222.fdx
> -rw-rw-r-- 1 feeddo feeddo 1.9G Dec 17 01:15 _222.fdt
> -rw-rw-r-- 1 feeddo feeddo 481M Dec 17 01:36 _222.tis
> -rw-rw-r-- 1 feeddo feeddo 15M Dec 17 01:36 _222.tii
> -rw-rw-r-- 1 feeddo feeddo 145M Dec 17 01:36 _222.prx
> -rw-rw-r-- 1 feeddo feeddo 281M Dec 17 01:36 _222.frq
> -rw-rw-r-- 1 feeddo feeddo 23M Dec 17 01:36 _222.nrm
> -rw-rw-r-- 1 feeddo feeddo 311 Dec 17 01:36 segments_1xv
> -rw-rw-r-- 1 feeddo feeddo 191 Dec 17 13:10 _233.fnm
> -rw-rw-r-- 1 feeddo feeddo 26M Dec 17 13:13 _233.fdx
> -rw-rw-r-- 1 feeddo feeddo 1.9G Dec 17 13:13 _233.fdt
> -rw-rw-r-- 1 feeddo feeddo 482M Dec 17 13:31 _233.tis
> -rw-rw-r-- 1 feeddo feeddo 15M Dec 17 13:31 _233.tii
> -rw-rw-r-- 1 feeddo feeddo 146M Dec 17 13:31 _233.prx
> -rw-rw-r-- 1 feeddo feeddo 282M Dec 17 13:31 _233.frq
> -rw-rw-r-- 1 feeddo feeddo 311 Dec 17 13:31 segments_1xx
> -rw-rw-r-- 1 feeddo feeddo 23M Dec 17 13:31 _233.nrm
> -rw-rw-r-- 1 feeddo feeddo 191 Dec 18 01:11 _24e.fnm
> -rw-rw-r-- 1 feeddo feeddo 26M Dec 18 01:12 _24e.fdx
> -rw-rw-r-- 1 feeddo feeddo 1.9G Dec 18 01:12 _24e.fdt
> -rw-rw-r-- 1 feeddo feeddo 483M Dec 18 01:23 _24e.tis
> -rw-rw-r-- 1 feeddo feeddo 15M Dec 18 01:23 _24e.tii
> -rw-rw-r-- 1 feeddo feeddo 146M Dec 18 01:23 _24e.prx
> -rw-rw-r-- 1 feeddo feeddo 283M Dec 18 01:23 _24e.frq
> -rw-rw-r-- 1 feeddo feeddo 311 Dec 18 01:24 segments_1xz
> -rw-rw-r-- 1 feeddo feeddo 23M Dec 18 01:24 _24e.nrm
> -rw-rw-r-- 1 feeddo feeddo 191 Dec 18 13:15 _25z.fnm
> -rw-rw-r-- 1 feeddo feeddo 26M Dec 18 13:16 _25z.fdx
> -rw-rw-r-- 1 feeddo feeddo 1.9G Dec 18 13:16 _25z.fdt
> -rw-rw-r-- 1 feeddo feeddo 484M Dec 18 13:35 _25z.tis
> -rw-rw-r-- 1 feeddo feeddo 15M Dec 18 13:35 _25z.tii
> -rw-rw-r-- 1 feeddo feeddo 146M Dec 18 13:35 _25z.prx
> -rw-rw-r-- 1 feeddo feeddo 284M Dec 18 13:35 _25z.frq
> -rw-rw-r-- 1 feeddo feeddo 311 Dec 18 13:35 segments_1y1
> -rw-rw-r-- 1 feeddo feeddo 23M Dec 18 13:35 _25z.nrm
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org