You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Joe Kessel <is...@hotmail.com> on 2010/01/15 23:50:28 UTC
OverlappingFileLockException when using startup
I have an instance of Solr that won't start since I have added the replication <str name="replicateAfter">startup</str>. I am using Solr 1.4 and only see this with my index that contains 200k documents with a total size of 400MB. Removing the replicate after startup and the instance starts without error. We found that we needed replicate after startup as there was no version information on the master after restarting the instance. Is there something special that needs to be done when using replicate after startup? Or is this a bug?
below is the solr portion of the stacktrace.
Thanks,
Joe
INFO: QuerySenderListener sending requests to Searcher@5a425eb9 main
Jan 15, 2010 5:29:46 PM org.apache.solr.common.SolrException log
SEVERE: java.nio.channels.OverlappingFileLockException
at sun.nio.ch.FileChannelImpl$SharedFileLockTable.checkList(FileChannelImpl.java:1170)
at sun.nio.ch.FileChannelImpl$SharedFileLockTable.add(FileChannelImpl.java:1072)
at sun.nio.ch.FileChannelImpl.tryLock(FileChannelImpl.java:878)
at java.nio.channels.FileChannel.tryLock(FileChannel.java:962)
at org.apache.lucene.store.NativeFSLock.obtain(NativeFSLockFactory.java:233)
at org.apache.lucene.store.Lock.obtain(Lock.java:73)
at org.apache.lucene.index.IndexWriter.init(IndexWriter.java:1545)
at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:1402)
at org.apache.solr.update.SolrIndexWriter.<init>(SolrIndexWriter.java:190)
at org.apache.solr.update.UpdateHandler.createMainIndexWriter(UpdateHandler.java:98)
at org.apache.solr.update.DirectUpdateHandler2.openWriter(DirectUpdateHandler2.java:173)
at org.apache.solr.update.DirectUpdateHandler2.forceOpenWriter(DirectUpdateHandler2.java:376)
at org.apache.solr.handler.ReplicationHandler.inform(ReplicationHandler.java:845)
at org.apache.solr.core.SolrResourceLoader.inform(SolrResourceLoader.java:486)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:588)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:428)
at org.apache.solr.core.CoreContainer.load(CoreContainer.java:278)
at org.apache.solr.core.CoreContainer$Initializer.initialize(CoreContainer.java:117)
at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:83)
at org.apache.catalina.core.ApplicationFilterConfig.getFilter(ApplicationFilterConfig.java:275)
_________________________________________________________________
Hotmail: Powerful Free email with security by Microsoft.
http://clk.atdmt.com/GBL/go/196390710/direct/01/
Re: OverlappingFileLockException when using startup
Posted by rabahb <ra...@gmail.com>.
Hi Guys,
I'm experiencing the same issue with a single war. I'm using a brand new
Solr war built from yestertay's version of the trunk.
I've got one master with 2 cores and one slave with a single core. I'm using
one core from master as the master of the second core (which is configured
as a repeater). So that, the slave's core can poll the repeater for index
changes.
( I was using solr 1.4, but experienced some issues with replication. While
rebuilding the index on the one master core, the new index was not
replicated succesfully to the other master core. Files were copied over but
the final commit failed on the snappuller. But sometimes, while restarting
the master, the replication would work fine between master cores, then no
replication would be successful from master to slave core. I had the same
issue as described here: https://issues.apache.org/jira/browse/SOLR-1769 .
Which seems to be fixed in the trunk.
So I moved on to the trunk version of solr in order to tests the fix. This
seems to work better. As master cores replication works fine. But I've got a
weird behavior on slave. The index replication is successful only the second
time the slave is trying to get it even if for each replication trial, slave
spits out the following Exception (see below).
There seems to be a concurrrency issue but I don't quite undestand where the
concurrency is really happening. Can you please help on that issue?
org.apache.solr.common.SolrException: Index fetch failed :
at
org.apache.solr.handler.SnapPuller.fetchLatestIndex(SnapPuller.java:329)
at
org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:264)
at org.apache.solr.handler.SnapPuller$1.run(SnapPuller.java:159)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at
java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecu
tor.java:98)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExec
utor.java:181)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.jav
a:205)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.nio.channels.OverlappingFileLockException
at
sun.nio.ch.FileChannelImpl$SharedFileLockTable.checkList(FileChannelImpl.java:1170)
at
sun.nio.ch.FileChannelImpl$SharedFileLockTable.add(FileChannelImpl.java:1072)
at sun.nio.ch.FileChannelImpl.tryLock(FileChannelImpl.java:878)
at java.nio.channels.FileChannel.tryLock(FileChannel.java:962)
at
org.apache.lucene.store.NativeFSLock.obtain(NativeFSLockFactory.java:260)
at org.apache.lucene.store.Lock.obtain(Lock.java:72)
at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:1061)
at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:950)
at
org.apache.solr.update.SolrIndexWriter.<init>(SolrIndexWriter.java:192)
at
org.apache.solr.update.UpdateHandler.createMainIndexWriter(UpdateHandler.java:99)
at
org.apache.solr.update.DirectUpdateHandler2.openWriter(DirectUpdateHandler2.java:173)
at
org.apache.solr.update.DirectUpdateHandler2.forceOpenWriter(DirectUpdateHandler2.java:376)
at org.apache.solr.handler.SnapPuller.doCommit(SnapPuller.java:471)
at
org.apache.solr.handler.SnapPuller.fetchLatestIndex(SnapPuller.java:319)
... 11 more
--
View this message in context: http://lucene.472066.n3.nabble.com/OverlappingFileLockException-when-using-str-name-replicateAfter-startup-str-tp488686p870589.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: OverlappingFileLockException when using startup
Posted by Yonik Seeley <yo...@lucidimagination.com>.
On Sat, Jan 16, 2010 at 7:38 AM, Joe Kessel <is...@hotmail.com> wrote:
> There is no issue here, we had patched our solr to include SOLR-1595 and our webapps directory contained two wars. With only a single war file there is no issue with this replication handler.
Thanks Joe, for now I've added a note to the example solrconfig.xml
about "native" not working for multiple solr webapps in the same JVM.
-Yonik
http://www.lucidimagination.com
RE: OverlappingFileLockException when using startup
Posted by Joe Kessel <is...@hotmail.com>.
There is no issue here, we had patched our solr to include SOLR-1595 and our webapps directory contained two wars. With only a single war file there is no issue with this replication handler.
thanks for the quick response.
Joe
> From: isjustjoe@hotmail.com
> To: solr-user@lucene.apache.org
> Subject: RE: OverlappingFileLockException when using <str name="replicateAfter">startup</str>
> Date: Fri, 15 Jan 2010 18:36:13 -0600
>
>
> I am using the example solrconfig.xml with only a few changes. Mainly the replication section for the master has been changed. I am not using any plugins that I am aware of. Here is my replication section:
>
>
>
> <requestHandler name="/replication" class="solr.ReplicationHandler" >
> <lst name="master">
> <str name="replicateAfter">startup</str>
> <str name="replicateAfter">optimize</str>
> </lst>
> </requestHandler>
>
>
>
> If this is valid, then I will open a jira issue.
>
>
>
> Thanks,
>
> Joe
>
> > Date: Fri, 15 Jan 2010 19:06:15 -0500
> > Subject: Re: OverlappingFileLockException when using <str name="replicateAfter">startup</str>
> > From: yonik@lucidimagination.com
> > To: solr-user@lucene.apache.org
> >
> > Interesting... this should be impossible.
> > Unless there is a bug in Lucene's NativeFSLock (and it doesn't look
> > like it), the only way I see that this could happen is if there were
> > multiple instances of that class loaded in different classloaders.
> > Are you using any kind of plugins?
> >
> > Could you open a JIRA issue for this?
> >
> > -Yonik
> > http://www.lucidimagination.com
> >
> >
> >
> > On Fri, Jan 15, 2010 at 5:50 PM, Joe Kessel <is...@hotmail.com> wrote:
> > >
> > > I have an instance of Solr that won't start since I have added the replication <str name="replicateAfter">startup</str>. I am using Solr 1.4 and only see this with my index that contains 200k documents with a total size of 400MB. Removing the replicate after startup and the instance starts without error. We found that we needed replicate after startup as there was no version information on the master after restarting the instance. Is there something special that needs to be done when using replicate after startup? Or is this a bug?
> > >
> > >
> > >
> > > below is the solr portion of the stacktrace.
> > >
> > >
> > >
> > > Thanks,
> > >
> > > Joe
> > >
> > >
> > >
> > > INFO: QuerySenderListener sending requests to Searcher@5a425eb9 main
> > > Jan 15, 2010 5:29:46 PM org.apache.solr.common.SolrException log
> > > SEVERE: java.nio.channels.OverlappingFileLockException
> > > at sun.nio.ch.FileChannelImpl$SharedFileLockTable.checkList(FileChannelImpl.java:1170)
> > > at sun.nio.ch.FileChannelImpl$SharedFileLockTable.add(FileChannelImpl.java:1072)
> > > at sun.nio.ch.FileChannelImpl.tryLock(FileChannelImpl.java:878)
> > > at java.nio.channels.FileChannel.tryLock(FileChannel.java:962)
> > > at org.apache.lucene.store.NativeFSLock.obtain(NativeFSLockFactory.java:233)
> > > at org.apache.lucene.store.Lock.obtain(Lock.java:73)
> > > at org.apache.lucene.index.IndexWriter.init(IndexWriter.java:1545)
> > > at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:1402)
> > > at org.apache.solr.update.SolrIndexWriter.<init>(SolrIndexWriter.java:190)
> > > at org.apache.solr.update.UpdateHandler.createMainIndexWriter(UpdateHandler.java:98)
> > > at org.apache.solr.update.DirectUpdateHandler2.openWriter(DirectUpdateHandler2.java:173)
> > > at org.apache.solr.update.DirectUpdateHandler2.forceOpenWriter(DirectUpdateHandler2.java:376)
> > > at org.apache.solr.handler.ReplicationHandler.inform(ReplicationHandler.java:845)
> > > at org.apache.solr.core.SolrResourceLoader.inform(SolrResourceLoader.java:486)
> > > at org.apache.solr.core.SolrCore.<init>(SolrCore.java:588)
> > > at org.apache.solr.core.CoreContainer.create(CoreContainer.java:428)
> > > at org.apache.solr.core.CoreContainer.load(CoreContainer.java:278)
> > > at org.apache.solr.core.CoreContainer$Initializer.initialize(CoreContainer.java:117)
> > > at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:83)
> > > at org.apache.catalina.core.ApplicationFilterConfig.getFilter(ApplicationFilterConfig.java:275)
> > >
> > > _________________________________________________________________
> > > Hotmail: Powerful Free email with security by Microsoft.
> > > http://clk.atdmt.com/GBL/go/196390710/direct/01/
>
> _________________________________________________________________
> Your E-mail and More On-the-Go. Get Windows Live Hotmail Free.
> http://clk.atdmt.com/GBL/go/196390709/direct/01/
_________________________________________________________________
Hotmail: Trusted email with powerful SPAM protection.
http://clk.atdmt.com/GBL/go/196390707/direct/01/
RE: OverlappingFileLockException when using startup
Posted by Joe Kessel <is...@hotmail.com>.
I am using the example solrconfig.xml with only a few changes. Mainly the replication section for the master has been changed. I am not using any plugins that I am aware of. Here is my replication section:
<requestHandler name="/replication" class="solr.ReplicationHandler" >
<lst name="master">
<str name="replicateAfter">startup</str>
<str name="replicateAfter">optimize</str>
</lst>
</requestHandler>
If this is valid, then I will open a jira issue.
Thanks,
Joe
> Date: Fri, 15 Jan 2010 19:06:15 -0500
> Subject: Re: OverlappingFileLockException when using <str name="replicateAfter">startup</str>
> From: yonik@lucidimagination.com
> To: solr-user@lucene.apache.org
>
> Interesting... this should be impossible.
> Unless there is a bug in Lucene's NativeFSLock (and it doesn't look
> like it), the only way I see that this could happen is if there were
> multiple instances of that class loaded in different classloaders.
> Are you using any kind of plugins?
>
> Could you open a JIRA issue for this?
>
> -Yonik
> http://www.lucidimagination.com
>
>
>
> On Fri, Jan 15, 2010 at 5:50 PM, Joe Kessel <is...@hotmail.com> wrote:
> >
> > I have an instance of Solr that won't start since I have added the replication <str name="replicateAfter">startup</str>. I am using Solr 1.4 and only see this with my index that contains 200k documents with a total size of 400MB. Removing the replicate after startup and the instance starts without error. We found that we needed replicate after startup as there was no version information on the master after restarting the instance. Is there something special that needs to be done when using replicate after startup? Or is this a bug?
> >
> >
> >
> > below is the solr portion of the stacktrace.
> >
> >
> >
> > Thanks,
> >
> > Joe
> >
> >
> >
> > INFO: QuerySenderListener sending requests to Searcher@5a425eb9 main
> > Jan 15, 2010 5:29:46 PM org.apache.solr.common.SolrException log
> > SEVERE: java.nio.channels.OverlappingFileLockException
> > at sun.nio.ch.FileChannelImpl$SharedFileLockTable.checkList(FileChannelImpl.java:1170)
> > at sun.nio.ch.FileChannelImpl$SharedFileLockTable.add(FileChannelImpl.java:1072)
> > at sun.nio.ch.FileChannelImpl.tryLock(FileChannelImpl.java:878)
> > at java.nio.channels.FileChannel.tryLock(FileChannel.java:962)
> > at org.apache.lucene.store.NativeFSLock.obtain(NativeFSLockFactory.java:233)
> > at org.apache.lucene.store.Lock.obtain(Lock.java:73)
> > at org.apache.lucene.index.IndexWriter.init(IndexWriter.java:1545)
> > at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:1402)
> > at org.apache.solr.update.SolrIndexWriter.<init>(SolrIndexWriter.java:190)
> > at org.apache.solr.update.UpdateHandler.createMainIndexWriter(UpdateHandler.java:98)
> > at org.apache.solr.update.DirectUpdateHandler2.openWriter(DirectUpdateHandler2.java:173)
> > at org.apache.solr.update.DirectUpdateHandler2.forceOpenWriter(DirectUpdateHandler2.java:376)
> > at org.apache.solr.handler.ReplicationHandler.inform(ReplicationHandler.java:845)
> > at org.apache.solr.core.SolrResourceLoader.inform(SolrResourceLoader.java:486)
> > at org.apache.solr.core.SolrCore.<init>(SolrCore.java:588)
> > at org.apache.solr.core.CoreContainer.create(CoreContainer.java:428)
> > at org.apache.solr.core.CoreContainer.load(CoreContainer.java:278)
> > at org.apache.solr.core.CoreContainer$Initializer.initialize(CoreContainer.java:117)
> > at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:83)
> > at org.apache.catalina.core.ApplicationFilterConfig.getFilter(ApplicationFilterConfig.java:275)
> >
> > _________________________________________________________________
> > Hotmail: Powerful Free email with security by Microsoft.
> > http://clk.atdmt.com/GBL/go/196390710/direct/01/
_________________________________________________________________
Your E-mail and More On-the-Go. Get Windows Live Hotmail Free.
http://clk.atdmt.com/GBL/go/196390709/direct/01/
Re: OverlappingFileLockException when using startup
Posted by Yonik Seeley <yo...@lucidimagination.com>.
Interesting... this should be impossible.
Unless there is a bug in Lucene's NativeFSLock (and it doesn't look
like it), the only way I see that this could happen is if there were
multiple instances of that class loaded in different classloaders.
Are you using any kind of plugins?
Could you open a JIRA issue for this?
-Yonik
http://www.lucidimagination.com
On Fri, Jan 15, 2010 at 5:50 PM, Joe Kessel <is...@hotmail.com> wrote:
>
> I have an instance of Solr that won't start since I have added the replication <str name="replicateAfter">startup</str>. I am using Solr 1.4 and only see this with my index that contains 200k documents with a total size of 400MB. Removing the replicate after startup and the instance starts without error. We found that we needed replicate after startup as there was no version information on the master after restarting the instance. Is there something special that needs to be done when using replicate after startup? Or is this a bug?
>
>
>
> below is the solr portion of the stacktrace.
>
>
>
> Thanks,
>
> Joe
>
>
>
> INFO: QuerySenderListener sending requests to Searcher@5a425eb9 main
> Jan 15, 2010 5:29:46 PM org.apache.solr.common.SolrException log
> SEVERE: java.nio.channels.OverlappingFileLockException
> at sun.nio.ch.FileChannelImpl$SharedFileLockTable.checkList(FileChannelImpl.java:1170)
> at sun.nio.ch.FileChannelImpl$SharedFileLockTable.add(FileChannelImpl.java:1072)
> at sun.nio.ch.FileChannelImpl.tryLock(FileChannelImpl.java:878)
> at java.nio.channels.FileChannel.tryLock(FileChannel.java:962)
> at org.apache.lucene.store.NativeFSLock.obtain(NativeFSLockFactory.java:233)
> at org.apache.lucene.store.Lock.obtain(Lock.java:73)
> at org.apache.lucene.index.IndexWriter.init(IndexWriter.java:1545)
> at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:1402)
> at org.apache.solr.update.SolrIndexWriter.<init>(SolrIndexWriter.java:190)
> at org.apache.solr.update.UpdateHandler.createMainIndexWriter(UpdateHandler.java:98)
> at org.apache.solr.update.DirectUpdateHandler2.openWriter(DirectUpdateHandler2.java:173)
> at org.apache.solr.update.DirectUpdateHandler2.forceOpenWriter(DirectUpdateHandler2.java:376)
> at org.apache.solr.handler.ReplicationHandler.inform(ReplicationHandler.java:845)
> at org.apache.solr.core.SolrResourceLoader.inform(SolrResourceLoader.java:486)
> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:588)
> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:428)
> at org.apache.solr.core.CoreContainer.load(CoreContainer.java:278)
> at org.apache.solr.core.CoreContainer$Initializer.initialize(CoreContainer.java:117)
> at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:83)
> at org.apache.catalina.core.ApplicationFilterConfig.getFilter(ApplicationFilterConfig.java:275)
>
> _________________________________________________________________
> Hotmail: Powerful Free email with security by Microsoft.
> http://clk.atdmt.com/GBL/go/196390710/direct/01/