You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by "Danzig, Scott" <Sc...@nymag.com> on 2013/03/14 16:55:22 UTC
Handling a closed IndexWriter in SOLR 4.0
Hey all,
We're using a Solr 4 core to handle our article data. When someone in our CMS publishes an article, we have a listener that indexes it straight to solr. We use the previously instantiated HttpSolrServer, build the solr document, add it with server.add(doc) .. then do a server.commit() right away. For some reason, sometimes this exception is thrown, which I suspect is related to a simultaneous data import done from another client which sometimes errors:
Feb 26, 2013 5:07:51 PM org.apache.solr.common.SolrException log
SEVERE: null:org.apache.solr.common.SolrException: Error opening new searcher
at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1310)
at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1422)
at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1200)
at org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:560)
at org.apache.solr.update.processor.RunUpdateProcessor.processCommit(RunUpdateProcessorFactory.java:87)
at org.apache.solr.update.processor.UpdateRequestProcessor.processCommit(UpdateRequestProcessor.java:64)
at org.apache.solr.update.processor.DistributedUpdateProcessor.processCommit(DistributedUpdateProcessor.java:1007)
at org.apache.solr.update.processor.LogUpdateProcessor.processCommit(LogUpdateProcessorFactory.java:157)
at org.apache.solr.handler.RequestHandlerUtils.handleCommit(RequestHandlerUtils.java:69)
at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:68)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1699)
at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:455)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:276)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:225)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:169)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:168)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:98)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:927)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:999)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:565)
at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:309)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:679)
Caused by: org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:550)
at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:563)
at org.apache.lucene.index.IndexWriter.nrtIsCurrent(IndexWriter.java:4196)
at org.apache.lucene.index.StandardDirectoryReader.doOpenFromWriter(StandardDirectoryReader.java:266)
at org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:245)
at org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:235)
at org.apache.lucene.index.DirectoryReader.openIfChanged(DirectoryReader.java:169)
at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1256)
... 28 more
I'm not sure if the error is causing the IndexWriter to close, and why an IndexWriter would be shared across clients, but usually, I can get around this by basically creating a new HttpSolrServer and trying again. But it doesn't always work, perhaps due to frequency… I don't like the idea of an "infinite loop of creating connections until it works". I'd rather understand what's going on. What's the proper way to fix this? I see I can add a doc with a commitWithMs of "0" and maybe this couples the add tightly with the commit and would prevent interference. But am I totally off the mark here as to the problem? Suggestions?
Posted this on java-user before, but then realized solr-user existed, so please forgive the redundancy…
Thanks for reading!
- Scott
Re: Handling a closed IndexWriter in SOLR 4.0
Posted by Mark Miller <ma...@gmail.com>.
I'll fix it - I put up a patch last night.
- Mark
On Mar 18, 2013, at 1:12 AM, mark12345 <ma...@yahoo.com.au> wrote:
> This looks similar to the issue I also have:
>
> *
> http://lucene.472066.n3.nabble.com/Solr-4-1-4-2-SolrException-Error-opening-new-searcher-td4046543.html
> <http://lucene.472066.n3.nabble.com/Solr-4-1-4-2-SolrException-Error-opening-new-searcher-td4046543.html>
> *https://issues.apache.org/jira/browse/SOLR-4605
>
>
>
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/Handling-a-closed-IndexWriter-in-SOLR-4-0-tp4047392p4048421.html
> Sent from the Solr - User mailing list archive at Nabble.com.
Re: Handling a closed IndexWriter in SOLR 4.0
Posted by mark12345 <ma...@yahoo.com.au>.
This looks similar to the issue I also have:
*
http://lucene.472066.n3.nabble.com/Solr-4-1-4-2-SolrException-Error-opening-new-searcher-td4046543.html
<http://lucene.472066.n3.nabble.com/Solr-4-1-4-2-SolrException-Error-opening-new-searcher-td4046543.html>
*https://issues.apache.org/jira/browse/SOLR-4605
--
View this message in context: http://lucene.472066.n3.nabble.com/Handling-a-closed-IndexWriter-in-SOLR-4-0-tp4047392p4048421.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Handling a closed IndexWriter in SOLR 4.0
Posted by Otis Gospodnetic <ot...@gmail.com>.
Hi Scott,
Not sure why IW would be closed, but:
* consider not (hard) committing after each doc, but just periodically,
every N minutes
* soft committing instead
* using 4.2
Otis
--
Solr & ElasticSearch Support
http://sematext.com/
On Thu, Mar 14, 2013 at 11:55 AM, Danzig, Scott <Sc...@nymag.com>wrote:
> Hey all,
>
> We're using a Solr 4 core to handle our article data. When someone in our
> CMS publishes an article, we have a listener that indexes it straight to
> solr. We use the previously instantiated HttpSolrServer, build the solr
> document, add it with server.add(doc) .. then do a server.commit() right
> away. For some reason, sometimes this exception is thrown, which I suspect
> is related to a simultaneous data import done from another client which
> sometimes errors:
>
> Feb 26, 2013 5:07:51 PM org.apache.solr.common.SolrException log
> SEVERE: null:org.apache.solr.common.SolrException: Error opening new
> searcher
> at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1310)
> at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1422)
> at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1200)
> at
> org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:560)
> at
> org.apache.solr.update.processor.RunUpdateProcessor.processCommit(RunUpdateProcessorFactory.java:87)
> at
> org.apache.solr.update.processor.UpdateRequestProcessor.processCommit(UpdateRequestProcessor.java:64)
> at
> org.apache.solr.update.processor.DistributedUpdateProcessor.processCommit(DistributedUpdateProcessor.java:1007)
> at
> org.apache.solr.update.processor.LogUpdateProcessor.processCommit(LogUpdateProcessorFactory.java:157)
> at
> org.apache.solr.handler.RequestHandlerUtils.handleCommit(RequestHandlerUtils.java:69)
> at
> org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:68)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1699)
> at
> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:455)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:276)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
> at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:225)
> at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:169)
> at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:168)
> at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:98)
> at
> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:927)
> at
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
> at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
> at
> org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:999)
> at
> org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:565)
> at
> org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:309)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> Caused by: org.apache.lucene.store.AlreadyClosedException: this
> IndexWriter is closed
> at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:550)
> at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:563)
> at org.apache.lucene.index.IndexWriter.nrtIsCurrent(IndexWriter.java:4196)
> at
> org.apache.lucene.index.StandardDirectoryReader.doOpenFromWriter(StandardDirectoryReader.java:266)
> at
> org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:245)
> at
> org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:235)
> at
> org.apache.lucene.index.DirectoryReader.openIfChanged(DirectoryReader.java:169)
> at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1256)
> ... 28 more
>
> I'm not sure if the error is causing the IndexWriter to close, and why an
> IndexWriter would be shared across clients, but usually, I can get around
> this by basically creating a new HttpSolrServer and trying again. But it
> doesn't always work, perhaps due to frequency… I don't like the idea of an
> "infinite loop of creating connections until it works". I'd rather
> understand what's going on. What's the proper way to fix this? I see I
> can add a doc with a commitWithMs of "0" and maybe this couples the add
> tightly with the commit and would prevent interference. But am I totally
> off the mark here as to the problem? Suggestions?
>
> Posted this on java-user before, but then realized solr-user existed, so
> please forgive the redundancy…
>
> Thanks for reading!
>
> - Scott