You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by legrand thomas <th...@yahoo.fr> on 2010/01/08 11:35:11 UTC
Concurrent access IndexReader / IndexWriter - FileNotFoundException
Hi,
I often get a FileNotFoundException when my single IndexWriter commits while the IndexReader also tries to read. My application is multithreaded (Tomcat uses the business APIs); I firstly thought the read/write access was thread-safe but I probably forget something.
Please help me to understand my mistakes:
- When should I close the IndexWriter ? Each time I add/update a document or never ?
- Should my java commit/read methods be synchronized ?
- Should I lock the directory and what's the best way to do it ?
I referred to the Lucene FAQ ("why do I sometimes get a FileNotFoundException when I search and update my index at the same time?") but I did not disable any locking for processes searching or updating the index.
I use lucene-2.4.1.
Thanks in advance,
Tom
Re: Concurrent access IndexReader / IndexWriter - FileNotFoundException
Posted by legrand thomas <th...@yahoo.fr>.
Michael,
The exception only occurs when the writer commits. But but the IndexReader can keep on reading.
The searches are performed by the IndexSearcher using the IndexReader.
My filesystem is ext2fs.
I give a few details below about the way I use them; the FNF exception occurs in the "commitIndexWriter" method.
Thanks,
Tom
// both reader and writer and static
protected static IndexWriter mAdIndexWriter=null;
protected static IndexReader mAdIndexReader=null;
// misc
protected static IndexSearcher mAdIndexSearcher=null;
protected static Analyzer mAdIndexAnalyser=null;
// create the writer once
private void initIndexWriter() throws EncheromaxException {
Directory adIndexDir=null;
// Where is the index ?
String indexLocation=mConfigProperties.getProperty(PropertiesEnum.ADVERTISEMENT_INDEX_LOCATION);
// Get the ad's analyser
Analyzer analyser = getAdvertisementIndexAnalyser();
// create the writer
if(mAdIndexWriter==null){
try{
adIndexDir=FSDirectory.getDirectory(indexLocation);
boolean isTheIndexNew=true;
mAdIndexWriter = new IndexWriter(adIndexDir, analyser,isTheIndexNew,IndexWriter.MaxFieldLength. UNLIMITED) ;
/* ... */
}catch(Exception ex){
/*...*/
}
}
}
// get the reader
private IndexReader getAdvertisementIndexReader() throws EncheromaxException {
String indexLocation=null;
Directory adIndexDir=null;
// Where is the index ?
try{
indexLocation=mConfigProperties.getProperty(PropertiesEnum.ADVERTISEMENT_INDEX_LOCATION);
adIndexDir=FSDirectory.getDirectory(indexLocation);
}catch(Exception direx){
/*...*/
}
// create the reader if it doesn't exist yet
if(mAdIndexReader==null){
try{
mAdIndexReader= IndexReader.open(adIndexDir);
}catch(Exception ex){
/*..*/
}
}else{
try{
/* do we need to re-open it ? */
if(mAdIndexReader.isCurrent()==false){
// synchronize on the reader
synchronized(mAdIndexReader){
mAdIndexReader= IndexReader.open(adIndexDir);
}
}
}catch(Exception ioex){
/*...*/
throw new EncheromaxException(CommonErrorEnum.INDEX_READER_CREATION_ERROR);
}
}
return mAdIndexReader;
}
// get the searcher when I want to search
private IndexSearcher getAdvertisementIndexSearcher() throws EncheromaxException {
// create the searcher if it doesn't exist yet
try{
if(mAdIndexSearcher==null){
try{
mAdIndexSearcher= new IndexSearcher(getAdvertisementIndexReader());
}catch(Exception ex){
/*...*/
}
}
else if(!mAdIndexSearcher.getIndexReader().isCurrent()){
synchronized(mAdIndexSearcher){
mAdIndexSearcher=new IndexSearcher(getAdvertisementIndexReader());
}
}
}catch(Exception ioex){
/*...*/
}
return mAdIndexSearcher;
}
// whenever I need to commit
public synchronized void commitIndexWriter() throws EncheromaxException{
try{
mAdIndexWriter.commit();
}catch(Exception docex){
/* THE FILE NOT FOUND EXCEPTION OCCURS HERE */
}
}
// add a document
private void addADocument(Document adDoc) throws EncheromaxException{
try{
mAdIndexWriter.addDocument(adDoc, getAdvertisementIndexAnalyser());
commitIndexWriter();
}catch(Exception docex){
/*...*/
}
}
--- En date de : Ven 8.1.10, Michael McCandless <lu...@mikemccandless.com> a écrit :
De: Michael McCandless <lu...@mikemccandless.com>
Objet: Re: Concurrent access IndexReader / IndexWriter - FileNotFoundException
À: java-user@lucene.apache.org
Date: Vendredi 8 Janvier 2010, 13h00
Normally, this (using an IndexReader, [re-]opening a new IndexReader
while an IndexWriter is committing) is perfectly fine. The reader
searches the point-in-time snapshot of the index as of when it was
opened.
But: what filesystem are you using? NFS presents challenges, for example.
Mike
On Fri, Jan 8, 2010 at 5:35 AM, legrand thomas <th...@yahoo.fr> wrote:
> Hi,
>
> I often get a FileNotFoundException when my single IndexWriter commits while the IndexReader also tries to read. My application is multithreaded (Tomcat uses the business APIs); I firstly thought the read/write access was thread-safe but I probably forget something.
>
> Please help me to understand my mistakes:
>
> - When should I close the IndexWriter ? Each time I add/update a document or never ?
> - Should my java commit/read methods be synchronized ?
> - Should I lock the directory and what's the best way to do it ?
>
> I referred to the Lucene FAQ ("why do I sometimes get a FileNotFoundException when I search and update my index at the same time?") but I did not disable any locking for processes searching or updating the index.
>
> I use lucene-2.4.1.
>
> Thanks in advance,
> Tom
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Concurrent access IndexReader / IndexWriter -
FileNotFoundException
Posted by Michael McCandless <lu...@mikemccandless.com>.
Normally, this (using an IndexReader, [re-]opening a new IndexReader
while an IndexWriter is committing) is perfectly fine. The reader
searches the point-in-time snapshot of the index as of when it was
opened.
But: what filesystem are you using? NFS presents challenges, for example.
Mike
On Fri, Jan 8, 2010 at 5:35 AM, legrand thomas <th...@yahoo.fr> wrote:
> Hi,
>
> I often get a FileNotFoundException when my single IndexWriter commits while the IndexReader also tries to read. My application is multithreaded (Tomcat uses the business APIs); I firstly thought the read/write access was thread-safe but I probably forget something.
>
> Please help me to understand my mistakes:
>
> - When should I close the IndexWriter ? Each time I add/update a document or never ?
> - Should my java commit/read methods be synchronized ?
> - Should I lock the directory and what's the best way to do it ?
>
> I referred to the Lucene FAQ ("why do I sometimes get a FileNotFoundException when I search and update my index at the same time?") but I did not disable any locking for processes searching or updating the index.
>
> I use lucene-2.4.1.
>
> Thanks in advance,
> Tom
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Concurrent access IndexReader / IndexWriter - FileNotFoundException
Posted by legrand thomas <th...@yahoo.fr>.
Ok Michael,
Thank you for your answers, I'll check the numbers of writers as no other process uses the directory. I'll let you know the way I make it work.
Tom
--- En date de : Sam 9.1.10, Michael McCandless <lu...@mikemccandless.com> a écrit :
De: Michael McCandless <lu...@mikemccandless.com>
Objet: Re: Concurrent access IndexReader / IndexWriter - FileNotFoundException
À: java-user@lucene.apache.org
Date: Samedi 9 Janvier 2010, 18h54
Can you double check that you're not creating 2 writers on the same
directory, somehow?
Or: is there any other process that removes files from this directory?
Answering your original questions...: commit/read does not require any
external synchronization or locking. You should generally keep your
IW open indefinitely and just periodically commit and/or get a new
reader (IndexWriter.getReader()) as needed.
Mike
On Sat, Jan 9, 2010 at 10:06 AM, legrand thomas
<th...@yahoo.fr> wrote:
>
> Here are two stack traces: add+remove a document:
>
> Tom
>
> ---
>
> Remove a document:
>
> java.io.FileNotFoundException: /home/ia/prod/current-deployment/indexes/advertisement/_0.cfs (No such file or directory)
> at java.io.RandomAccessFile.open(Native Method)
> at java.io.RandomAccessFile.<init>(RandomAccessFile.java:212)
> at org.apache.lucene.store.FSDirectory$FSIndexInput$Descriptor.<init>(FSDirectory.java:552)
> at org.apache.lucene.store.FSDirectory$FSIndexInput.<init>(FSDirectory.java:582)
> at org.apache.lucene.store.FSDirectory.openInput(FSDirectory.java:488)
> at org.apache.lucene.index.CompoundFileReader.<init>(CompoundFileReader.java:70)
> at org.apache.lucene.index.SegmentReader.initialize(SegmentReader.java:321)
> at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:306)
> at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:236)
> at org.apache.lucene.index.DocumentsWriter.applyDeletes(DocumentsWriter.java:915)
> at org.apache.lucene.index.IndexWriter.applyDeletes(IndexWriter.java:4339)
> at org.apache.lucene.index.IndexWriter.doFlush(IndexWriter.java:3579)
> at org.apache.lucene.index.IndexWriter.flush(IndexWriter.java:3450)
> at org.apache.lucene.index.IndexWriter.prepareCommit(IndexWriter.java:3363)
> at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:3408)
> at com.encheromax.business.advertisement.implementation.AdvertisementManager.commitIndexWriter(AdvertisementManager.java:1728)
> at com.encheromax.business.advertisement.implementation.AdvertisementManager.removeAdvertisementFromIndex(AdvertisementManager.java:635)
> at com.encheromax.api.implementation.admin.ApiAdminBurnAdvertisement.executeRequest(ApiAdminBurnAdvertisement.java:210)
> at com.encheromax.controller.admin.Rule.doReqFinal(Rule.java:100)
> at net.tmp.fwk.fw1.controller.Action.process(Action.java:302)
> at net.tmp.fwk.fw1.controller.Action.doPost(Action.java:87)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:710)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
> at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:269)
> at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
> at com.encheromax.servlet.filter.EncheromaxBeanFilter.doFilter(EncheromaxBeanFilter.java:53)
> at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
> at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
> at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
> at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:174)
> at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
> at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117)
> at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108)
> at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:174)
> at org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:200)
> at org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:283)
> at org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:773)
> at org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:703)
> at org.apache.jk.common.ChannelSocket$SocketConnection.runIt(ChannelSocket.java:895)
> at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:689)
>
> Add a document:
>
> java.io.FileNotFoundException: /home/ia/prod/current-deployment/indexes/advertisement/_v.cfs (No such file or directory)
> at java.io.RandomAccessFile.open(Native Method)
> at java.io.RandomAccessFile.<init>(RandomAccessFile.java:212)
> at org.apache.lucene.store.FSDirectory$FSIndexInput$Descriptor.<init>(FSDirectory.java:552)
> at org.apache.lucene.store.FSDirectory$FSIndexInput.<init>(FSDirectory.java:582)
> at org.apache.lucene.store.FSDirectory.openInput(FSDirectory.java:488)
> at org.apache.lucene.index.CompoundFileReader.<init>(CompoundFileReader.java:70)
> at org.apache.lucene.index.SegmentReader.initialize(SegmentReader.java:321)
> at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:306)
> at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:236)
> at org.apache.lucene.index.DocumentsWriter.applyDeletes(DocumentsWriter.java:915)
> at org.apache.lucene.index.IndexWriter.applyDeletes(IndexWriter.java:4339)
> at org.apache.lucene.index.IndexWriter.doFlush(IndexWriter.java:3579)
> at org.apache.lucene.index.IndexWriter.flush(IndexWriter.java:3450)
> at org.apache.lucene.index.IndexWriter.prepareCommit(IndexWriter.java:3363)
> at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:3408)
> at com.encheromax.business.advertisement.implementation.AdvertisementManager.commitIndexWriter(AdvertisementManager.java:1728)
> at com.encheromax.business.advertisement.implementation.AdvertisementManager.addAdvertisementDocument(AdvertisementManager.java:536)
> at com.encheromax.business.advertisement.implementation.AdvertisementManager.appendFullAdvertisementToIndex(AdvertisementManager.java:713)
> at com.encheromax.api.implementation.ApiFinalizeAdvertisement.executeRequest(ApiFinalizeAdvertisement.java:224)
> at com.encheromax.controller.advertisement.CreateAdvertisement.doReq(CreateAdvertisement.java:294)
> at net.tmp.fwk.fw1.controller.Security.doReqFinal(Security.java:135)
> at net.tmp.fwk.fw1.controller.Action.process(Action.java:302)
> at net.tmp.fwk.fw1.controller.Action.doPost(Action.java:87)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:710)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
> at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:269)
> at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
> at com.encheromax.servlet.filter.EncheromaxBeanFilter.doFilter(EncheromaxBeanFilter.java:53)
> at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
> at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
> at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
> at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:174)
> at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
> at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117)
> at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108)
> at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:174)
> at org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:200)
> at org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:283)
> at org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:773)
> at org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:703)
> at org.apache.jk.common.ChannelSocket$SocketConnection.runIt(ChannelSocket.java:895)
> at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:689)
> at java.lang.Thread.run(Thread.java:619)
>
>
>
>
>
>
> --- En date de : Sam 9.1.10, Michael McCandless <lu...@mikemccandless.com> a écrit :
>
> De: Michael McCandless <lu...@mikemccandless.com>
> Objet: Re: Concurrent access IndexReader / IndexWriter - FileNotFoundException
> À: java-user@lucene.apache.org
> Date: Samedi 9 Janvier 2010, 14h51
>
> Can you post the full FNFE stack trace?
>
> Mike
>
> On Fri, Jan 8, 2010 at 5:35 AM, legrand thomas <th...@yahoo.fr> wrote:
>> Hi,
>>
>> I often get a FileNotFoundException when my single IndexWriter commits while the IndexReader also tries to read. My application is multithreaded (Tomcat uses the business APIs); I firstly thought the read/write access was thread-safe but I probably forget something.
>>
>> Please help me to understand my mistakes:
>>
>> - When should I close the IndexWriter ? Each time I add/update a document or never ?
>> - Should my java commit/read methods be synchronized ?
>> - Should I lock the directory and what's the best way to do it ?
>>
>> I referred to the Lucene FAQ ("why do I sometimes get a FileNotFoundException when I search and update my index at the same time?") but I did not disable any locking for processes searching or updating the index.
>>
>> I use lucene-2.4.1.
>>
>> Thanks in advance,
>> Tom
>>
>>
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Concurrent access IndexReader / IndexWriter -
FileNotFoundException
Posted by Michael McCandless <lu...@mikemccandless.com>.
Can you double check that you're not creating 2 writers on the same
directory, somehow?
Or: is there any other process that removes files from this directory?
Answering your original questions...: commit/read does not require any
external synchronization or locking. You should generally keep your
IW open indefinitely and just periodically commit and/or get a new
reader (IndexWriter.getReader()) as needed.
Mike
On Sat, Jan 9, 2010 at 10:06 AM, legrand thomas
<th...@yahoo.fr> wrote:
>
> Here are two stack traces: add+remove a document:
>
> Tom
>
> ---
>
> Remove a document:
>
> java.io.FileNotFoundException: /home/ia/prod/current-deployment/indexes/advertisement/_0.cfs (No such file or directory)
> at java.io.RandomAccessFile.open(Native Method)
> at java.io.RandomAccessFile.<init>(RandomAccessFile.java:212)
> at org.apache.lucene.store.FSDirectory$FSIndexInput$Descriptor.<init>(FSDirectory.java:552)
> at org.apache.lucene.store.FSDirectory$FSIndexInput.<init>(FSDirectory.java:582)
> at org.apache.lucene.store.FSDirectory.openInput(FSDirectory.java:488)
> at org.apache.lucene.index.CompoundFileReader.<init>(CompoundFileReader.java:70)
> at org.apache.lucene.index.SegmentReader.initialize(SegmentReader.java:321)
> at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:306)
> at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:236)
> at org.apache.lucene.index.DocumentsWriter.applyDeletes(DocumentsWriter.java:915)
> at org.apache.lucene.index.IndexWriter.applyDeletes(IndexWriter.java:4339)
> at org.apache.lucene.index.IndexWriter.doFlush(IndexWriter.java:3579)
> at org.apache.lucene.index.IndexWriter.flush(IndexWriter.java:3450)
> at org.apache.lucene.index.IndexWriter.prepareCommit(IndexWriter.java:3363)
> at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:3408)
> at com.encheromax.business.advertisement.implementation.AdvertisementManager.commitIndexWriter(AdvertisementManager.java:1728)
> at com.encheromax.business.advertisement.implementation.AdvertisementManager.removeAdvertisementFromIndex(AdvertisementManager.java:635)
> at com.encheromax.api.implementation.admin.ApiAdminBurnAdvertisement.executeRequest(ApiAdminBurnAdvertisement.java:210)
> at com.encheromax.controller.admin.Rule.doReqFinal(Rule.java:100)
> at net.tmp.fwk.fw1.controller.Action.process(Action.java:302)
> at net.tmp.fwk.fw1.controller.Action.doPost(Action.java:87)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:710)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
> at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:269)
> at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
> at com.encheromax.servlet.filter.EncheromaxBeanFilter.doFilter(EncheromaxBeanFilter.java:53)
> at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
> at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
> at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
> at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:174)
> at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
> at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117)
> at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108)
> at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:174)
> at org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:200)
> at org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:283)
> at org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:773)
> at org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:703)
> at org.apache.jk.common.ChannelSocket$SocketConnection.runIt(ChannelSocket.java:895)
> at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:689)
>
> Add a document:
>
> java.io.FileNotFoundException: /home/ia/prod/current-deployment/indexes/advertisement/_v.cfs (No such file or directory)
> at java.io.RandomAccessFile.open(Native Method)
> at java.io.RandomAccessFile.<init>(RandomAccessFile.java:212)
> at org.apache.lucene.store.FSDirectory$FSIndexInput$Descriptor.<init>(FSDirectory.java:552)
> at org.apache.lucene.store.FSDirectory$FSIndexInput.<init>(FSDirectory.java:582)
> at org.apache.lucene.store.FSDirectory.openInput(FSDirectory.java:488)
> at org.apache.lucene.index.CompoundFileReader.<init>(CompoundFileReader.java:70)
> at org.apache.lucene.index.SegmentReader.initialize(SegmentReader.java:321)
> at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:306)
> at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:236)
> at org.apache.lucene.index.DocumentsWriter.applyDeletes(DocumentsWriter.java:915)
> at org.apache.lucene.index.IndexWriter.applyDeletes(IndexWriter.java:4339)
> at org.apache.lucene.index.IndexWriter.doFlush(IndexWriter.java:3579)
> at org.apache.lucene.index.IndexWriter.flush(IndexWriter.java:3450)
> at org.apache.lucene.index.IndexWriter.prepareCommit(IndexWriter.java:3363)
> at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:3408)
> at com.encheromax.business.advertisement.implementation.AdvertisementManager.commitIndexWriter(AdvertisementManager.java:1728)
> at com.encheromax.business.advertisement.implementation.AdvertisementManager.addAdvertisementDocument(AdvertisementManager.java:536)
> at com.encheromax.business.advertisement.implementation.AdvertisementManager.appendFullAdvertisementToIndex(AdvertisementManager.java:713)
> at com.encheromax.api.implementation.ApiFinalizeAdvertisement.executeRequest(ApiFinalizeAdvertisement.java:224)
> at com.encheromax.controller.advertisement.CreateAdvertisement.doReq(CreateAdvertisement.java:294)
> at net.tmp.fwk.fw1.controller.Security.doReqFinal(Security.java:135)
> at net.tmp.fwk.fw1.controller.Action.process(Action.java:302)
> at net.tmp.fwk.fw1.controller.Action.doPost(Action.java:87)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:710)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
> at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:269)
> at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
> at com.encheromax.servlet.filter.EncheromaxBeanFilter.doFilter(EncheromaxBeanFilter.java:53)
> at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
> at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
> at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
> at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:174)
> at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
> at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117)
> at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108)
> at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:174)
> at org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:200)
> at org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:283)
> at org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:773)
> at org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:703)
> at org.apache.jk.common.ChannelSocket$SocketConnection.runIt(ChannelSocket.java:895)
> at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:689)
> at java.lang.Thread.run(Thread.java:619)
>
>
>
>
>
>
> --- En date de : Sam 9.1.10, Michael McCandless <lu...@mikemccandless.com> a écrit :
>
> De: Michael McCandless <lu...@mikemccandless.com>
> Objet: Re: Concurrent access IndexReader / IndexWriter - FileNotFoundException
> À: java-user@lucene.apache.org
> Date: Samedi 9 Janvier 2010, 14h51
>
> Can you post the full FNFE stack trace?
>
> Mike
>
> On Fri, Jan 8, 2010 at 5:35 AM, legrand thomas <th...@yahoo.fr> wrote:
>> Hi,
>>
>> I often get a FileNotFoundException when my single IndexWriter commits while the IndexReader also tries to read. My application is multithreaded (Tomcat uses the business APIs); I firstly thought the read/write access was thread-safe but I probably forget something.
>>
>> Please help me to understand my mistakes:
>>
>> - When should I close the IndexWriter ? Each time I add/update a document or never ?
>> - Should my java commit/read methods be synchronized ?
>> - Should I lock the directory and what's the best way to do it ?
>>
>> I referred to the Lucene FAQ ("why do I sometimes get a FileNotFoundException when I search and update my index at the same time?") but I did not disable any locking for processes searching or updating the index.
>>
>> I use lucene-2.4.1.
>>
>> Thanks in advance,
>> Tom
>>
>>
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Concurrent access IndexReader / IndexWriter - FileNotFoundException
Posted by legrand thomas <th...@yahoo.fr>.
Here are two stack traces: add+remove a document:
Tom
---
Remove a document:
java.io.FileNotFoundException: /home/ia/prod/current-deployment/indexes/advertisement/_0.cfs (No such file or directory)
at java.io.RandomAccessFile.open(Native Method)
at java.io.RandomAccessFile.<init>(RandomAccessFile.java:212)
at org.apache.lucene.store.FSDirectory$FSIndexInput$Descriptor.<init>(FSDirectory.java:552)
at org.apache.lucene.store.FSDirectory$FSIndexInput.<init>(FSDirectory.java:582)
at org.apache.lucene.store.FSDirectory.openInput(FSDirectory.java:488)
at org.apache.lucene.index.CompoundFileReader.<init>(CompoundFileReader.java:70)
at org.apache.lucene.index.SegmentReader.initialize(SegmentReader.java:321)
at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:306)
at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:236)
at org.apache.lucene.index.DocumentsWriter.applyDeletes(DocumentsWriter.java:915)
at org.apache.lucene.index.IndexWriter.applyDeletes(IndexWriter.java:4339)
at org.apache.lucene.index.IndexWriter.doFlush(IndexWriter.java:3579)
at org.apache.lucene.index.IndexWriter.flush(IndexWriter.java:3450)
at org.apache.lucene.index.IndexWriter.prepareCommit(IndexWriter.java:3363)
at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:3408)
at com.encheromax.business.advertisement.implementation.AdvertisementManager.commitIndexWriter(AdvertisementManager.java:1728)
at com.encheromax.business.advertisement.implementation.AdvertisementManager.removeAdvertisementFromIndex(AdvertisementManager.java:635)
at com.encheromax.api.implementation.admin.ApiAdminBurnAdvertisement.executeRequest(ApiAdminBurnAdvertisement.java:210)
at com.encheromax.controller.admin.Rule.doReqFinal(Rule.java:100)
at net.tmp.fwk.fw1.controller.Action.process(Action.java:302)
at net.tmp.fwk.fw1.controller.Action.doPost(Action.java:87)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:710)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:269)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
at com.encheromax.servlet.filter.EncheromaxBeanFilter.doFilter(EncheromaxBeanFilter.java:53)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:174)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:174)
at org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:200)
at org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:283)
at org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:773)
at org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:703)
at org.apache.jk.common.ChannelSocket$SocketConnection.runIt(ChannelSocket.java:895)
at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:689)
Add a document:
java.io.FileNotFoundException: /home/ia/prod/current-deployment/indexes/advertisement/_v.cfs (No such file or directory)
at java.io.RandomAccessFile.open(Native Method)
at java.io.RandomAccessFile.<init>(RandomAccessFile.java:212)
at org.apache.lucene.store.FSDirectory$FSIndexInput$Descriptor.<init>(FSDirectory.java:552)
at org.apache.lucene.store.FSDirectory$FSIndexInput.<init>(FSDirectory.java:582)
at org.apache.lucene.store.FSDirectory.openInput(FSDirectory.java:488)
at org.apache.lucene.index.CompoundFileReader.<init>(CompoundFileReader.java:70)
at org.apache.lucene.index.SegmentReader.initialize(SegmentReader.java:321)
at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:306)
at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:236)
at org.apache.lucene.index.DocumentsWriter.applyDeletes(DocumentsWriter.java:915)
at org.apache.lucene.index.IndexWriter.applyDeletes(IndexWriter.java:4339)
at org.apache.lucene.index.IndexWriter.doFlush(IndexWriter.java:3579)
at org.apache.lucene.index.IndexWriter.flush(IndexWriter.java:3450)
at org.apache.lucene.index.IndexWriter.prepareCommit(IndexWriter.java:3363)
at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:3408)
at com.encheromax.business.advertisement.implementation.AdvertisementManager.commitIndexWriter(AdvertisementManager.java:1728)
at com.encheromax.business.advertisement.implementation.AdvertisementManager.addAdvertisementDocument(AdvertisementManager.java:536)
at com.encheromax.business.advertisement.implementation.AdvertisementManager.appendFullAdvertisementToIndex(AdvertisementManager.java:713)
at com.encheromax.api.implementation.ApiFinalizeAdvertisement.executeRequest(ApiFinalizeAdvertisement.java:224)
at com.encheromax.controller.advertisement.CreateAdvertisement.doReq(CreateAdvertisement.java:294)
at net.tmp.fwk.fw1.controller.Security.doReqFinal(Security.java:135)
at net.tmp.fwk.fw1.controller.Action.process(Action.java:302)
at net.tmp.fwk.fw1.controller.Action.doPost(Action.java:87)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:710)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:269)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
at com.encheromax.servlet.filter.EncheromaxBeanFilter.doFilter(EncheromaxBeanFilter.java:53)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:174)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:174)
at org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:200)
at org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:283)
at org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:773)
at org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:703)
at org.apache.jk.common.ChannelSocket$SocketConnection.runIt(ChannelSocket.java:895)
at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:689)
at java.lang.Thread.run(Thread.java:619)
--- En date de : Sam 9.1.10, Michael McCandless <lu...@mikemccandless.com> a écrit :
De: Michael McCandless <lu...@mikemccandless.com>
Objet: Re: Concurrent access IndexReader / IndexWriter - FileNotFoundException
À: java-user@lucene.apache.org
Date: Samedi 9 Janvier 2010, 14h51
Can you post the full FNFE stack trace?
Mike
On Fri, Jan 8, 2010 at 5:35 AM, legrand thomas <th...@yahoo.fr> wrote:
> Hi,
>
> I often get a FileNotFoundException when my single IndexWriter commits while the IndexReader also tries to read. My application is multithreaded (Tomcat uses the business APIs); I firstly thought the read/write access was thread-safe but I probably forget something.
>
> Please help me to understand my mistakes:
>
> - When should I close the IndexWriter ? Each time I add/update a document or never ?
> - Should my java commit/read methods be synchronized ?
> - Should I lock the directory and what's the best way to do it ?
>
> I referred to the Lucene FAQ ("why do I sometimes get a FileNotFoundException when I search and update my index at the same time?") but I did not disable any locking for processes searching or updating the index.
>
> I use lucene-2.4.1.
>
> Thanks in advance,
> Tom
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Concurrent access IndexReader / IndexWriter -
FileNotFoundException
Posted by Michael McCandless <lu...@mikemccandless.com>.
Can you post the full FNFE stack trace?
Mike
On Fri, Jan 8, 2010 at 5:35 AM, legrand thomas <th...@yahoo.fr> wrote:
> Hi,
>
> I often get a FileNotFoundException when my single IndexWriter commits while the IndexReader also tries to read. My application is multithreaded (Tomcat uses the business APIs); I firstly thought the read/write access was thread-safe but I probably forget something.
>
> Please help me to understand my mistakes:
>
> - When should I close the IndexWriter ? Each time I add/update a document or never ?
> - Should my java commit/read methods be synchronized ?
> - Should I lock the directory and what's the best way to do it ?
>
> I referred to the Lucene FAQ ("why do I sometimes get a FileNotFoundException when I search and update my index at the same time?") but I did not disable any locking for processes searching or updating the index.
>
> I use lucene-2.4.1.
>
> Thanks in advance,
> Tom
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org