Obtain Lock file timeout during deleteDocument()

[Martin Kobele <ma...@netsweeper.com>]

Hi,

I am a little confused, probably, because I missed some detail when looking 
through the code of lucene 2.1.

Scenario: Deleting documents works for a while, eventually, I get the 
exception that obtaining the lock files has timed out.


I was trying to find an answer to this.
I call IndexReader.deleteDocument() for the _first_ time.
If my index has several segments, my IndexReader is actually a MultiReader.
Therefore the variable directoryOwner is set to true and as the first step, a 
lock file is created. After that, the document is marked as deleted.

If I call deleteDocument again, it may or may not work.
Now by just reading the code, and I am sure I am missing some details, I am 
wondering, how can I successfully call deleteDocument again? The code will 
try to obtain the lockfile again, but since it is already there, it will time 
out. That is the point where I am confused. After I deleted a document, the 
write.lock file is still there, and directoryOwner is still true.


Maybe knowing more about this will help me to find out why I get the 
exception "Lock obtain timed out" after a while and after several successful 
document deletions.

Thank you!

Regards,
Martin

Re: Obtain Lock file timeout during deleteDocument()

[Martin Kobele <ma...@netsweeper.com>]

On Wednesday 30 May 2007 11:53:09 Martin Kobele wrote:
> On Wednesday 30 May 2007 11:49:41 Michael McCandless wrote:
> > You are only using a single instance of IndexReader, right?  If for
> > example you try to make a new instance of IndexReader and then call
> > deleteDocument on that new one, then you would hit the exception
> > unless you had closed the first one.
>
> Yeah, I use only one single instance of the IndexReader of this particular
> index. But I have like 20 IndexReaders of 20 different indexes open.
>
oh dear, it was my own fault.....
I was hoping I could monitor the last modified time of the directory to 
determine whether somebody has replaced the index. But lucene itself modifies 
the time stamp of the directory when you delete documents.
I am wiser now ;)

Thanks!

Martin


-- 
Martin Kobele
Software Developer
t. 519-826-5222 ext #224
f. 519-826-5228
martin.kobele@netsweeper.com
Netsweeper Corporate Head Office
104 Dawson Road
Guelph, Ontario
N1H 1A7

Re: Obtain Lock file timeout during deleteDocument()

[Martin Kobele <ma...@netsweeper.com>]

On Wednesday 30 May 2007 11:49:41 Michael McCandless wrote:
> "Martin Kobele" <ma...@netsweeper.com> wrote:
> > I was trying to find an answer to this.
> > I call IndexReader.deleteDocument() for the _first_ time.
> > If my index has several segments, my IndexReader is actually a
> > MultiReader. Therefore the variable directoryOwner is set to true and as
> > the first step, a lock file is created. After that, the document is
> > marked as deleted.
> >
> > If I call deleteDocument again, it may or may not work.
> > Now by just reading the code, and I am sure I am missing some details, I
> > am wondering, how can I successfully call deleteDocument again? The code
> > will try to obtain the lockfile again, but since it is already there, it
> > will time out. That is the point where I am confused. After I deleted a
> > document, the write.lock file is still there, and directoryOwner is still
> > true.
>
> The lock should be acquired only on the first call to deleteDocument.
> That method calls acquireWriteLock which only acquires the write lock
> if it hasn't already (ie, writeLock == null).  So the 2nd call to
> deleteDocument would see that the lock was already held and should not
> then try to acquire it again.

oh yeah, darn, I knew I missed that little detail! ;)

>
> You are only using a single instance of IndexReader, right?  If for
> example you try to make a new instance of IndexReader and then call
> deleteDocument on that new one, then you would hit the exception
> unless you had closed the first one.

Yeah, I use only one single instance of the IndexReader of this particular 
index. But I have like 20 IndexReaders of 20 different indexes open.


Thanks,
Martin

Re: Obtain Lock file timeout during deleteDocument()

[Michael McCandless <lu...@mikemccandless.com>]

"Martin Kobele" <ma...@netsweeper.com> wrote:

> I was trying to find an answer to this.
> I call IndexReader.deleteDocument() for the _first_ time.
> If my index has several segments, my IndexReader is actually a MultiReader.
> Therefore the variable directoryOwner is set to true and as the first step, a 
> lock file is created. After that, the document is marked as deleted.
> 
> If I call deleteDocument again, it may or may not work.
> Now by just reading the code, and I am sure I am missing some details, I am 
> wondering, how can I successfully call deleteDocument again? The code will 
> try to obtain the lockfile again, but since it is already there, it will time 
> out. That is the point where I am confused. After I deleted a document, the 
> write.lock file is still there, and directoryOwner is still true.

The lock should be acquired only on the first call to deleteDocument.
That method calls acquireWriteLock which only acquires the write lock
if it hasn't already (ie, writeLock == null).  So the 2nd call to
deleteDocument would see that the lock was already held and should not
then try to acquire it again.

You are only using a single instance of IndexReader, right?  If for
example you try to make a new instance of IndexReader and then call
deleteDocument on that new one, then you would hit the exception
unless you had closed the first one.

Mike

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org