Testing whether a document is up-to-date in Lucene 4.0

[Arjen van der Meijden <ac...@tweakers.net>]

Hello List,

I'm currently trying to update my Lucene 3.6-application to 4.0.

Most of it works (although your migration guide lacks a bit of aspects I 
had to figure out myself), but for one fairly large database I want to 
check whether the Document in the Lucene database is already at the 
latest modification time. If so, I can continue to the next item on the 
list.

So to do that, I index an additional field in the Document with the 
modification time and later retrieve that.

With 3.6, I did this:
IndexReader reader = writer.getReader();
Term idTerm = new Term("idField, "idText");
TermDocs termDocs = reader.termDocs(idTerm);

if(termDocs.next())
{
  int docNum = termDocs.doc();
  Document doc = reader.document(docNum);
  // etc
}

It worked as far as I can tell, besides, it worked regardless of whether 
the writer was actually started with a empty directory (on disk) or one 
that had data in it.

What is the correct way to implement something like the above procedure 
in 4.0? The writer doesn't have a getReader() anymore and the 
IndexReader doesn't have a termDocs()-method (and the docsEnum is only 
on specific IndexReader's?).

The "migration guide" isn't very clear on this and I wasn't really able 
to find any useful documentation on how the new reader/writer-stuff is 
actually to be used... Did I overlook the documentation?

And there are some additional things to consider, like the database may 
not have been created yet (i.e. it may be empty at the moment of the 
above code). And in my current code, I only have access to the 
IndexWriter, a reader or searcher hasn't even been created yet.

Best regards,

Arjen

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org

RE: Testing whether a document is up-to-date in Lucene 4.0

[Uwe Schindler <uw...@thetaphi.de>]

Maybe also read: http://blog.thetaphi.de/2012/02/is-your-indexreader-atomic-major.html
A simple idea would be to wrap the NRT reader by SlowCompositeReaderWrapper, which has DocsEnum. If you don't do this for searching, the slowness does not matter.

Uwe

-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de


> -----Original Message-----
> From: Arjen van der Meijden [mailto:acmmailing@tweakers.net]
> Sent: Friday, October 12, 2012 4:04 PM
> To: java-user@lucene.apache.org
> Subject: Testing whether a document is up-to-date in Lucene 4.0
> 
> Hello List,
> 
> I'm currently trying to update my Lucene 3.6-application to 4.0.
> 
> Most of it works (although your migration guide lacks a bit of aspects I had to
> figure out myself), but for one fairly large database I want to check whether
> the Document in the Lucene database is already at the latest modification time.
> If so, I can continue to the next item on the list.
> 
> So to do that, I index an additional field in the Document with the modification
> time and later retrieve that.
> 
> With 3.6, I did this:
> IndexReader reader = writer.getReader(); Term idTerm = new Term("idField,
> "idText"); TermDocs termDocs = reader.termDocs(idTerm);
> 
> if(termDocs.next())
> {
>   int docNum = termDocs.doc();
>   Document doc = reader.document(docNum);
>   // etc
> }
> 
> It worked as far as I can tell, besides, it worked regardless of whether the
> writer was actually started with a empty directory (on disk) or one that had
> data in it.
> 
> What is the correct way to implement something like the above procedure in
> 4.0? The writer doesn't have a getReader() anymore and the IndexReader
> doesn't have a termDocs()-method (and the docsEnum is only on specific
> IndexReader's?).
> 
> The "migration guide" isn't very clear on this and I wasn't really able to find any
> useful documentation on how the new reader/writer-stuff is actually to be
> used... Did I overlook the documentation?
> 
> And there are some additional things to consider, like the database may not
> have been created yet (i.e. it may be empty at the moment of the above code).
> And in my current code, I only have access to the IndexWriter, a reader or
> searcher hasn't even been created yet.
> 
> Best regards,
> 
> Arjen
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org