SearcherTaxonomyManager usage

[Joe Eckard <ec...@gmail.com>]

Hello,

I'm new to lucene and I am having some trouble figuring out the right way
to use a SearcherTaxonomyManager for NRT faceted search. Assuming I set up
the STM with a reopen thread:

    // Index Writer
    Directory indexDir = FSDirectory.open(new File(indexDirectoryPath));
    IndexWriterConfig cfg = new IndexWriterConfig(VERSION, ANALYZER);
    TrackingIndexWriter trackingIndexWriter = new TrackingIndexWriter(new
IndexWriter(indexDir, cfg));

    // Taxonomy Writer
    Directory taxoDirectory = FSDirectory.open(new File(taxoDirectoryPath));
    DirectoryTaxonomyWriter taxoWriter = new
DirectoryTaxonomyWriter(taxoDirectory);

    // SearcherTaxonomyManager
    SearcherTaxonomyManager stm = new
SearcherTaxonomyManager(trackingIndexWriter.getIndexWriter(), true, new
SearcherFactory(), taxoWriter);

    // Reopen Thread
    Thread reopenThread = new
ControlledRealTimeReopenThread<SearcherTaxonomyManager.SearcherAndTaxonomy>(trackingIndexWriter,
stm, 5.0, 0.05);
    reopenThread.setName("Reopen Thread");
    reopenThread.setPriority(Math.min(Thread.currentThread().getPriority()
+ 2, Thread.MAX_PRIORITY));
    reopenThread.setDaemon(true);
    reopenThread.start();

and I always use the tracking index writer to index documents:

    trackingIndexWriter.updateDocument(...)

and I always use the STM to perform searches:

    SearcherTaxonomyManager.SearcherAndTaxonomy st = stm.acquire();
    try {
        DrillSideways drillSideways = new DrillSideways(st.searcher,
st.taxonomyReader);
        DrillSideways.DrillSidewaysResult dsr = drillSideways.search(...);
        ...
    } finally {
        stm.release(st);
    }

My understanding is that the index writer and the taxo writer will handle
flushing updates to disk periodically (whenever their buffers are full),
and the ReopenThread will handle flushing the deletes periodically (and
making sure I have a reader that sees the latest changes).

If that is true, and I call close() on the index writer and the taxonomy
writer during shutdown:

    reopenThread.interrupt();
    reopenThread.close();
    taxoWriter.close();
    trackingIndexWriter.getIndexWriter().close(true);

do I ever need to explicitly call commit() on the index writer or the taxo
writer?

Re: SearcherTaxonomyManager usage

[Michael McCandless <lu...@mikemccandless.com>]

When you close the taxoWriter and indexWriter, that calls commit() internally.

You can also call commit periodically if you want to, to make sure
changes are on durable storage so that if the hardware loses power, or
OS crashes, etc., you'll still have a valid index as of the last
commit.  Or you can skip it entirely, if rebuilding the index after a
crash is no big deal ...

Mike McCandless

http://blog.mikemccandless.com


On Sat, Oct 26, 2013 at 11:44 AM, Joe Eckard <ec...@gmail.com> wrote:
> Hello,
>
> I'm new to lucene and I am having some trouble figuring out the right way
> to use a SearcherTaxonomyManager for NRT faceted search. Assuming I set up
> the STM with a reopen thread:
>
>     // Index Writer
>     Directory indexDir = FSDirectory.open(new File(indexDirectoryPath));
>     IndexWriterConfig cfg = new IndexWriterConfig(VERSION, ANALYZER);
>     TrackingIndexWriter trackingIndexWriter = new TrackingIndexWriter(new
> IndexWriter(indexDir, cfg));
>
>     // Taxonomy Writer
>     Directory taxoDirectory = FSDirectory.open(new File(taxoDirectoryPath));
>     DirectoryTaxonomyWriter taxoWriter = new
> DirectoryTaxonomyWriter(taxoDirectory);
>
>     // SearcherTaxonomyManager
>     SearcherTaxonomyManager stm = new
> SearcherTaxonomyManager(trackingIndexWriter.getIndexWriter(), true, new
> SearcherFactory(), taxoWriter);
>
>     // Reopen Thread
>     Thread reopenThread = new
> ControlledRealTimeReopenThread<SearcherTaxonomyManager.SearcherAndTaxonomy>(trackingIndexWriter,
> stm, 5.0, 0.05);
>     reopenThread.setName("Reopen Thread");
>     reopenThread.setPriority(Math.min(Thread.currentThread().getPriority()
> + 2, Thread.MAX_PRIORITY));
>     reopenThread.setDaemon(true);
>     reopenThread.start();
>
> and I always use the tracking index writer to index documents:
>
>     trackingIndexWriter.updateDocument(...)
>
> and I always use the STM to perform searches:
>
>     SearcherTaxonomyManager.SearcherAndTaxonomy st = stm.acquire();
>     try {
>         DrillSideways drillSideways = new DrillSideways(st.searcher,
> st.taxonomyReader);
>         DrillSideways.DrillSidewaysResult dsr = drillSideways.search(...);
>         ...
>     } finally {
>         stm.release(st);
>     }
>
> My understanding is that the index writer and the taxo writer will handle
> flushing updates to disk periodically (whenever their buffers are full),
> and the ReopenThread will handle flushing the deletes periodically (and
> making sure I have a reader that sees the latest changes).
>
> If that is true, and I call close() on the index writer and the taxonomy
> writer during shutdown:
>
>     reopenThread.interrupt();
>     reopenThread.close();
>     taxoWriter.close();
>     trackingIndexWriter.getIndexWriter().close(true);
>
> do I ever need to explicitly call commit() on the index writer or the taxo
> writer?

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org