Can I still use SearcherManager in this situation?

[Trejkaz <tr...@trypticon.org>]

In our application, most users sit around in read-only mode all the time
but there is one place where write access can occur, which is essentially
scripted at the moment. (*)

Currently, we start out opening an IndexReader. When the caller declares
that they are going to start writing, we open an IndexWriter. When they
finish writing, they declare that, at which point we close the IndexWriter
and reopen the IndexReader.

NRTManager vaguely sounded like the thing for the job but since we don't
have an IndexWriter from the start, I guess it cannot be used (actually, at
the start, you don't even know if you have write access - and even if you
do, you shouldn't be locking the index unless you're really going to make
changes. Otherwise you will be depriving some other user of doing the
same...)

Currently I'm looking at SearcherManager. It looks like the current API has
maybeRefresh(), which spins off a refresh in the background, but unlike
NRTManager, doesn't appear to have any way to wait for that refresh to
finish. Essentially I want to know when I can get a new IndexSearcher which
is up to date with my written changes.

Is there some "right" way to do this? I hope I don't have to reimplement
some weird combination of SearcherManager and NRTManager myself.

TX


(*) The larger case of writing bulk data to the index is done while the
index is not openable by the user, so it is excluded from the problem space
for now. Later it would be nice to do something about that as well, but
there are various problems.

Re: Can I still use SearcherManager in this situation?

[Trejkaz <tr...@trypticon.org>]

On Thu, Nov 8, 2012 at 8:29 AM, Trejkaz <tr...@trypticon.org> wrote:
> It's not only protected... but the class is final as well (the method
> might as well be private so that it doesn't give a false sense of hope
> that it can be overridden.)
>
> I might have to clone the whole class just to make the method public.

What I have ended up doing is wrapping SearcherManager and putting in
my own locking logic borrowed from Lucene 4. So when I call
maybeRefresh(), I know only one thread is doing it, which means I know
the refresh will occur before the method returns. This seems to work
fine and avoids duplicating much code. Once we switch to Lucene 4, I
can throw away my wrapper class as the method names match those of
Lucene 4's SearcherManager.

TX

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org

Re: Can I still use SearcherManager in this situation?

[Trejkaz <tr...@trypticon.org>]

On Wed, Nov 7, 2012 at 11:10 PM, Ian Lea <ia...@gmail.com> wrote:
>
> Sorry, didn't notice that refreshIfNeeded is protected.

It's not only protected... but the class is final as well (the method
might as well be private so that it doesn't give a false sense of hope
that it can be overridden.)

I might have to clone the whole class just to make the method public.

> You could run IndexUpgrader on your old indexes to bring them up to date, in stages
> from v2 to v3 then v4, then upgrade.

If it were our own indexes I would just do that, but this is an
application used by others, and our users complained the last time I
tried to put in any kind of index-wide migration.

TX

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org

Re: Can I still use SearcherManager in this situation?

[Ian Lea <ia...@gmail.com>]

Sorry, didn't notice that refreshIfNeeded is protected.  You could run
IndexUpgrader on your old indexes to bring them up to date, in stages
from v2 to v3 then v4, then upgrade.


--
Ian.


On Wed, Nov 7, 2012 at 12:04 PM, Trejkaz <tr...@trypticon.org> wrote:
> On Wed, Nov 7, 2012 at 10:11 PM, Ian Lea <ia...@gmail.com> wrote:
>> 4.0 has maybeRefreshBlocking which "is useful if you want to guarantee
>> that the next call to acquire() will return a refreshed instance".
>> You don't say what version you're using.
>>
>> If you're stuck on 3.6.1 can you do something with refreshIfNeeded()
>> or isSearcherCurrent()?
>
> Ah, yes. I'm stuck on 3.6 at the moment (for backwards compatibility
> with v2 indexes.) refreshIfNeeded() might work, but it's protected so
> I will have to subclass to make it public (and hope that nothing bad
> happens by accessing it directly.)
>
> TX
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org

Re: Can I still use SearcherManager in this situation?

[Trejkaz <tr...@trypticon.org>]

On Wed, Nov 7, 2012 at 10:11 PM, Ian Lea <ia...@gmail.com> wrote:
> 4.0 has maybeRefreshBlocking which "is useful if you want to guarantee
> that the next call to acquire() will return a refreshed instance".
> You don't say what version you're using.
>
> If you're stuck on 3.6.1 can you do something with refreshIfNeeded()
> or isSearcherCurrent()?

Ah, yes. I'm stuck on 3.6 at the moment (for backwards compatibility
with v2 indexes.) refreshIfNeeded() might work, but it's protected so
I will have to subclass to make it public (and hope that nothing bad
happens by accessing it directly.)

TX

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org

Re: Can I still use SearcherManager in this situation?

[Ian Lea <ia...@gmail.com>]

4.0 has maybeRefreshBlocking which "is useful if you want to guarantee
that the next call to acquire() will return a refreshed instance".
You don't say what version you're using.

If you're stuck on 3.6.1 can you do something with refreshIfNeeded()
or isSearcherCurrent()?


--
Ian.


On Wed, Nov 7, 2012 at 4:08 AM, Trejkaz <tr...@trypticon.org> wrote:
> In our application, most users sit around in read-only mode all the time
> but there is one place where write access can occur, which is essentially
> scripted at the moment. (*)
>
> Currently, we start out opening an IndexReader. When the caller declares
> that they are going to start writing, we open an IndexWriter. When they
> finish writing, they declare that, at which point we close the IndexWriter
> and reopen the IndexReader.
>
> NRTManager vaguely sounded like the thing for the job but since we don't
> have an IndexWriter from the start, I guess it cannot be used (actually, at
> the start, you don't even know if you have write access - and even if you
> do, you shouldn't be locking the index unless you're really going to make
> changes. Otherwise you will be depriving some other user of doing the
> same...)
>
> Currently I'm looking at SearcherManager. It looks like the current API has
> maybeRefresh(), which spins off a refresh in the background, but unlike
> NRTManager, doesn't appear to have any way to wait for that refresh to
> finish. Essentially I want to know when I can get a new IndexSearcher which
> is up to date with my written changes.
>
> Is there some "right" way to do this? I hope I don't have to reimplement
> some weird combination of SearcherManager and NRTManager myself.
>
> TX
>
>
> (*) The larger case of writing bulk data to the index is done while the
> index is not openable by the user, so it is excluded from the problem space
> for now. Later it would be nice to do something about that as well, but
> there are various problems.

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org