You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Shay Banon <ki...@gmail.com> on 2009/05/15 03:37:10 UTC
Getting an IndexReader from a committed IndexWriter
Hi,
I just had a look at the job done in IndexWriter in order to get an
IndexReader with all the current ongoing changes done using the IndexWriter.
This feature is very useful, and I was wondering if another feature, which
(I think) is simple to implement (compared to the previous one) might make
sense.
Many times, an application opens an IndexWriter, does whatever changes it
does, and then commits the changes. It would be nice to get an IndexReader
(read only one is fine) that corresponds to the committed (or even closed)
IndexWriter. This will allow for a cache of IndexReader that is already used
to be updated with a fresh IndexReader, without the need to reopen one
(which should be slower than opening one based on the IndexWriter
information). The main difference is the fact that the mentioned IndexReader
could still be reopened without the need to throw an AlreadyClosedException.
Does it make sense?
Cheers,
Shay
--
View this message in context: http://www.nabble.com/Getting-an-IndexReader-from-a-committed-IndexWriter-tp23551978p23551978.html
Sent from the Lucene - Java Developer mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: Getting an IndexReader from a committed IndexWriter
Posted by Shay Banon <ki...@gmail.com>.
Done: https://issues.apache.org/jira/browse/LUCENE-1637.
Jason Rutherglen-2 wrote:
>
> Ok, that makes sense. Can you open an issue?
>
> On Thu, May 14, 2009 at 6:56 PM, Shay Banon <ki...@gmail.com> wrote:
>
>>
>> I was referring in my original post to the code in trunk. What I am after
>> is
>> a "getCommittedReader" method similar to getReader that allows to open
>> (fast) an IndexReader reflecting the latest committed state (or closed
>> state) of the IndexWriter. The problem with the getReader method is the
>> fact
>> that it is tied to the IndexWriter instance.
>>
>>
>> Jason Rutherglen-2 wrote:
>> >
>> > Hi Shay,
>> >
>> > I think IndexWriter.getReader from LUCENE-1516 in trunk is what
>> > you're talking about? It pools readers internally so there's no
>> > need to call IndexReader.reopen, one simply calls IW.getReader
>> > to get new readers containing recent updates.
>> >
>> > -J
>> >
>> > BTW I replied to the message on java-user@lucene.apache.org.
>> >
>> > On Thu, May 14, 2009 at 6:37 PM, Shay Banon <ki...@gmail.com> wrote:
>> >
>> >>
>> >> Hi,
>> >>
>> >> I just had a look at the job done in IndexWriter in order to get an
>> >> IndexReader with all the current ongoing changes done using the
>> >> IndexWriter.
>> >> This feature is very useful, and I was wondering if another feature,
>> >> which
>> >> (I think) is simple to implement (compared to the previous one) might
>> >> make
>> >> sense.
>> >>
>> >> Many times, an application opens an IndexWriter, does whatever
>> changes
>> >> it
>> >> does, and then commits the changes. It would be nice to get an
>> >> IndexReader
>> >> (read only one is fine) that corresponds to the committed (or even
>> >> closed)
>> >> IndexWriter. This will allow for a cache of IndexReader that is
>> already
>> >> used
>> >> to be updated with a fresh IndexReader, without the need to reopen one
>> >> (which should be slower than opening one based on the IndexWriter
>> >> information). The main difference is the fact that the mentioned
>> >> IndexReader
>> >> could still be reopened without the need to throw an
>> >> AlreadyClosedException.
>> >>
>> >> Does it make sense?
>> >>
>> >> Cheers,
>> >> Shay
>> >> --
>> >> View this message in context:
>> >>
>> http://www.nabble.com/Getting-an-IndexReader-from-a-committed-IndexWriter-tp23551978p23551978.html
>> >> Sent from the Lucene - Java Developer mailing list archive at
>> Nabble.com.
>> >>
>> >>
>> >> ---------------------------------------------------------------------
>> >> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
>> >> For additional commands, e-mail: java-dev-help@lucene.apache.org
>> >>
>> >>
>> >
>> >
>>
>> --
>> View this message in context:
>> http://www.nabble.com/Getting-an-IndexReader-from-a-committed-IndexWriter-tp23551978p23552133.html
>> Sent from the Lucene - Java Developer mailing list archive at Nabble.com.
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-dev-help@lucene.apache.org
>>
>>
>
>
--
View this message in context: http://www.nabble.com/Getting-an-IndexReader-from-a-committed-IndexWriter-tp23551978p23552816.html
Sent from the Lucene - Java Developer mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: Getting an IndexReader from a committed IndexWriter
Posted by Jason Rutherglen <ja...@gmail.com>.
Ok, that makes sense. Can you open an issue?
On Thu, May 14, 2009 at 6:56 PM, Shay Banon <ki...@gmail.com> wrote:
>
> I was referring in my original post to the code in trunk. What I am after
> is
> a "getCommittedReader" method similar to getReader that allows to open
> (fast) an IndexReader reflecting the latest committed state (or closed
> state) of the IndexWriter. The problem with the getReader method is the
> fact
> that it is tied to the IndexWriter instance.
>
>
> Jason Rutherglen-2 wrote:
> >
> > Hi Shay,
> >
> > I think IndexWriter.getReader from LUCENE-1516 in trunk is what
> > you're talking about? It pools readers internally so there's no
> > need to call IndexReader.reopen, one simply calls IW.getReader
> > to get new readers containing recent updates.
> >
> > -J
> >
> > BTW I replied to the message on java-user@lucene.apache.org.
> >
> > On Thu, May 14, 2009 at 6:37 PM, Shay Banon <ki...@gmail.com> wrote:
> >
> >>
> >> Hi,
> >>
> >> I just had a look at the job done in IndexWriter in order to get an
> >> IndexReader with all the current ongoing changes done using the
> >> IndexWriter.
> >> This feature is very useful, and I was wondering if another feature,
> >> which
> >> (I think) is simple to implement (compared to the previous one) might
> >> make
> >> sense.
> >>
> >> Many times, an application opens an IndexWriter, does whatever changes
> >> it
> >> does, and then commits the changes. It would be nice to get an
> >> IndexReader
> >> (read only one is fine) that corresponds to the committed (or even
> >> closed)
> >> IndexWriter. This will allow for a cache of IndexReader that is already
> >> used
> >> to be updated with a fresh IndexReader, without the need to reopen one
> >> (which should be slower than opening one based on the IndexWriter
> >> information). The main difference is the fact that the mentioned
> >> IndexReader
> >> could still be reopened without the need to throw an
> >> AlreadyClosedException.
> >>
> >> Does it make sense?
> >>
> >> Cheers,
> >> Shay
> >> --
> >> View this message in context:
> >>
> http://www.nabble.com/Getting-an-IndexReader-from-a-committed-IndexWriter-tp23551978p23551978.html
> >> Sent from the Lucene - Java Developer mailing list archive at
> Nabble.com.
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> >> For additional commands, e-mail: java-dev-help@lucene.apache.org
> >>
> >>
> >
> >
>
> --
> View this message in context:
> http://www.nabble.com/Getting-an-IndexReader-from-a-committed-IndexWriter-tp23551978p23552133.html
> Sent from the Lucene - Java Developer mailing list archive at Nabble.com.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>
>
Re: Getting an IndexReader from a committed IndexWriter
Posted by Shay Banon <ki...@gmail.com>.
I was referring in my original post to the code in trunk. What I am after is
a "getCommittedReader" method similar to getReader that allows to open
(fast) an IndexReader reflecting the latest committed state (or closed
state) of the IndexWriter. The problem with the getReader method is the fact
that it is tied to the IndexWriter instance.
Jason Rutherglen-2 wrote:
>
> Hi Shay,
>
> I think IndexWriter.getReader from LUCENE-1516 in trunk is what
> you're talking about? It pools readers internally so there's no
> need to call IndexReader.reopen, one simply calls IW.getReader
> to get new readers containing recent updates.
>
> -J
>
> BTW I replied to the message on java-user@lucene.apache.org.
>
> On Thu, May 14, 2009 at 6:37 PM, Shay Banon <ki...@gmail.com> wrote:
>
>>
>> Hi,
>>
>> I just had a look at the job done in IndexWriter in order to get an
>> IndexReader with all the current ongoing changes done using the
>> IndexWriter.
>> This feature is very useful, and I was wondering if another feature,
>> which
>> (I think) is simple to implement (compared to the previous one) might
>> make
>> sense.
>>
>> Many times, an application opens an IndexWriter, does whatever changes
>> it
>> does, and then commits the changes. It would be nice to get an
>> IndexReader
>> (read only one is fine) that corresponds to the committed (or even
>> closed)
>> IndexWriter. This will allow for a cache of IndexReader that is already
>> used
>> to be updated with a fresh IndexReader, without the need to reopen one
>> (which should be slower than opening one based on the IndexWriter
>> information). The main difference is the fact that the mentioned
>> IndexReader
>> could still be reopened without the need to throw an
>> AlreadyClosedException.
>>
>> Does it make sense?
>>
>> Cheers,
>> Shay
>> --
>> View this message in context:
>> http://www.nabble.com/Getting-an-IndexReader-from-a-committed-IndexWriter-tp23551978p23551978.html
>> Sent from the Lucene - Java Developer mailing list archive at Nabble.com.
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-dev-help@lucene.apache.org
>>
>>
>
>
--
View this message in context: http://www.nabble.com/Getting-an-IndexReader-from-a-committed-IndexWriter-tp23551978p23552133.html
Sent from the Lucene - Java Developer mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: Getting an IndexReader from a committed IndexWriter
Posted by Jason Rutherglen <ja...@gmail.com>.
Hi Shay,
I think IndexWriter.getReader from LUCENE-1516 in trunk is what
you're talking about? It pools readers internally so there's no
need to call IndexReader.reopen, one simply calls IW.getReader
to get new readers containing recent updates.
-J
BTW I replied to the message on java-user@lucene.apache.org.
On Thu, May 14, 2009 at 6:37 PM, Shay Banon <ki...@gmail.com> wrote:
>
> Hi,
>
> I just had a look at the job done in IndexWriter in order to get an
> IndexReader with all the current ongoing changes done using the
> IndexWriter.
> This feature is very useful, and I was wondering if another feature, which
> (I think) is simple to implement (compared to the previous one) might make
> sense.
>
> Many times, an application opens an IndexWriter, does whatever changes it
> does, and then commits the changes. It would be nice to get an IndexReader
> (read only one is fine) that corresponds to the committed (or even closed)
> IndexWriter. This will allow for a cache of IndexReader that is already
> used
> to be updated with a fresh IndexReader, without the need to reopen one
> (which should be slower than opening one based on the IndexWriter
> information). The main difference is the fact that the mentioned
> IndexReader
> could still be reopened without the need to throw an
> AlreadyClosedException.
>
> Does it make sense?
>
> Cheers,
> Shay
> --
> View this message in context:
> http://www.nabble.com/Getting-an-IndexReader-from-a-committed-IndexWriter-tp23551978p23551978.html
> Sent from the Lucene - Java Developer mailing list archive at Nabble.com.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>
>
Re: Getting an IndexReader from a committed IndexWriter
Posted by Jason Rutherglen <ja...@gmail.com>.
Hi Shay,
I think IndexWriter.getReader from LUCENE-1516 in trunk is what
you're talking about? It pools readers internally so there's no
need to call IndexReader.reopen, one simply calls IW.getReader
to get new readers containing recent updates.
-J
BTW I replied to the message on java-user@lucene.apache.org.
On Thu, May 14, 2009 at 6:37 PM, Shay Banon <ki...@gmail.com> wrote:
>
> Hi,
>
> I just had a look at the job done in IndexWriter in order to get an
> IndexReader with all the current ongoing changes done using the
> IndexWriter.
> This feature is very useful, and I was wondering if another feature, which
> (I think) is simple to implement (compared to the previous one) might make
> sense.
>
> Many times, an application opens an IndexWriter, does whatever changes it
> does, and then commits the changes. It would be nice to get an IndexReader
> (read only one is fine) that corresponds to the committed (or even closed)
> IndexWriter. This will allow for a cache of IndexReader that is already
> used
> to be updated with a fresh IndexReader, without the need to reopen one
> (which should be slower than opening one based on the IndexWriter
> information). The main difference is the fact that the mentioned
> IndexReader
> could still be reopened without the need to throw an
> AlreadyClosedException.
>
> Does it make sense?
>
> Cheers,
> Shay
> --
> View this message in context:
> http://www.nabble.com/Getting-an-IndexReader-from-a-committed-IndexWriter-tp23551978p23551978.html
> Sent from the Lucene - Java Developer mailing list archive at Nabble.com.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>
>