You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Danil ŢORIN <to...@gmail.com> on 2009/03/02 08:42:43 UTC
Re: segments.gen file
What about opening IndexReader/Searcher on certain commit point?
Will it be implemented in 2.9?
The use-case is similar to the one Michael Busch described, only I
would like to open (or reopen) a searcher on specific commit point.
On Fri, Feb 27, 2009 at 20:31, Michael McCandless
<lu...@mikemccandless.com> wrote:
>
> Michael Busch wrote:
>
>> Actually I personally don't need Lucene to be "write once". The reason why
>> I started this thread about the segments.gen file was that in our project we
>> sometimes need to rollback to a previous commit-point (using Lucene 2.4.0)
>> that we keep around with the SnapshotDeletionPolicy. To get rid of the
>> newest commit-point we simply delete the most recent segments file. But then
>> we also have to delete the segments.gen file, otherwise Lucene will read the
>> generation from it and try to find the segments file we deleted. Then Lucene
>> will recreate the segments.gen file. This just made me think that this is
>> not very clean (deleting and recreating the segments.gen) especially because
>> we use a local FS and don't even need the .gen file.
>
> Ahh... OK. In trunk/2.9, you can explicitly open an IndexWriter on a prior
> commit point, which will take care of the gen file for you.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: segments.gen file
Posted by Michael McCandless <lu...@mikemccandless.com>.
Super, thanks!
Mike
Danil ŢORIN wrote:
> https://issues.apache.org/jira/browse/LUCENE-1551
>
> On Mon, Mar 2, 2009 at 13:40, Michael McCandless
> <lu...@mikemccandless.com> wrote:
>>
>> Ahh good point, that is missing. Can you open a Jira issue? Thanks.
>>
>> Mike
>>
>> Danil ŢORIN wrote:
>>
>>> I'm more interested in reopen(IndexCommit) so I could have full
>>> benefits of LUCENE-1483 (faster open, warm caches, less GC and so
>>> on)
>>> but on specific commit point.
>>>
>>> I'm not sure it's possible in current trunk, and if we have
>>> open(IndexCommit) & company, maybe we should also have similar set
>>> of
>>> methods for reopen?
>>>
>>> On Mon, Mar 2, 2009 at 12:12, Michael McCandless
>>> <lu...@mikemccandless.com> wrote:
>>>>
>>>> This is already available, as of 2.4.
>>>>
>>>> Mike
>>>>
>>>> Danil ŢORIN wrote:
>>>>
>>>>> What about opening IndexReader/Searcher on certain commit point?
>>>>> Will it be implemented in 2.9?
>>>>>
>>>>> The use-case is similar to the one Michael Busch described, only I
>>>>> would like to open (or reopen) a searcher on specific commit
>>>>> point.
>>>>>
>>>>>
>>>>> On Fri, Feb 27, 2009 at 20:31, Michael McCandless
>>>>> <lu...@mikemccandless.com> wrote:
>>>>>>
>>>>>> Michael Busch wrote:
>>>>>>
>>>>>>> Actually I personally don't need Lucene to be "write once".
>>>>>>> The reason
>>>>>>> why
>>>>>>> I started this thread about the segments.gen file was that in
>>>>>>> our
>>>>>>> project we
>>>>>>> sometimes need to rollback to a previous commit-point (using
>>>>>>> Lucene
>>>>>>> 2.4.0)
>>>>>>> that we keep around with the SnapshotDeletionPolicy. To get
>>>>>>> rid of the
>>>>>>> newest commit-point we simply delete the most recent segments
>>>>>>> file.
>>>>>>> But
>>>>>>> then
>>>>>>> we also have to delete the segments.gen file, otherwise Lucene
>>>>>>> will
>>>>>>> read
>>>>>>> the
>>>>>>> generation from it and try to find the segments file we
>>>>>>> deleted. Then
>>>>>>> Lucene
>>>>>>> will recreate the segments.gen file. This just made me think
>>>>>>> that this
>>>>>>> is
>>>>>>> not very clean (deleting and recreating the segments.gen)
>>>>>>> especially
>>>>>>> because
>>>>>>> we use a local FS and don't even need the .gen file.
>>>>>>
>>>>>> Ahh... OK. In trunk/2.9, you can explicitly open an
>>>>>> IndexWriter on a
>>>>>> prior
>>>>>> commit point, which will take care of the gen file for you.
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
>>>>> For additional commands, e-mail: java-dev-help@lucene.apache.org
>>>>>
>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
>>>> For additional commands, e-mail: java-dev-help@lucene.apache.org
>>>>
>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: java-dev-help@lucene.apache.org
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-dev-help@lucene.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: segments.gen file
Posted by Danil ŢORIN <to...@gmail.com>.
https://issues.apache.org/jira/browse/LUCENE-1551
On Mon, Mar 2, 2009 at 13:40, Michael McCandless
<lu...@mikemccandless.com> wrote:
>
> Ahh good point, that is missing. Can you open a Jira issue? Thanks.
>
> Mike
>
> Danil ŢORIN wrote:
>
>> I'm more interested in reopen(IndexCommit) so I could have full
>> benefits of LUCENE-1483 (faster open, warm caches, less GC and so on)
>> but on specific commit point.
>>
>> I'm not sure it's possible in current trunk, and if we have
>> open(IndexCommit) & company, maybe we should also have similar set of
>> methods for reopen?
>>
>> On Mon, Mar 2, 2009 at 12:12, Michael McCandless
>> <lu...@mikemccandless.com> wrote:
>>>
>>> This is already available, as of 2.4.
>>>
>>> Mike
>>>
>>> Danil ŢORIN wrote:
>>>
>>>> What about opening IndexReader/Searcher on certain commit point?
>>>> Will it be implemented in 2.9?
>>>>
>>>> The use-case is similar to the one Michael Busch described, only I
>>>> would like to open (or reopen) a searcher on specific commit point.
>>>>
>>>>
>>>> On Fri, Feb 27, 2009 at 20:31, Michael McCandless
>>>> <lu...@mikemccandless.com> wrote:
>>>>>
>>>>> Michael Busch wrote:
>>>>>
>>>>>> Actually I personally don't need Lucene to be "write once". The reason
>>>>>> why
>>>>>> I started this thread about the segments.gen file was that in our
>>>>>> project we
>>>>>> sometimes need to rollback to a previous commit-point (using Lucene
>>>>>> 2.4.0)
>>>>>> that we keep around with the SnapshotDeletionPolicy. To get rid of the
>>>>>> newest commit-point we simply delete the most recent segments file.
>>>>>> But
>>>>>> then
>>>>>> we also have to delete the segments.gen file, otherwise Lucene will
>>>>>> read
>>>>>> the
>>>>>> generation from it and try to find the segments file we deleted. Then
>>>>>> Lucene
>>>>>> will recreate the segments.gen file. This just made me think that this
>>>>>> is
>>>>>> not very clean (deleting and recreating the segments.gen) especially
>>>>>> because
>>>>>> we use a local FS and don't even need the .gen file.
>>>>>
>>>>> Ahh... OK. In trunk/2.9, you can explicitly open an IndexWriter on a
>>>>> prior
>>>>> commit point, which will take care of the gen file for you.
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
>>>> For additional commands, e-mail: java-dev-help@lucene.apache.org
>>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: java-dev-help@lucene.apache.org
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-dev-help@lucene.apache.org
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: segments.gen file
Posted by Michael McCandless <lu...@mikemccandless.com>.
Ahh good point, that is missing. Can you open a Jira issue? Thanks.
Mike
Danil ŢORIN wrote:
> I'm more interested in reopen(IndexCommit) so I could have full
> benefits of LUCENE-1483 (faster open, warm caches, less GC and so on)
> but on specific commit point.
>
> I'm not sure it's possible in current trunk, and if we have
> open(IndexCommit) & company, maybe we should also have similar set of
> methods for reopen?
>
> On Mon, Mar 2, 2009 at 12:12, Michael McCandless
> <lu...@mikemccandless.com> wrote:
>>
>> This is already available, as of 2.4.
>>
>> Mike
>>
>> Danil ŢORIN wrote:
>>
>>> What about opening IndexReader/Searcher on certain commit point?
>>> Will it be implemented in 2.9?
>>>
>>> The use-case is similar to the one Michael Busch described, only I
>>> would like to open (or reopen) a searcher on specific commit point.
>>>
>>>
>>> On Fri, Feb 27, 2009 at 20:31, Michael McCandless
>>> <lu...@mikemccandless.com> wrote:
>>>>
>>>> Michael Busch wrote:
>>>>
>>>>> Actually I personally don't need Lucene to be "write once". The
>>>>> reason
>>>>> why
>>>>> I started this thread about the segments.gen file was that in our
>>>>> project we
>>>>> sometimes need to rollback to a previous commit-point (using
>>>>> Lucene
>>>>> 2.4.0)
>>>>> that we keep around with the SnapshotDeletionPolicy. To get rid
>>>>> of the
>>>>> newest commit-point we simply delete the most recent segments
>>>>> file. But
>>>>> then
>>>>> we also have to delete the segments.gen file, otherwise Lucene
>>>>> will read
>>>>> the
>>>>> generation from it and try to find the segments file we deleted.
>>>>> Then
>>>>> Lucene
>>>>> will recreate the segments.gen file. This just made me think
>>>>> that this
>>>>> is
>>>>> not very clean (deleting and recreating the segments.gen)
>>>>> especially
>>>>> because
>>>>> we use a local FS and don't even need the .gen file.
>>>>
>>>> Ahh... OK. In trunk/2.9, you can explicitly open an IndexWriter
>>>> on a
>>>> prior
>>>> commit point, which will take care of the gen file for you.
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: java-dev-help@lucene.apache.org
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-dev-help@lucene.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: segments.gen file
Posted by Danil ŢORIN <to...@gmail.com>.
I'm more interested in reopen(IndexCommit) so I could have full
benefits of LUCENE-1483 (faster open, warm caches, less GC and so on)
but on specific commit point.
I'm not sure it's possible in current trunk, and if we have
open(IndexCommit) & company, maybe we should also have similar set of
methods for reopen?
On Mon, Mar 2, 2009 at 12:12, Michael McCandless
<lu...@mikemccandless.com> wrote:
>
> This is already available, as of 2.4.
>
> Mike
>
> Danil ŢORIN wrote:
>
>> What about opening IndexReader/Searcher on certain commit point?
>> Will it be implemented in 2.9?
>>
>> The use-case is similar to the one Michael Busch described, only I
>> would like to open (or reopen) a searcher on specific commit point.
>>
>>
>> On Fri, Feb 27, 2009 at 20:31, Michael McCandless
>> <lu...@mikemccandless.com> wrote:
>>>
>>> Michael Busch wrote:
>>>
>>>> Actually I personally don't need Lucene to be "write once". The reason
>>>> why
>>>> I started this thread about the segments.gen file was that in our
>>>> project we
>>>> sometimes need to rollback to a previous commit-point (using Lucene
>>>> 2.4.0)
>>>> that we keep around with the SnapshotDeletionPolicy. To get rid of the
>>>> newest commit-point we simply delete the most recent segments file. But
>>>> then
>>>> we also have to delete the segments.gen file, otherwise Lucene will read
>>>> the
>>>> generation from it and try to find the segments file we deleted. Then
>>>> Lucene
>>>> will recreate the segments.gen file. This just made me think that this
>>>> is
>>>> not very clean (deleting and recreating the segments.gen) especially
>>>> because
>>>> we use a local FS and don't even need the .gen file.
>>>
>>> Ahh... OK. In trunk/2.9, you can explicitly open an IndexWriter on a
>>> prior
>>> commit point, which will take care of the gen file for you.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-dev-help@lucene.apache.org
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: segments.gen file
Posted by Michael McCandless <lu...@mikemccandless.com>.
This is already available, as of 2.4.
Mike
Danil ŢORIN wrote:
> What about opening IndexReader/Searcher on certain commit point?
> Will it be implemented in 2.9?
>
> The use-case is similar to the one Michael Busch described, only I
> would like to open (or reopen) a searcher on specific commit point.
>
>
> On Fri, Feb 27, 2009 at 20:31, Michael McCandless
> <lu...@mikemccandless.com> wrote:
>>
>> Michael Busch wrote:
>>
>>> Actually I personally don't need Lucene to be "write once". The
>>> reason why
>>> I started this thread about the segments.gen file was that in our
>>> project we
>>> sometimes need to rollback to a previous commit-point (using
>>> Lucene 2.4.0)
>>> that we keep around with the SnapshotDeletionPolicy. To get rid of
>>> the
>>> newest commit-point we simply delete the most recent segments
>>> file. But then
>>> we also have to delete the segments.gen file, otherwise Lucene
>>> will read the
>>> generation from it and try to find the segments file we deleted.
>>> Then Lucene
>>> will recreate the segments.gen file. This just made me think that
>>> this is
>>> not very clean (deleting and recreating the segments.gen)
>>> especially because
>>> we use a local FS and don't even need the .gen file.
>>
>> Ahh... OK. In trunk/2.9, you can explicitly open an IndexWriter on
>> a prior
>> commit point, which will take care of the gen file for you.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org