You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Ted Stockwell <em...@yahoo.com> on 2009/09/01 14:54:52 UTC
Question about IndexCommit
Hi All,
I am interested in using Lucene to index RDF (Resource Description Format) data.
Ultimately I want to create a transactional interface to the data with proper transaction isolation.
Therefore I am trying to educate myself on the details of index readers and writers, I am using v2.9rc2.
One thing I have noticed is that it seems that Lucene is designed with the intent that changes to the index by a writer should not be visible to index readers until the writer commits its changes.
However, one thing that I have noticed in the code is that when documents are deleted by a writer the writers change the .del (deleted documents) files in the segment in which the document exists.
Thus, it appears that document deletions may become visible to readers *before* a write actually commits its changes.
Am I correct about how document deletions work?
Since I'm new to Lucene I'm wondering if I'm missing something...
Thanks,
ted stockwell
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Question about IndexCommit
Posted by Chris Hostetter <ho...@fucit.org>.
: Subject: Question about IndexCommit
: In-Reply-To: <9a...@mail.gmail.com>
http://people.apache.org/~hossman/#threadhijack
Thread Hijacking on Mailing Lists
When starting a new discussion on a mailing list, please do not reply to
an existing message, instead start a fresh email. Even if you change the
subject line of your email, other mail headers still track which thread
you replied to and your question is "hidden" in that thread and gets less
attention. It makes following discussions in the mailing list archives
particularly difficult.
See Also: http://en.wikipedia.org/wiki/Thread_hijacking
-Hoss
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Question about IndexCommit
Posted by Ted Stockwell <em...@yahoo.com>.
That's excellent.
Thanks very much for the explanations....
----- Original Message ----
> From: Michael McCandless <lu...@mikemccandless.com>
> To: java-user@lucene.apache.org
> Sent: Tuesday, September 1, 2009 8:26:45 AM
> Subject: Re: Question about IndexCommit
>
> Further, when IndexWriter writes new .del files, it's always to a new
> (next generation) filename, so that the old .del file remains present.
> This means if a fresh IndexReader is opened, it will load the old
> .del file, and still not see any of IndexWriter's pending changes.
>
> Mike
>
> On Tue, Sep 1, 2009 at 9:04 AM, Shai Erera wrote:
> > If I'm not mistaken, IndexReader reads the .del file into memory, and
> > therefore subsequent updates to it won't be visible to it.
> >
> > Shai
> >
> > On Tue, Sep 1, 2009 at 3:54 PM, Ted Stockwell wrote:
> >
> >> Hi All,
> >>
> >> I am interested in using Lucene to index RDF (Resource Description Format)
> >> data.
> >> Ultimately I want to create a transactional interface to the data with
> >> proper transaction isolation.
> >> Therefore I am trying to educate myself on the details of index readers and
> >> writers, I am using v2.9rc2.
> >>
> >> One thing I have noticed is that it seems that Lucene is designed with the
> >> intent that changes to the index by a writer should not be visible to index
> >> readers until the writer commits its changes.
> >> However, one thing that I have noticed in the code is that when documents
> >> are deleted by a writer the writers change the .del (deleted documents)
> >> files in the segment in which the document exists.
> >> Thus, it appears that document deletions may become visible to readers
> >> *before* a write actually commits its changes.
> >>
> >> Am I correct about how document deletions work?
> >> Since I'm new to Lucene I'm wondering if I'm missing something...
> >>
> >>
> >> Thanks,
> >> ted stockwell
> >>
> >>
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> >> For additional commands, e-mail: java-user-help@lucene.apache.org
> >>
> >>
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Question about IndexCommit
Posted by Michael McCandless <lu...@mikemccandless.com>.
Further, when IndexWriter writes new .del files, it's always to a new
(next generation) filename, so that the old .del file remains present.
This means if a fresh IndexReader is opened, it will load the old
.del file, and still not see any of IndexWriter's pending changes.
Mike
On Tue, Sep 1, 2009 at 9:04 AM, Shai Erera<se...@gmail.com> wrote:
> If I'm not mistaken, IndexReader reads the .del file into memory, and
> therefore subsequent updates to it won't be visible to it.
>
> Shai
>
> On Tue, Sep 1, 2009 at 3:54 PM, Ted Stockwell <em...@yahoo.com> wrote:
>
>> Hi All,
>>
>> I am interested in using Lucene to index RDF (Resource Description Format)
>> data.
>> Ultimately I want to create a transactional interface to the data with
>> proper transaction isolation.
>> Therefore I am trying to educate myself on the details of index readers and
>> writers, I am using v2.9rc2.
>>
>> One thing I have noticed is that it seems that Lucene is designed with the
>> intent that changes to the index by a writer should not be visible to index
>> readers until the writer commits its changes.
>> However, one thing that I have noticed in the code is that when documents
>> are deleted by a writer the writers change the .del (deleted documents)
>> files in the segment in which the document exists.
>> Thus, it appears that document deletions may become visible to readers
>> *before* a write actually commits its changes.
>>
>> Am I correct about how document deletions work?
>> Since I'm new to Lucene I'm wondering if I'm missing something...
>>
>>
>> Thanks,
>> ted stockwell
>>
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Question about IndexCommit
Posted by Shai Erera <se...@gmail.com>.
If I'm not mistaken, IndexReader reads the .del file into memory, and
therefore subsequent updates to it won't be visible to it.
Shai
On Tue, Sep 1, 2009 at 3:54 PM, Ted Stockwell <em...@yahoo.com> wrote:
> Hi All,
>
> I am interested in using Lucene to index RDF (Resource Description Format)
> data.
> Ultimately I want to create a transactional interface to the data with
> proper transaction isolation.
> Therefore I am trying to educate myself on the details of index readers and
> writers, I am using v2.9rc2.
>
> One thing I have noticed is that it seems that Lucene is designed with the
> intent that changes to the index by a writer should not be visible to index
> readers until the writer commits its changes.
> However, one thing that I have noticed in the code is that when documents
> are deleted by a writer the writers change the .del (deleted documents)
> files in the segment in which the document exists.
> Thus, it appears that document deletions may become visible to readers
> *before* a write actually commits its changes.
>
> Am I correct about how document deletions work?
> Since I'm new to Lucene I'm wondering if I'm missing something...
>
>
> Thanks,
> ted stockwell
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>