some thoughts about adding transactions.

[Peter Veentjer - Anchor Men <p....@anchormen.nl>]

If have a question about transactions .
 
Lucene doesn`t support transactions but I find it very important and I think it is possible to add some kind of rollback/commit functionality to make sure the index doesn`t corrupt..
 
With lucene every segment is immutable (this is a perfect starting point), so after it has been created it will remain forever in a valid state. There are 3 ways to alter the index
1) deleting documents 
2) adding documents
3) optimization
 
If I delete a document, a del file appears (but doesn`t alter the segment because it is immutable). 
-if crash: the del files could be deleted to do a rollback.
-if succes: the del files finally will be used by the writer to skip those documents in the new segment.
 
If a new document is added, a new segment is created (finally). 
-if succes: the new segment is created and the old segments can be deleted.
-if crash: the new segment (maybe it`s corrupted) can be deleted to do a rollback.
 
If the index is optimized a new segment is created based on older segments.
-if succes: the old segments can be deleted.
-if crash: the new segment (maybe it`s corrupted) can be deleted to do a rollback.
 
With this information it wouldn`t be to much trouble to add some kind of rollback/transaction functionality?
 
And how about those 'per index' files? Can these be corrupted? Can these be removed and recreated succesfully? Would it be an idea to make copies of these files and restore them if the tranaction is rollbacked? 

---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org

Re: some thoughts about adding transactions.

[Scott Ganyo <sc...@ganyo.com>]

I didn't want to let this drop this on the floor, but I haven't had the 
time to craft a response to it either.  So, just for the record I agree 
that transactions would be nice.  I think that it is important that the 
solution address change visibility and concurrent transactions within 
multiple VMs.  Also, it should be backward compatible so that 
applications can run without transactions.  So, I think that a good 
solution is probably more complex than it initially looks...

S

On Jan 8, 2005, at 6:47 AM, Peter Veentjer - Anchor Men wrote:

> If have a question about transactions .
>
> Lucene doesn`t support transactions but I find it very important and I 
> think it is possible to add some kind of rollback/commit functionality 
> to make sure the index doesn`t corrupt..
>
> With lucene every segment is immutable (this is a perfect starting 
> point), so after it has been created it will remain forever in a valid 
> state. There are 3 ways to alter the index
> 1) deleting documents
> 2) adding documents
> 3) optimization
>
> If I delete a document, a del file appears (but doesn`t alter the 
> segment because it is immutable).
> -if crash: the del files could be deleted to do a rollback.
> -if succes: the del files finally will be used by the writer to skip 
> those documents in the new segment.
>
> If a new document is added, a new segment is created (finally).
> -if succes: the new segment is created and the old segments can be 
> deleted.
> -if crash: the new segment (maybe it`s corrupted) can be deleted to do 
> a rollback.
>
> If the index is optimized a new segment is created based on older 
> segments.
> -if succes: the old segments can be deleted.
> -if crash: the new segment (maybe it`s corrupted) can be deleted to do 
> a rollback.
>
> With this information it wouldn`t be to much trouble to add some kind 
> of rollback/transaction functionality?
>
> And how about those 'per index' files? Can these be corrupted? Can 
> these be removed and recreated succesfully? Would it be an idea to 
> make copies of these files and restore them if the tranaction is 
> rollbacked?
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: lucene-user-help@jakarta.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org