[jira] Commented: (LUCENE-1149) add XA transaction support

["Adrian Tarau (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/LUCENE-1149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12605705#action_12605705 ] 

Adrian Tarau commented on LUCENE-1149:
--------------------------------------

I had the same problem, so one year ago I implemented transactions on top of Lucene(I had an idea how to do it, but I also peeked a little bit in Compass sources). Basically I create a new index every time when a new transaction is started and this new index is made visible if commit is successful. Of course, you will still need to refresh your IndexReader(s). I planned also for a form of two-phase commit but right now it works pretty well doing everything manually - controlling when to commit/rollback/restore.

How it works :

1. start 2 transactions, one with MySQL one with Lucene(I have an abstract layer on top of both engines so the API calls are the same), commit first Lucene.
2. if Lucene fails, rollback MySQL.
3. if Lucene commit(precommit actually - everything is stored, I have a new optimized index, ready to be used) is successful, commit MySQL.
4. if MySQL fails, "cancel" Lucene transaction(remove index). There is a slightly chance to not being able to cancel the Lucene transaction(which is a new index). This index will hang there on disk/memory until the session is closed or the application is restarted.
5. if MySQL commit is successful, the newly created Lucene index is "activated". There is a slightly chance to not being able to "activate" Lucene transaction(application crashes, disk full so cannot write state on disk) - so in this case the global transaction is "in-doubt". In this case a system administrator must act and activate the index(the index is fine, but not active) - after he fixes any existing problems - or decided to clear the database(you need eventually a unique global transaction id stored in the database too, as an additional column). Your data is in the database, the application works fine but some entries are "invisible" when searching.

So is kind of a two-phase commit. 

> add XA transaction support
> --------------------------
>
>                 Key: LUCENE-1149
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1149
>             Project: Lucene - Java
>          Issue Type: New Feature
>          Components: Index
>            Reporter: robert engels
>
> Need to add XA transaction support to Lucene.
> Without XA support, it is difficult to keep disparate resources (e.g. database) in sync with the Lucene index.
> A review of the XA support added to Hibernate might be a good start (although Hibernate almost always uses a XA capable backing store database).
> It would be ideal to have a combined IndexReaderWriter instance, then create a XAIndexReaderWriter which wraps it.
> The implementation might be as simple as a XA log file which lists the XA transaction id, and the segments XXX number(s), since Lucene already allows you to rollback to a previous version (??? for sure, or does it only allow you to abort the current commit).
> If operating under a XA transaction, the no explicit commits or rollbacks should be allowed on the instance.
> The index would be committed during XA prepare(), and then if needed rolledback when requested. The XA commit() would be a no-op.
> There is a lot more to this but this should get the ball rolling.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org