You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Michael McCandless (JIRA)" <ji...@apache.org> on 2008/10/02 13:39:44 UTC
[jira] Created: (LUCENE-1411) Enable IndexWriter to open an
arbitrary commit point
Enable IndexWriter to open an arbitrary commit point
----------------------------------------------------
Key: LUCENE-1411
URL: https://issues.apache.org/jira/browse/LUCENE-1411
Project: Lucene - Java
Issue Type: Improvement
Components: Index
Affects Versions: 2.4
Reporter: Michael McCandless
Assignee: Michael McCandless
Priority: Minor
Fix For: 2.9
With a 2-phase commit involving multiple resources, each resource
first does its prepareCommit and then if all are successful they each
commit. If an exception or timeout/power loss is hit in any of the
resources during prepareCommit or commit, all of the resources must
then rollback.
But, because IndexWriter always opens the most recent commit, getting
Lucene to rollback after commit() has been called is not easy, unless
you make Lucene the last resource to commit. A simple workaround is
to simply remove the segments_N files of the newer commits but that's
sort of a hassle.
To fix this, we just need to add a ctor to IndexWriter that takes an
IndexCommit. We recently added this for IndexReader (LUCENE-1311) as
well. This ctor is definitely an "expert" method, and only makes
sense if you have a custom DeletionPolicy that preserves more than
just the most recent commit.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-1411) Enable IndexWriter to open an
arbitrary commit point
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael McCandless updated LUCENE-1411:
---------------------------------------
Attachment: LUCENE-1411.patch
Attached patch. I think it's ready to commit but I'll wait a few days.
When you open the IndexWriter on a prior commit, what actually happens
is the IndexWriter opens all commits, and then pulls in the specific
segments referenced by the prior commit into its in-memory
SegmentInfos instance. So this is analagous to doing a "reverse svn
merge" to undo a series of changes into your local checkout, but, not
committing that undo until IW.commit() is explicitly called.
This means it's still up to the deletion policy to decide what to do
with the "future" commits on opening the prior commit.
> Enable IndexWriter to open an arbitrary commit point
> ----------------------------------------------------
>
> Key: LUCENE-1411
> URL: https://issues.apache.org/jira/browse/LUCENE-1411
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Index
> Affects Versions: 2.4
> Reporter: Michael McCandless
> Assignee: Michael McCandless
> Priority: Minor
> Fix For: 2.9
>
> Attachments: LUCENE-1411.patch
>
>
> With a 2-phase commit involving multiple resources, each resource
> first does its prepareCommit and then if all are successful they each
> commit. If an exception or timeout/power loss is hit in any of the
> resources during prepareCommit or commit, all of the resources must
> then rollback.
> But, because IndexWriter always opens the most recent commit, getting
> Lucene to rollback after commit() has been called is not easy, unless
> you make Lucene the last resource to commit. A simple workaround is
> to simply remove the segments_N files of the newer commits but that's
> sort of a hassle.
> To fix this, we just need to add a ctor to IndexWriter that takes an
> IndexCommit. We recently added this for IndexReader (LUCENE-1311) as
> well. This ctor is definitely an "expert" method, and only makes
> sense if you have a custom DeletionPolicy that preserves more than
> just the most recent commit.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Resolved: (LUCENE-1411) Enable IndexWriter to open an
arbitrary commit point
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael McCandless resolved LUCENE-1411.
----------------------------------------
Resolution: Fixed
> Enable IndexWriter to open an arbitrary commit point
> ----------------------------------------------------
>
> Key: LUCENE-1411
> URL: https://issues.apache.org/jira/browse/LUCENE-1411
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Index
> Affects Versions: 2.4
> Reporter: Michael McCandless
> Assignee: Michael McCandless
> Priority: Minor
> Fix For: 2.9
>
> Attachments: LUCENE-1411.patch
>
>
> With a 2-phase commit involving multiple resources, each resource
> first does its prepareCommit and then if all are successful they each
> commit. If an exception or timeout/power loss is hit in any of the
> resources during prepareCommit or commit, all of the resources must
> then rollback.
> But, because IndexWriter always opens the most recent commit, getting
> Lucene to rollback after commit() has been called is not easy, unless
> you make Lucene the last resource to commit. A simple workaround is
> to simply remove the segments_N files of the newer commits but that's
> sort of a hassle.
> To fix this, we just need to add a ctor to IndexWriter that takes an
> IndexCommit. We recently added this for IndexReader (LUCENE-1311) as
> well. This ctor is definitely an "expert" method, and only makes
> sense if you have a custom DeletionPolicy that preserves more than
> just the most recent commit.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org