You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Mark Harwood (JIRA)" <ji...@apache.org> on 2010/07/13 14:42:50 UTC
[jira] Created: (LUCENE-2536) Rollback doesn't preserve integrity
of original index
Rollback doesn't preserve integrity of original index
-----------------------------------------------------
Key: LUCENE-2536
URL: https://issues.apache.org/jira/browse/LUCENE-2536
Project: Lucene - Java
Issue Type: Bug
Components: Index
Affects Versions: 3.0.2
Environment: Windows XP pro
Reporter: Mark Harwood
Attachments: TestRollback.java
After several "updateDocuments" calls a rollback call does not return the index to the prior state.
This seems to occur if the number of updates exceeds the RAM buffer size i.e. when some flushing of updates occurs.
Test fails in Lucene 2.4, 2.9, 3.0.1 and 3.0.2
JUnit to follow.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Resolved: (LUCENE-2536) Rollback doesn't preserve integrity
of original index
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael McCandless resolved LUCENE-2536.
----------------------------------------
Fix Version/s: 2.4.2
2.9.4
3.0.3
3.1
4.0
Resolution: Fixed
Thanks Mark -- what a doozie!
> Rollback doesn't preserve integrity of original index
> -----------------------------------------------------
>
> Key: LUCENE-2536
> URL: https://issues.apache.org/jira/browse/LUCENE-2536
> Project: Lucene - Java
> Issue Type: Bug
> Components: Index
> Affects Versions: 3.0.2
> Environment: Windows XP pro
> Reporter: Mark Harwood
> Assignee: Michael McCandless
> Fix For: 2.4.2, 2.9.4, 3.0.3, 3.1, 4.0
>
> Attachments: LUCENE-2536.patch, LUCENE-2536.patch, TestRollback.java, TestRollback.java
>
>
> After several "updateDocuments" calls a rollback call does not return the index to the prior state.
> This seems to occur if the number of updates exceeds the RAM buffer size i.e. when some flushing of updates occurs.
> Test fails in Lucene 2.4, 2.9, 3.0.1 and 3.0.2
> JUnit to follow.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2536) Rollback doesn't preserve integrity
of original index
Posted by "Mark Harwood (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12887813#action_12887813 ]
Mark Harwood commented on LUCENE-2536:
--------------------------------------
Thanks for the rapid response Mike/Shai.
I'll need to patch a couple of production systems using 2.4.0 and 2.9.1 with this.
For the record - with this same change applied, the test passes OK on these older versions too.
> Rollback doesn't preserve integrity of original index
> -----------------------------------------------------
>
> Key: LUCENE-2536
> URL: https://issues.apache.org/jira/browse/LUCENE-2536
> Project: Lucene - Java
> Issue Type: Bug
> Components: Index
> Affects Versions: 3.0.2
> Environment: Windows XP pro
> Reporter: Mark Harwood
> Assignee: Michael McCandless
> Attachments: LUCENE-2536.patch, LUCENE-2536.patch, TestRollback.java, TestRollback.java
>
>
> After several "updateDocuments" calls a rollback call does not return the index to the prior state.
> This seems to occur if the number of updates exceeds the RAM buffer size i.e. when some flushing of updates occurs.
> Test fails in Lucene 2.4, 2.9, 3.0.1 and 3.0.2
> JUnit to follow.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Updated: (LUCENE-2536) Rollback doesn't preserve integrity
of original index
Posted by "Shai Erera (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Shai Erera updated LUCENE-2536:
-------------------------------
Attachment: TestRollback.java
It did not fail for me on 3.1, however did fail on 3.0.1.
I'm attaching a version of this test, which creates far less docs (5, instead of 10,000) and uses a RAMDirectory). The test still fails, which is expected. I assume if we want to make that an official unit test, the test case which passes should go away? (or not)
> Rollback doesn't preserve integrity of original index
> -----------------------------------------------------
>
> Key: LUCENE-2536
> URL: https://issues.apache.org/jira/browse/LUCENE-2536
> Project: Lucene - Java
> Issue Type: Bug
> Components: Index
> Affects Versions: 3.0.2
> Environment: Windows XP pro
> Reporter: Mark Harwood
> Attachments: TestRollback.java, TestRollback.java
>
>
> After several "updateDocuments" calls a rollback call does not return the index to the prior state.
> This seems to occur if the number of updates exceeds the RAM buffer size i.e. when some flushing of updates occurs.
> Test fails in Lucene 2.4, 2.9, 3.0.1 and 3.0.2
> JUnit to follow.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Assigned: (LUCENE-2536) Rollback doesn't preserve integrity
of original index
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael McCandless reassigned LUCENE-2536:
------------------------------------------
Assignee: Michael McCandless
> Rollback doesn't preserve integrity of original index
> -----------------------------------------------------
>
> Key: LUCENE-2536
> URL: https://issues.apache.org/jira/browse/LUCENE-2536
> Project: Lucene - Java
> Issue Type: Bug
> Components: Index
> Affects Versions: 3.0.2
> Environment: Windows XP pro
> Reporter: Mark Harwood
> Assignee: Michael McCandless
> Attachments: TestRollback.java, TestRollback.java
>
>
> After several "updateDocuments" calls a rollback call does not return the index to the prior state.
> This seems to occur if the number of updates exceeds the RAM buffer size i.e. when some flushing of updates occurs.
> Test fails in Lucene 2.4, 2.9, 3.0.1 and 3.0.2
> JUnit to follow.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Updated: (LUCENE-2536) Rollback doesn't preserve integrity
of original index
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael McCandless updated LUCENE-2536:
---------------------------------------
Attachment: LUCENE-2536.patch
Thanks Shai -- new patch.
> Rollback doesn't preserve integrity of original index
> -----------------------------------------------------
>
> Key: LUCENE-2536
> URL: https://issues.apache.org/jira/browse/LUCENE-2536
> Project: Lucene - Java
> Issue Type: Bug
> Components: Index
> Affects Versions: 3.0.2
> Environment: Windows XP pro
> Reporter: Mark Harwood
> Assignee: Michael McCandless
> Attachments: LUCENE-2536.patch, LUCENE-2536.patch, TestRollback.java, TestRollback.java
>
>
> After several "updateDocuments" calls a rollback call does not return the index to the prior state.
> This seems to occur if the number of updates exceeds the RAM buffer size i.e. when some flushing of updates occurs.
> Test fails in Lucene 2.4, 2.9, 3.0.1 and 3.0.2
> JUnit to follow.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2536) Rollback doesn't preserve integrity
of original index
Posted by "Mark Harwood (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12887757#action_12887757 ]
Mark Harwood commented on LUCENE-2536:
--------------------------------------
Looks like on the "close" call IndexWriter is flushing a superfluous ".del" file of buffered deletes.
> Rollback doesn't preserve integrity of original index
> -----------------------------------------------------
>
> Key: LUCENE-2536
> URL: https://issues.apache.org/jira/browse/LUCENE-2536
> Project: Lucene - Java
> Issue Type: Bug
> Components: Index
> Affects Versions: 3.0.2
> Environment: Windows XP pro
> Reporter: Mark Harwood
> Attachments: TestRollback.java
>
>
> After several "updateDocuments" calls a rollback call does not return the index to the prior state.
> This seems to occur if the number of updates exceeds the RAM buffer size i.e. when some flushing of updates occurs.
> Test fails in Lucene 2.4, 2.9, 3.0.1 and 3.0.2
> JUnit to follow.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2536) Rollback doesn't preserve integrity
of original index
Posted by "Shai Erera (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12887793#action_12887793 ]
Shai Erera commented on LUCENE-2536:
------------------------------------
Looks good to me. I think now that the test class contains just this method, we can consolidate all the helper ones into 1? It was required before 'cause there were two test cases. And we can remove the first assert in the test, since we're not checking here that the index contains the number of added docs after it was committed - there are plenty of other tests that do that.
> Rollback doesn't preserve integrity of original index
> -----------------------------------------------------
>
> Key: LUCENE-2536
> URL: https://issues.apache.org/jira/browse/LUCENE-2536
> Project: Lucene - Java
> Issue Type: Bug
> Components: Index
> Affects Versions: 3.0.2
> Environment: Windows XP pro
> Reporter: Mark Harwood
> Assignee: Michael McCandless
> Attachments: LUCENE-2536.patch, TestRollback.java, TestRollback.java
>
>
> After several "updateDocuments" calls a rollback call does not return the index to the prior state.
> This seems to occur if the number of updates exceeds the RAM buffer size i.e. when some flushing of updates occurs.
> Test fails in Lucene 2.4, 2.9, 3.0.1 and 3.0.2
> JUnit to follow.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2536) Rollback doesn't preserve integrity
of original index
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12887775#action_12887775 ]
Michael McCandless commented on LUCENE-2536:
--------------------------------------------
Duh -- in DocumentsWriter.abort, right after deletesInRAM.clear() we must add deletesFlush.clear(). So, indeed, any deletions pending (either due to .updateDoc or .deleteDocs) when segments were flushed, fail to be cleared and are then applied on close.
How awful!
> Rollback doesn't preserve integrity of original index
> -----------------------------------------------------
>
> Key: LUCENE-2536
> URL: https://issues.apache.org/jira/browse/LUCENE-2536
> Project: Lucene - Java
> Issue Type: Bug
> Components: Index
> Affects Versions: 3.0.2
> Environment: Windows XP pro
> Reporter: Mark Harwood
> Assignee: Michael McCandless
> Attachments: TestRollback.java, TestRollback.java
>
>
> After several "updateDocuments" calls a rollback call does not return the index to the prior state.
> This seems to occur if the number of updates exceeds the RAM buffer size i.e. when some flushing of updates occurs.
> Test fails in Lucene 2.4, 2.9, 3.0.1 and 3.0.2
> JUnit to follow.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Updated: (LUCENE-2536) Rollback doesn't preserve integrity
of original index
Posted by "Mark Harwood (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mark Harwood updated LUCENE-2536:
---------------------------------
Attachment: TestRollback.java
Example JUnit
> Rollback doesn't preserve integrity of original index
> -----------------------------------------------------
>
> Key: LUCENE-2536
> URL: https://issues.apache.org/jira/browse/LUCENE-2536
> Project: Lucene - Java
> Issue Type: Bug
> Components: Index
> Affects Versions: 3.0.2
> Environment: Windows XP pro
> Reporter: Mark Harwood
> Attachments: TestRollback.java
>
>
> After several "updateDocuments" calls a rollback call does not return the index to the prior state.
> This seems to occur if the number of updates exceeds the RAM buffer size i.e. when some flushing of updates occurs.
> Test fails in Lucene 2.4, 2.9, 3.0.1 and 3.0.2
> JUnit to follow.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Updated: (LUCENE-2536) Rollback doesn't preserve integrity
of original index
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael McCandless updated LUCENE-2536:
---------------------------------------
Attachment: LUCENE-2536.patch
Attached patch -- started from Shai's patch, tweaked it a bit, added the one line fix & CHANGES entry.
I see the test in fact failing on 3.x.
> Rollback doesn't preserve integrity of original index
> -----------------------------------------------------
>
> Key: LUCENE-2536
> URL: https://issues.apache.org/jira/browse/LUCENE-2536
> Project: Lucene - Java
> Issue Type: Bug
> Components: Index
> Affects Versions: 3.0.2
> Environment: Windows XP pro
> Reporter: Mark Harwood
> Assignee: Michael McCandless
> Attachments: LUCENE-2536.patch, TestRollback.java, TestRollback.java
>
>
> After several "updateDocuments" calls a rollback call does not return the index to the prior state.
> This seems to occur if the number of updates exceeds the RAM buffer size i.e. when some flushing of updates occurs.
> Test fails in Lucene 2.4, 2.9, 3.0.1 and 3.0.2
> JUnit to follow.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org