You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Sanne Grinovero (JIRA)" <ji...@apache.org> on 2009/11/25 22:20:39 UTC
[jira] Created: (LUCENE-2095) Document not guaranteed to be found
after write and commit
Document not guaranteed to be found after write and commit
----------------------------------------------------------
Key: LUCENE-2095
URL: https://issues.apache.org/jira/browse/LUCENE-2095
Project: Lucene - Java
Issue Type: Bug
Components: Index
Affects Versions: 2.9.1, 2.4.1
Environment: Linux 64bit
Reporter: Sanne Grinovero
after same email on developer list:
"I developed a stress test to assert that a new document containing a
specific term "X" is always found after a commit on the IndexWriter.
This works most of the time, but it fails under load in rare occasions.
I'm testing with 40 Threads, both with a SerialMergeScheduler and a
ConcurrentMergeScheduler, all sharing a common IndexWriter.
Attached testcase is using a RAMDirectory only, but I verified a
FSDirectory behaves in the same way so I don't believe it's the
Directory implementation or the MergeScheduler.
This test is slow, so I don't consider it a functional or unit test.
It might give false positives: it doesn't always fail, sorry I
couldn't find out how to make it more likely to happen, besides
scheduling it to run for a longer time."
I tested this to affect versions 2.4.1 and 2.9.1;
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2095) Document not guaranteed to be found
after write and commit
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12782715#action_12782715 ]
Michael McCandless commented on LUCENE-2095:
--------------------------------------------
I think the main cost is often fsync'ing the new files.
But I agree we should simply lock so only one thread can be in commit
at once. Concurrency inside commit (when fscyning) seems silly (it
used to be slightly more interesting when we had autoCommit=true).
We shouldn't use IW's lock. First, it's overkill (doing so would
unnecessarily block other sync'd ops from running). Second, it leads
to deadlock, at least in CMS (if it waits because too many merges are
running). I'll use a separate lock.
I'm not adding locking around the separate calls to prepareCommit then
commit. An app must ensure these two calls are always balanced.
Making flush unsynchronized would be great but I think wouldn't be
that big a gain in practice, unless we could make it truly unsync'd
even with adding new docs. Ie, if while we are flushing the last
segment, we can accept adding/deleting docs into a new ram buffer in
DW, that'd be quite interesting. But that's a big change!
> Document not guaranteed to be found after write and commit
> ----------------------------------------------------------
>
> Key: LUCENE-2095
> URL: https://issues.apache.org/jira/browse/LUCENE-2095
> Project: Lucene - Java
> Issue Type: Bug
> Components: Index
> Affects Versions: 2.4.1, 2.9.1
> Environment: Linux 64bit
> Reporter: Sanne Grinovero
> Assignee: Michael McCandless
> Fix For: 2.9.2, 3.0.1, 3.1
>
> Attachments: lucene-stresstest.patch
>
>
> after same email on developer list:
> "I developed a stress test to assert that a new document containing a
> specific term "X" is always found after a commit on the IndexWriter.
> This works most of the time, but it fails under load in rare occasions.
> I'm testing with 40 Threads, both with a SerialMergeScheduler and a
> ConcurrentMergeScheduler, all sharing a common IndexWriter.
> Attached testcase is using a RAMDirectory only, but I verified a
> FSDirectory behaves in the same way so I don't believe it's the
> Directory implementation or the MergeScheduler.
> This test is slow, so I don't consider it a functional or unit test.
> It might give false positives: it doesn't always fail, sorry I
> couldn't find out how to make it more likely to happen, besides
> scheduling it to run for a longer time."
> I tested this to affect versions 2.4.1 and 2.9.1;
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-2095) Document not guaranteed to be found
after write and commit
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael McCandless updated LUCENE-2095:
---------------------------------------
Attachment: LUCENE-2095.patch
Patch attached. I think it's ready to commit:
* Added a simple monitor lock during commit() to serialize threads.
* Boiled down the test case to a new test case in TestIndexWriter.
* Found & fixed the [unrelated] deadlock in TestCrash, and cutover
RAMDir's tracking of sizeInBytes to an AtomicLong, as discussed on
java-dev.
> Document not guaranteed to be found after write and commit
> ----------------------------------------------------------
>
> Key: LUCENE-2095
> URL: https://issues.apache.org/jira/browse/LUCENE-2095
> Project: Lucene - Java
> Issue Type: Bug
> Components: Index
> Affects Versions: 2.4.1, 2.9.1
> Environment: Linux 64bit
> Reporter: Sanne Grinovero
> Assignee: Michael McCandless
> Fix For: 2.9.2, 3.0.1, 3.1
>
> Attachments: LUCENE-2095.patch, lucene-stresstest.patch
>
>
> after same email on developer list:
> "I developed a stress test to assert that a new document containing a
> specific term "X" is always found after a commit on the IndexWriter.
> This works most of the time, but it fails under load in rare occasions.
> I'm testing with 40 Threads, both with a SerialMergeScheduler and a
> ConcurrentMergeScheduler, all sharing a common IndexWriter.
> Attached testcase is using a RAMDirectory only, but I verified a
> FSDirectory behaves in the same way so I don't believe it's the
> Directory implementation or the MergeScheduler.
> This test is slow, so I don't consider it a functional or unit test.
> It might give false positives: it doesn't always fail, sorry I
> couldn't find out how to make it more likely to happen, besides
> scheduling it to run for a longer time."
> I tested this to affect versions 2.4.1 and 2.9.1;
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2095) Document not guaranteed to be found
after write and commit
Posted by "Jason Rutherglen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12782659#action_12782659 ]
Jason Rutherglen commented on LUCENE-2095:
------------------------------------------
Why do we allow un-synchronized commit if doFlush is
synchronized? The main cost of commit is most likely in the
flush as it doesn't wait for merges? There's a todo above
doFlush indicating we may want to make it un-synchronized in the
future to not block merges, how tenable is this?
> Document not guaranteed to be found after write and commit
> ----------------------------------------------------------
>
> Key: LUCENE-2095
> URL: https://issues.apache.org/jira/browse/LUCENE-2095
> Project: Lucene - Java
> Issue Type: Bug
> Components: Index
> Affects Versions: 2.4.1, 2.9.1
> Environment: Linux 64bit
> Reporter: Sanne Grinovero
> Assignee: Michael McCandless
> Fix For: 2.9.2, 3.0.1, 3.1
>
> Attachments: lucene-stresstest.patch
>
>
> after same email on developer list:
> "I developed a stress test to assert that a new document containing a
> specific term "X" is always found after a commit on the IndexWriter.
> This works most of the time, but it fails under load in rare occasions.
> I'm testing with 40 Threads, both with a SerialMergeScheduler and a
> ConcurrentMergeScheduler, all sharing a common IndexWriter.
> Attached testcase is using a RAMDirectory only, but I verified a
> FSDirectory behaves in the same way so I don't believe it's the
> Directory implementation or the MergeScheduler.
> This test is slow, so I don't consider it a functional or unit test.
> It might give false positives: it doesn't always fail, sorry I
> couldn't find out how to make it more likely to happen, besides
> scheduling it to run for a longer time."
> I tested this to affect versions 2.4.1 and 2.9.1;
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2095) Document not guaranteed to be found
after write and commit
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12783222#action_12783222 ]
Uwe Schindler commented on LUCENE-2095:
---------------------------------------
Your patch only works for trunk and 3.0. If you want to fix it in 2.9 you must remove the j.u.concurrent* class usage.
> Document not guaranteed to be found after write and commit
> ----------------------------------------------------------
>
> Key: LUCENE-2095
> URL: https://issues.apache.org/jira/browse/LUCENE-2095
> Project: Lucene - Java
> Issue Type: Bug
> Components: Index
> Affects Versions: 2.4.1, 2.9.1
> Environment: Linux 64bit
> Reporter: Sanne Grinovero
> Assignee: Michael McCandless
> Fix For: 2.9.2, 3.0.1, 3.1
>
> Attachments: LUCENE-2095.patch, lucene-stresstest.patch
>
>
> after same email on developer list:
> "I developed a stress test to assert that a new document containing a
> specific term "X" is always found after a commit on the IndexWriter.
> This works most of the time, but it fails under load in rare occasions.
> I'm testing with 40 Threads, both with a SerialMergeScheduler and a
> ConcurrentMergeScheduler, all sharing a common IndexWriter.
> Attached testcase is using a RAMDirectory only, but I verified a
> FSDirectory behaves in the same way so I don't believe it's the
> Directory implementation or the MergeScheduler.
> This test is slow, so I don't consider it a functional or unit test.
> It might give false positives: it doesn't always fail, sorry I
> couldn't find out how to make it more likely to happen, besides
> scheduling it to run for a longer time."
> I tested this to affect versions 2.4.1 and 2.9.1;
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2095) Document not guaranteed to be found
after write and commit
Posted by "Sanne Grinovero (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12785943#action_12785943 ]
Sanne Grinovero commented on LUCENE-2095:
-----------------------------------------
Thanks a lot Michael, this makes my distributed testing reliable again :-)
I see you didn't apply my testcase, do you think it's not needed to have such a test?
If you need I could change it as you wish.
> Document not guaranteed to be found after write and commit
> ----------------------------------------------------------
>
> Key: LUCENE-2095
> URL: https://issues.apache.org/jira/browse/LUCENE-2095
> Project: Lucene - Java
> Issue Type: Bug
> Components: Index
> Affects Versions: 2.4.1, 2.9.1
> Environment: Linux 64bit
> Reporter: Sanne Grinovero
> Assignee: Michael McCandless
> Fix For: 2.9.2, 3.0.1, 3.1
>
> Attachments: LUCENE-2095.patch, lucene-stresstest.patch
>
>
> after same email on developer list:
> "I developed a stress test to assert that a new document containing a
> specific term "X" is always found after a commit on the IndexWriter.
> This works most of the time, but it fails under load in rare occasions.
> I'm testing with 40 Threads, both with a SerialMergeScheduler and a
> ConcurrentMergeScheduler, all sharing a common IndexWriter.
> Attached testcase is using a RAMDirectory only, but I verified a
> FSDirectory behaves in the same way so I don't believe it's the
> Directory implementation or the MergeScheduler.
> This test is slow, so I don't consider it a functional or unit test.
> It might give false positives: it doesn't always fail, sorry I
> couldn't find out how to make it more likely to happen, besides
> scheduling it to run for a longer time."
> I tested this to affect versions 2.4.1 and 2.9.1;
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2095) Document not guaranteed to be found
after write and commit
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12784974#action_12784974 ]
Michael McCandless commented on LUCENE-2095:
--------------------------------------------
Thanks Sanne!
> Document not guaranteed to be found after write and commit
> ----------------------------------------------------------
>
> Key: LUCENE-2095
> URL: https://issues.apache.org/jira/browse/LUCENE-2095
> Project: Lucene - Java
> Issue Type: Bug
> Components: Index
> Affects Versions: 2.4.1, 2.9.1
> Environment: Linux 64bit
> Reporter: Sanne Grinovero
> Assignee: Michael McCandless
> Fix For: 2.9.2, 3.0.1, 3.1
>
> Attachments: LUCENE-2095.patch, lucene-stresstest.patch
>
>
> after same email on developer list:
> "I developed a stress test to assert that a new document containing a
> specific term "X" is always found after a commit on the IndexWriter.
> This works most of the time, but it fails under load in rare occasions.
> I'm testing with 40 Threads, both with a SerialMergeScheduler and a
> ConcurrentMergeScheduler, all sharing a common IndexWriter.
> Attached testcase is using a RAMDirectory only, but I verified a
> FSDirectory behaves in the same way so I don't believe it's the
> Directory implementation or the MergeScheduler.
> This test is slow, so I don't consider it a functional or unit test.
> It might give false positives: it doesn't always fail, sorry I
> couldn't find out how to make it more likely to happen, besides
> scheduling it to run for a longer time."
> I tested this to affect versions 2.4.1 and 2.9.1;
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2095) Document not guaranteed to be found
after write and commit
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12782644#action_12782644 ]
Michael McCandless commented on LUCENE-2095:
--------------------------------------------
I can get this to fail fairly quickly, using 2 threads.
It's a thread safety issue of commit itself. If you only allow 1 thread into commit at a time I believe the issue won't happen.
It has to do with what thread #2 does when it enters commit and thread #1 is already in the process of committing. In this case thread #2 basically waits for thread #1 to finish and then returns, whereas it should start its own new commit.
> Document not guaranteed to be found after write and commit
> ----------------------------------------------------------
>
> Key: LUCENE-2095
> URL: https://issues.apache.org/jira/browse/LUCENE-2095
> Project: Lucene - Java
> Issue Type: Bug
> Components: Index
> Affects Versions: 2.4.1, 2.9.1
> Environment: Linux 64bit
> Reporter: Sanne Grinovero
> Attachments: lucene-stresstest.patch
>
>
> after same email on developer list:
> "I developed a stress test to assert that a new document containing a
> specific term "X" is always found after a commit on the IndexWriter.
> This works most of the time, but it fails under load in rare occasions.
> I'm testing with 40 Threads, both with a SerialMergeScheduler and a
> ConcurrentMergeScheduler, all sharing a common IndexWriter.
> Attached testcase is using a RAMDirectory only, but I verified a
> FSDirectory behaves in the same way so I don't believe it's the
> Directory implementation or the MergeScheduler.
> This test is slow, so I don't consider it a functional or unit test.
> It might give false positives: it doesn't always fail, sorry I
> couldn't find out how to make it more likely to happen, besides
> scheduling it to run for a longer time."
> I tested this to affect versions 2.4.1 and 2.9.1;
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Assigned: (LUCENE-2095) Document not guaranteed to be found
after write and commit
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael McCandless reassigned LUCENE-2095:
------------------------------------------
Assignee: Michael McCandless
> Document not guaranteed to be found after write and commit
> ----------------------------------------------------------
>
> Key: LUCENE-2095
> URL: https://issues.apache.org/jira/browse/LUCENE-2095
> Project: Lucene - Java
> Issue Type: Bug
> Components: Index
> Affects Versions: 2.4.1, 2.9.1
> Environment: Linux 64bit
> Reporter: Sanne Grinovero
> Assignee: Michael McCandless
> Attachments: lucene-stresstest.patch
>
>
> after same email on developer list:
> "I developed a stress test to assert that a new document containing a
> specific term "X" is always found after a commit on the IndexWriter.
> This works most of the time, but it fails under load in rare occasions.
> I'm testing with 40 Threads, both with a SerialMergeScheduler and a
> ConcurrentMergeScheduler, all sharing a common IndexWriter.
> Attached testcase is using a RAMDirectory only, but I verified a
> FSDirectory behaves in the same way so I don't believe it's the
> Directory implementation or the MergeScheduler.
> This test is slow, so I don't consider it a functional or unit test.
> It might give false positives: it doesn't always fail, sorry I
> couldn't find out how to make it more likely to happen, besides
> scheduling it to run for a longer time."
> I tested this to affect versions 2.4.1 and 2.9.1;
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-2095) Document not guaranteed to be found
after write and commit
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael McCandless updated LUCENE-2095:
---------------------------------------
Fix Version/s: 3.1
3.0.1
2.9.2
> Document not guaranteed to be found after write and commit
> ----------------------------------------------------------
>
> Key: LUCENE-2095
> URL: https://issues.apache.org/jira/browse/LUCENE-2095
> Project: Lucene - Java
> Issue Type: Bug
> Components: Index
> Affects Versions: 2.4.1, 2.9.1
> Environment: Linux 64bit
> Reporter: Sanne Grinovero
> Assignee: Michael McCandless
> Fix For: 2.9.2, 3.0.1, 3.1
>
> Attachments: lucene-stresstest.patch
>
>
> after same email on developer list:
> "I developed a stress test to assert that a new document containing a
> specific term "X" is always found after a commit on the IndexWriter.
> This works most of the time, but it fails under load in rare occasions.
> I'm testing with 40 Threads, both with a SerialMergeScheduler and a
> ConcurrentMergeScheduler, all sharing a common IndexWriter.
> Attached testcase is using a RAMDirectory only, but I verified a
> FSDirectory behaves in the same way so I don't believe it's the
> Directory implementation or the MergeScheduler.
> This test is slow, so I don't consider it a functional or unit test.
> It might give false positives: it doesn't always fail, sorry I
> couldn't find out how to make it more likely to happen, besides
> scheduling it to run for a longer time."
> I tested this to affect versions 2.4.1 and 2.9.1;
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2095) Document not guaranteed to be found
after write and commit
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12785965#action_12785965 ]
Michael McCandless commented on LUCENE-2095:
--------------------------------------------
bq. Thanks a lot Michael,
Thank you!
bq. this makes my distributed testing reliable again
I'm glad to hear it :) Thanks for bringing closure...
bq. I see you didn't apply my testcase
Actually, I did: I boiled your testcase down and added it, in TestIndexWriter.java (testCommitThreadSafety). The more tests the better!
> Document not guaranteed to be found after write and commit
> ----------------------------------------------------------
>
> Key: LUCENE-2095
> URL: https://issues.apache.org/jira/browse/LUCENE-2095
> Project: Lucene - Java
> Issue Type: Bug
> Components: Index
> Affects Versions: 2.4.1, 2.9.1
> Environment: Linux 64bit
> Reporter: Sanne Grinovero
> Assignee: Michael McCandless
> Fix For: 2.9.2, 3.0.1, 3.1
>
> Attachments: LUCENE-2095.patch, lucene-stresstest.patch
>
>
> after same email on developer list:
> "I developed a stress test to assert that a new document containing a
> specific term "X" is always found after a commit on the IndexWriter.
> This works most of the time, but it fails under load in rare occasions.
> I'm testing with 40 Threads, both with a SerialMergeScheduler and a
> ConcurrentMergeScheduler, all sharing a common IndexWriter.
> Attached testcase is using a RAMDirectory only, but I verified a
> FSDirectory behaves in the same way so I don't believe it's the
> Directory implementation or the MergeScheduler.
> This test is slow, so I don't consider it a functional or unit test.
> It might give false positives: it doesn't always fail, sorry I
> couldn't find out how to make it more likely to happen, besides
> scheduling it to run for a longer time."
> I tested this to affect versions 2.4.1 and 2.9.1;
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2095) Document not guaranteed to be found
after write and commit
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12783223#action_12783223 ]
Michael McCandless commented on LUCENE-2095:
--------------------------------------------
Thanks for reminding me -- I'll remove it on backport.
> Document not guaranteed to be found after write and commit
> ----------------------------------------------------------
>
> Key: LUCENE-2095
> URL: https://issues.apache.org/jira/browse/LUCENE-2095
> Project: Lucene - Java
> Issue Type: Bug
> Components: Index
> Affects Versions: 2.4.1, 2.9.1
> Environment: Linux 64bit
> Reporter: Sanne Grinovero
> Assignee: Michael McCandless
> Fix For: 2.9.2, 3.0.1, 3.1
>
> Attachments: LUCENE-2095.patch, lucene-stresstest.patch
>
>
> after same email on developer list:
> "I developed a stress test to assert that a new document containing a
> specific term "X" is always found after a commit on the IndexWriter.
> This works most of the time, but it fails under load in rare occasions.
> I'm testing with 40 Threads, both with a SerialMergeScheduler and a
> ConcurrentMergeScheduler, all sharing a common IndexWriter.
> Attached testcase is using a RAMDirectory only, but I verified a
> FSDirectory behaves in the same way so I don't believe it's the
> Directory implementation or the MergeScheduler.
> This test is slow, so I don't consider it a functional or unit test.
> It might give false positives: it doesn't always fail, sorry I
> couldn't find out how to make it more likely to happen, besides
> scheduling it to run for a longer time."
> I tested this to affect versions 2.4.1 and 2.9.1;
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Resolved: (LUCENE-2095) Document not guaranteed to be found
after write and commit
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael McCandless resolved LUCENE-2095.
----------------------------------------
Resolution: Fixed
> Document not guaranteed to be found after write and commit
> ----------------------------------------------------------
>
> Key: LUCENE-2095
> URL: https://issues.apache.org/jira/browse/LUCENE-2095
> Project: Lucene - Java
> Issue Type: Bug
> Components: Index
> Affects Versions: 2.4.1, 2.9.1
> Environment: Linux 64bit
> Reporter: Sanne Grinovero
> Assignee: Michael McCandless
> Fix For: 2.9.2, 3.0.1, 3.1
>
> Attachments: LUCENE-2095.patch, lucene-stresstest.patch
>
>
> after same email on developer list:
> "I developed a stress test to assert that a new document containing a
> specific term "X" is always found after a commit on the IndexWriter.
> This works most of the time, but it fails under load in rare occasions.
> I'm testing with 40 Threads, both with a SerialMergeScheduler and a
> ConcurrentMergeScheduler, all sharing a common IndexWriter.
> Attached testcase is using a RAMDirectory only, but I verified a
> FSDirectory behaves in the same way so I don't believe it's the
> Directory implementation or the MergeScheduler.
> This test is slow, so I don't consider it a functional or unit test.
> It might give false positives: it doesn't always fail, sorry I
> couldn't find out how to make it more likely to happen, besides
> scheduling it to run for a longer time."
> I tested this to affect versions 2.4.1 and 2.9.1;
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-2095) Document not guaranteed to be found
after write and commit
Posted by "Sanne Grinovero (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sanne Grinovero updated LUCENE-2095:
------------------------------------
Attachment: lucene-stresstest.patch
attaching the testcase, apply to version 2.9.1.
It's slow, please be patient.
> Document not guaranteed to be found after write and commit
> ----------------------------------------------------------
>
> Key: LUCENE-2095
> URL: https://issues.apache.org/jira/browse/LUCENE-2095
> Project: Lucene - Java
> Issue Type: Bug
> Components: Index
> Affects Versions: 2.4.1, 2.9.1
> Environment: Linux 64bit
> Reporter: Sanne Grinovero
> Attachments: lucene-stresstest.patch
>
>
> after same email on developer list:
> "I developed a stress test to assert that a new document containing a
> specific term "X" is always found after a commit on the IndexWriter.
> This works most of the time, but it fails under load in rare occasions.
> I'm testing with 40 Threads, both with a SerialMergeScheduler and a
> ConcurrentMergeScheduler, all sharing a common IndexWriter.
> Attached testcase is using a RAMDirectory only, but I verified a
> FSDirectory behaves in the same way so I don't believe it's the
> Directory implementation or the MergeScheduler.
> This test is slow, so I don't consider it a functional or unit test.
> It might give false positives: it doesn't always fail, sorry I
> couldn't find out how to make it more likely to happen, besides
> scheduling it to run for a longer time."
> I tested this to affect versions 2.4.1 and 2.9.1;
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org