You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Robert Muir (JIRA)" <ji...@apache.org> on 2010/11/10 16:08:13 UTC
[jira] Created: (LUCENE-2751) add LuceneTestCase.newSearcher()
add LuceneTestCase.newSearcher()
--------------------------------
Key: LUCENE-2751
URL: https://issues.apache.org/jira/browse/LUCENE-2751
Project: Lucene - Java
Issue Type: Test
Components: Build
Reporter: Robert Muir
Fix For: 3.1, 4.0
Most tests in the search package don't care about what kind of searcher they use.
we should randomly use MultiSearcher or ParallelMultiSearcher sometimes in tests.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Updated: (LUCENE-2751) add LuceneTestCase.newSearcher()
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael McCandless updated LUCENE-2751:
---------------------------------------
Attachment: LUCENE-2751.patch
I think the attached patch fixes the thread safety issue w/ FieldCacheImpl.get -- I just made the non-noop validate methods of the *ValuesCreator sync'd.
> add LuceneTestCase.newSearcher()
> --------------------------------
>
> Key: LUCENE-2751
> URL: https://issues.apache.org/jira/browse/LUCENE-2751
> Project: Lucene - Java
> Issue Type: Test
> Components: Build
> Reporter: Robert Muir
> Fix For: 3.1, 4.0
>
> Attachments: LUCENE-2751.patch, LUCENE-2751.patch
>
>
> Most tests in the search package don't care about what kind of searcher they use.
> we should randomly use MultiSearcher or ParallelMultiSearcher sometimes in tests.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Updated: (LUCENE-2751) add LuceneTestCase.newSearcher()
Posted by "Yonik Seeley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Yonik Seeley updated LUCENE-2751:
---------------------------------
Attachment: LUCENE-2751.patch
Whew, this code is complex, and I don't really have a solution I like.
Here's a patch to IntValuesCreator that's only for brainstorming purposes...
- synchronize on the entry instead of the creator so more than one segment may be loaded concurrently. This is needed to publish objects safely.
- when guessing the parser, don't set it until it succeeds - basically split up setParserAndResetCounts() to set the counts first, but the parser last.
- when setting/checking the parser, we need to synchronize on the creator
I also just realized that the mere act of checking the cache creates objects (see the base getCacheKey() ).
> add LuceneTestCase.newSearcher()
> --------------------------------
>
> Key: LUCENE-2751
> URL: https://issues.apache.org/jira/browse/LUCENE-2751
> Project: Lucene - Java
> Issue Type: Test
> Components: Build
> Reporter: Robert Muir
> Fix For: 4.0
>
> Attachments: LUCENE-2751.patch, LUCENE-2751.patch, LUCENE-2751.patch
>
>
> Most tests in the search package don't care about what kind of searcher they use.
> we should randomly use MultiSearcher or ParallelMultiSearcher sometimes in tests.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Updated: (LUCENE-2751) add LuceneTestCase.newSearcher()
Posted by "Robert Muir (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Muir updated LUCENE-2751:
--------------------------------
Attachment: LUCENE-2751_branch3x.patch
It seems this uncovered some monsters in branch_3x (unless i severely screwed things up).
Attached is a patch of my merge... there are problems: some tests totally hang (e.g. TestLazyProxSkipping), and if TestSort gets a parallel executor, testNormalizedScores fails:
{noformat}
[junit] ------------- Standard Error -----------------
[junit] NOTE: reproduce with: ant test -Dtestcase=TestSort -Dtestmethod=testNormalizedScores -Dtests.seed=367124575585283127:1927326607030239825
[junit] NOTE: test params are: locale=zh_SG, timezone=America/Tijuana
[junit] NOTE: all tests run in this JVM:
[junit] [TestSort]
[junit] NOTE: Windows Vista 6.0 x86/Sun Microsystems Inc. 1.6.0_23 (32-bit)/cpus=4,threads=1,free=12557672,total=16252928
[junit] ------------- ---------------- ---------------
[junit] Testcase: testNormalizedScores(org.apache.lucene.search.TestSort): FAILED
[junit] expected:<1.6163856983184814> but was:<NaN>
[junit] junit.framework.AssertionFailedError: expected:<1.6163856983184814> but was:<NaN>
[junit] at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1045)
[junit] at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:977)
[junit] at org.apache.lucene.search.TestSort.assertSameValues(TestSort.java:1094)
[junit] at org.apache.lucene.search.TestSort.testNormalizedScores(TestSort.java:668)
[junit]
[junit]
[junit] Test org.apache.lucene.search.TestSort FAILED
{noformat}
Posting the patch of the merge so we can hopefully debug through these.
> add LuceneTestCase.newSearcher()
> --------------------------------
>
> Key: LUCENE-2751
> URL: https://issues.apache.org/jira/browse/LUCENE-2751
> Project: Lucene - Java
> Issue Type: Test
> Components: Build
> Reporter: Robert Muir
> Assignee: Robert Muir
> Fix For: 3.1, 4.0
>
> Attachments: LUCENE-2751.patch, LUCENE-2751.patch, LUCENE-2751.patch, LUCENE-2751.patch, LUCENE-2751.patch, LUCENE-2751.patch, LUCENE-2751_branch3x.patch
>
>
> Most tests in the search package don't care about what kind of searcher they use.
> we should randomly use MultiSearcher or ParallelMultiSearcher sometimes in tests.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Updated: (LUCENE-2751) add LuceneTestCase.newSearcher()
Posted by "Robert Muir (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Muir updated LUCENE-2751:
--------------------------------
Attachment: LUCENE-2751.patch
patch, with all tests fixed to close their searchers. tests are passing but there are two problems:
# see the nocommit in IndexSearcher, when it sets itself as its own child, this crazy recursive thing happens. I don't understand why this code is needed, it shouldn't set itself as its own child?
# some tests in spans fail when they are run with multiple threads. don't know whats going on here:
{noformat}
junit-sequential:
[junit] Testsuite: org.apache.lucene.search.spans.TestSpansAdvanced2
[junit] Tests run: 4, Failures: 2, Errors: 0, Time elapsed: 0.833 sec
[junit]
[junit] ------------- Standard Output ---------------
[junit] 0 warning, expected score: 0.625, actual 0.23211958
[junit] 0.23211958 = (MATCH) fieldWeight(TEXT:should in 1), product of:
[junit] 1.0 = tf(phraseFreq=1.0)
[junit] 0.37139133 = idf(TEXT: should=14)
[junit] 0.625 = fieldNorm(field=TEXT, doc=1)
[junit]
[junit] 0 warning, expected score: 1.0191123, actual 0.69635624
[junit] 0.69635624 = (MATCH) sum of:
[junit] 0.041781675 = (MATCH) weight(TEXT:should in 3), product of:
[junit] 0.24494985 = queryWeight(TEXT:should), product of:
[junit] 0.37139133 = idf(TEXT: should=14)
[junit] 0.6595465 = queryNorm
[junit] 0.17057237 = (MATCH) fieldWeight(TEXT:should in 3), product of:
[junit] 1.2247449 = tf(phraseFreq=1.5)
[junit] 0.37139133 = idf(TEXT: should=14)
[junit] 0.375 = fieldNorm(field=TEXT, doc=3)
[junit] 0.6545746 = (MATCH) weight(TEXT:we in 3), product of:
[junit] 0.9695357 = queryWeight(TEXT:we), product of:
[junit] 1.4700036 = idf(TEXT: we=4)
[junit] 0.6595465 = queryNorm
[junit] 0.6751423 = (MATCH) fieldWeight(TEXT:we in 3), product of:
[junit] 1.2247449 = tf(phraseFreq=1.5)
[junit] 1.4700036 = idf(TEXT: we=4)
[junit] 0.375 = fieldNorm(field=TEXT, doc=3)
[junit]
[junit] ------------- ---------------- ---------------
[junit] ------------- Standard Error -----------------
[junit] NOTE: reproduce with: ant test -Dtestcase=TestSpansAdvanced2 -Dtestmethod=testSingleSpanQuery -Dtests.seed=5923986663359901096:-8292211890903536598
[junit] NOTE: reproduce with: ant test -Dtestcase=TestSpansAdvanced2 -Dtestmethod=testMultipleDifferentSpanQueries -Dtests.seed=5923986663359901096:-2877104144909248514
[junit] NOTE: test params are: codec=RandomCodecProvider: {TEXT=MockVariableIntBlock(baseBlockSize=59), ID=Pulsing(freqCutoff=12)}, locale=pl, timezone=Africa/Lubumbashi
[junit] NOTE: all tests run in this JVM:
[junit] [TestSpansAdvanced2]
[junit] NOTE: Windows Vista 6.0 x86/Sun Microsystems Inc. 1.6.0_23 (32-bit)/cpus=4,threads=1,free=6497408,total=28114944
[junit] ------------- ---------------- ---------------
[junit] Testcase: testSingleSpanQuery(org.apache.lucene.search.spans.TestSpansAdvanced2): FAILED
[junit] expected:<0.625> but was:<0.2321195751428604>
[junit] junit.framework.AssertionFailedError: expected:<0.625> but was:<0.2321195751428604>
[junit] at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1183)
[junit] at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1115)
[junit] at org.apache.lucene.search.spans.TestSpansAdvanced.assertHits(TestSpansAdvanced.java:172)
[junit] at org.apache.lucene.search.spans.TestSpansAdvanced2.testSingleSpanQuery(TestSpansAdvanced2.java:93)
[junit]
[junit]
[junit] Testcase: testMultipleDifferentSpanQueries(org.apache.lucene.search.spans.TestSpansAdvanced2): FAILED
[junit] expected:<1.0191123485565186> but was:<0.6963562369346619>
[junit] junit.framework.AssertionFailedError: expected:<1.0191123485565186> but was:<0.6963562369346619>
[junit] at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1183)
[junit] at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1115)
[junit] at org.apache.lucene.search.spans.TestSpansAdvanced.assertHits(TestSpansAdvanced.java:172)
[junit] at org.apache.lucene.search.spans.TestSpansAdvanced2.testMultipleDifferentSpanQueries(TestSpansAdvanced2.java:113)
[junit]
[junit]
[junit] Test org.apache.lucene.search.spans.TestSpansAdvanced2 FAILED
{noformat}
> add LuceneTestCase.newSearcher()
> --------------------------------
>
> Key: LUCENE-2751
> URL: https://issues.apache.org/jira/browse/LUCENE-2751
> Project: Lucene - Java
> Issue Type: Test
> Components: Build
> Reporter: Robert Muir
> Fix For: 4.0
>
> Attachments: LUCENE-2751.patch, LUCENE-2751.patch, LUCENE-2751.patch, LUCENE-2751.patch, LUCENE-2751.patch, LUCENE-2751.patch
>
>
> Most tests in the search package don't care about what kind of searcher they use.
> we should randomly use MultiSearcher or ParallelMultiSearcher sometimes in tests.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2751) add LuceneTestCase.newSearcher()
Posted by "Robert Muir (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12980808#action_12980808 ]
Robert Muir commented on LUCENE-2751:
-------------------------------------
There is a downside to this whole issue of course... i think its going to be harder to reproduce test fails since we will be using more multithreading.
But I think its a worthwhile tradeoff in being able to detect more thread-safety bugs.
If it becomes a huge hassle, we could always disable it by default or enable only with a flag or something like that.
> add LuceneTestCase.newSearcher()
> --------------------------------
>
> Key: LUCENE-2751
> URL: https://issues.apache.org/jira/browse/LUCENE-2751
> Project: Lucene - Java
> Issue Type: Test
> Components: Build
> Reporter: Robert Muir
> Fix For: 3.1, 4.0
>
> Attachments: LUCENE-2751.patch, LUCENE-2751.patch
>
>
> Most tests in the search package don't care about what kind of searcher they use.
> we should randomly use MultiSearcher or ParallelMultiSearcher sometimes in tests.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Updated: (LUCENE-2751) add LuceneTestCase.newSearcher()
Posted by "Robert Muir (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Muir updated LUCENE-2751:
--------------------------------
Fix Version/s: (was: 3.1)
I'd like to push this out from 3.1 also.
Of the two problems i found, the concurrency issue only affected 4.0's fieldcache.
Additionally the combine() is now fixed and has an explicit test.
> add LuceneTestCase.newSearcher()
> --------------------------------
>
> Key: LUCENE-2751
> URL: https://issues.apache.org/jira/browse/LUCENE-2751
> Project: Lucene - Java
> Issue Type: Test
> Components: Build
> Reporter: Robert Muir
> Fix For: 4.0
>
> Attachments: LUCENE-2751.patch, LUCENE-2751.patch
>
>
> Most tests in the search package don't care about what kind of searcher they use.
> we should randomly use MultiSearcher or ParallelMultiSearcher sometimes in tests.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Updated: (LUCENE-2751) add LuceneTestCase.newSearcher()
Posted by "Robert Muir (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Muir updated LUCENE-2751:
--------------------------------
Attachment: LUCENE-2751.patch
really quick hacked-up patch for the core o.a.l.search tests.
some tests fail.
> add LuceneTestCase.newSearcher()
> --------------------------------
>
> Key: LUCENE-2751
> URL: https://issues.apache.org/jira/browse/LUCENE-2751
> Project: Lucene - Java
> Issue Type: Test
> Components: Build
> Reporter: Robert Muir
> Fix For: 3.1, 4.0
>
> Attachments: LUCENE-2751.patch
>
>
> Most tests in the search package don't care about what kind of searcher they use.
> we should randomly use MultiSearcher or ParallelMultiSearcher sometimes in tests.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2751) add LuceneTestCase.newSearcher()
Posted by "Robert Muir (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12989846#comment-12989846 ]
Robert Muir commented on LUCENE-2751:
-------------------------------------
Thanks Simon! I'll commit this and take a look at backporting this monster to 3.x
> add LuceneTestCase.newSearcher()
> --------------------------------
>
> Key: LUCENE-2751
> URL: https://issues.apache.org/jira/browse/LUCENE-2751
> Project: Lucene - Java
> Issue Type: Test
> Components: Build
> Reporter: Robert Muir
> Assignee: Robert Muir
> Fix For: 3.1, 4.0
>
> Attachments: LUCENE-2751.patch, LUCENE-2751.patch, LUCENE-2751.patch, LUCENE-2751.patch, LUCENE-2751.patch, LUCENE-2751.patch
>
>
> Most tests in the search package don't care about what kind of searcher they use.
> we should randomly use MultiSearcher or ParallelMultiSearcher sometimes in tests.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2751) add LuceneTestCase.newSearcher()
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12980672#action_12980672 ]
Michael McCandless commented on LUCENE-2751:
--------------------------------------------
bq. Hmmm, it looks like the committed patch serializes loading of caches of multiple segments (for the same field?)
Ugh, you're right. I had thought validate was "only" used after initial creation (eg, "typically" to add valid bits in), but in fact, create() calls validate().
Yonik do you have a patch in mind to fix the root cause correctly?
I have to say... the new FieldCache code is rather hairy.
> add LuceneTestCase.newSearcher()
> --------------------------------
>
> Key: LUCENE-2751
> URL: https://issues.apache.org/jira/browse/LUCENE-2751
> Project: Lucene - Java
> Issue Type: Test
> Components: Build
> Reporter: Robert Muir
> Fix For: 3.1, 4.0
>
> Attachments: LUCENE-2751.patch, LUCENE-2751.patch
>
>
> Most tests in the search package don't care about what kind of searcher they use.
> we should randomly use MultiSearcher or ParallelMultiSearcher sometimes in tests.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Updated: (LUCENE-2751) add LuceneTestCase.newSearcher()
Posted by "Robert Muir (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Muir updated LUCENE-2751:
--------------------------------
Attachment: LUCENE-2751.patch
updated newSearcher tests pass for trunk.
note, lots of tests will warn about leftover threads, because they still need to be fixed to close their IndexSearchers (i fixed a few, but not all yet)
> add LuceneTestCase.newSearcher()
> --------------------------------
>
> Key: LUCENE-2751
> URL: https://issues.apache.org/jira/browse/LUCENE-2751
> Project: Lucene - Java
> Issue Type: Test
> Components: Build
> Reporter: Robert Muir
> Fix For: 4.0
>
> Attachments: LUCENE-2751.patch, LUCENE-2751.patch, LUCENE-2751.patch, LUCENE-2751.patch
>
>
> Most tests in the search package don't care about what kind of searcher they use.
> we should randomly use MultiSearcher or ParallelMultiSearcher sometimes in tests.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2751) add LuceneTestCase.newSearcher()
Posted by "Yonik Seeley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12980484#action_12980484 ]
Yonik Seeley commented on LUCENE-2751:
--------------------------------------
Hmmm, it looks like the committed patch serializes loading of caches of multiple segments (for the same field?)
I had thought that the root of the problem was only when "parser" wasn't specified and the creator attempted to guess? If so, it seems we could synchronize only when the parser was not originally specified.
> add LuceneTestCase.newSearcher()
> --------------------------------
>
> Key: LUCENE-2751
> URL: https://issues.apache.org/jira/browse/LUCENE-2751
> Project: Lucene - Java
> Issue Type: Test
> Components: Build
> Reporter: Robert Muir
> Fix For: 3.1, 4.0
>
> Attachments: LUCENE-2751.patch, LUCENE-2751.patch
>
>
> Most tests in the search package don't care about what kind of searcher they use.
> we should randomly use MultiSearcher or ParallelMultiSearcher sometimes in tests.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2751) add LuceneTestCase.newSearcher()
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12981220#action_12981220 ]
Michael McCandless commented on LUCENE-2751:
--------------------------------------------
bq. There is a downside to this whole issue of course... i think its going to be harder to reproduce test fails since we will be using more multithreading.
Right.
But I think this (losing reproducibility sometimes) is the lesser evil? Ie, making sure we tease out thread safety bugs trumps reproducibility...
> add LuceneTestCase.newSearcher()
> --------------------------------
>
> Key: LUCENE-2751
> URL: https://issues.apache.org/jira/browse/LUCENE-2751
> Project: Lucene - Java
> Issue Type: Test
> Components: Build
> Reporter: Robert Muir
> Fix For: 3.1, 4.0
>
> Attachments: LUCENE-2751.patch, LUCENE-2751.patch
>
>
> Most tests in the search package don't care about what kind of searcher they use.
> we should randomly use MultiSearcher or ParallelMultiSearcher sometimes in tests.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Resolved: (LUCENE-2751) add LuceneTestCase.newSearcher()
Posted by "Simon Willnauer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Simon Willnauer resolved LUCENE-2751.
-------------------------------------
Resolution: Fixed
bq. I think we are good to resolve this?
yeah - no failure during the night! Nice!
> add LuceneTestCase.newSearcher()
> --------------------------------
>
> Key: LUCENE-2751
> URL: https://issues.apache.org/jira/browse/LUCENE-2751
> Project: Lucene - Java
> Issue Type: Test
> Components: Build
> Reporter: Robert Muir
> Assignee: Robert Muir
> Fix For: 3.1, 4.0
>
> Attachments: LUCENE-2751.patch, LUCENE-2751.patch, LUCENE-2751.patch, LUCENE-2751.patch, LUCENE-2751.patch, LUCENE-2751.patch, LUCENE-2751_branch3x.patch
>
>
> Most tests in the search package don't care about what kind of searcher they use.
> we should randomly use MultiSearcher or ParallelMultiSearcher sometimes in tests.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2751) add LuceneTestCase.newSearcher()
Posted by "Simon Willnauer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12990221#comment-12990221 ]
Simon Willnauer commented on LUCENE-2751:
-----------------------------------------
Robert, I committed a fix with revision 1066904 on 3x I think we should wait a bit and close this issue.
simon
> add LuceneTestCase.newSearcher()
> --------------------------------
>
> Key: LUCENE-2751
> URL: https://issues.apache.org/jira/browse/LUCENE-2751
> Project: Lucene - Java
> Issue Type: Test
> Components: Build
> Reporter: Robert Muir
> Assignee: Robert Muir
> Fix For: 3.1, 4.0
>
> Attachments: LUCENE-2751.patch, LUCENE-2751.patch, LUCENE-2751.patch, LUCENE-2751.patch, LUCENE-2751.patch, LUCENE-2751.patch, LUCENE-2751_branch3x.patch
>
>
> Most tests in the search package don't care about what kind of searcher they use.
> we should randomly use MultiSearcher or ParallelMultiSearcher sometimes in tests.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2751) add LuceneTestCase.newSearcher()
Posted by "Robert Muir (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12930641#action_12930641 ]
Robert Muir commented on LUCENE-2751:
-------------------------------------
As mentioned on the user list, we can do something like this here,
and fix all the bugs and corner cases (probably with a lot of hairiness)...
or we could deprecate and remove MultiSearcher.
> add LuceneTestCase.newSearcher()
> --------------------------------
>
> Key: LUCENE-2751
> URL: https://issues.apache.org/jira/browse/LUCENE-2751
> Project: Lucene - Java
> Issue Type: Test
> Components: Build
> Reporter: Robert Muir
> Fix For: 3.1, 4.0
>
> Attachments: LUCENE-2751.patch
>
>
> Most tests in the search package don't care about what kind of searcher they use.
> we should randomly use MultiSearcher or ParallelMultiSearcher sometimes in tests.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2751) add LuceneTestCase.newSearcher()
Posted by "Robert Muir (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12989944#comment-12989944 ]
Robert Muir commented on LUCENE-2751:
-------------------------------------
I fixed the hangs, it was due to a similar recursion bug that trunk had (IndexSearcher
setting itself as a sub).
I committed the patch, but i added an @Ignore to the testNormalizeScores:
{noformat}
[junit] Testsuite: org.apache.lucene.search.TestSort
[junit] Tests run: 26, Failures: 0, Errors: 0, Time elapsed: 2.471 sec
[junit]
[junit] ------------- Standard Error -----------------
[junit] NOTE: Ignoring test method 'testNormalizedScores': Fix me! Fails if one of the subs is a threaded indexsearcher
[junit] ------------- ---------------- ---------------
{noformat}
I think we should get to the bottom of why this one fails... I'll keep the issue open.
> add LuceneTestCase.newSearcher()
> --------------------------------
>
> Key: LUCENE-2751
> URL: https://issues.apache.org/jira/browse/LUCENE-2751
> Project: Lucene - Java
> Issue Type: Test
> Components: Build
> Reporter: Robert Muir
> Assignee: Robert Muir
> Fix For: 3.1, 4.0
>
> Attachments: LUCENE-2751.patch, LUCENE-2751.patch, LUCENE-2751.patch, LUCENE-2751.patch, LUCENE-2751.patch, LUCENE-2751.patch, LUCENE-2751_branch3x.patch
>
>
> Most tests in the search package don't care about what kind of searcher they use.
> we should randomly use MultiSearcher or ParallelMultiSearcher sometimes in tests.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Assigned: (LUCENE-2751) add LuceneTestCase.newSearcher()
Posted by "Robert Muir (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Muir reassigned LUCENE-2751:
-----------------------------------
Assignee: Robert Muir
> add LuceneTestCase.newSearcher()
> --------------------------------
>
> Key: LUCENE-2751
> URL: https://issues.apache.org/jira/browse/LUCENE-2751
> Project: Lucene - Java
> Issue Type: Test
> Components: Build
> Reporter: Robert Muir
> Assignee: Robert Muir
> Fix For: 3.1, 4.0
>
> Attachments: LUCENE-2751.patch, LUCENE-2751.patch, LUCENE-2751.patch, LUCENE-2751.patch, LUCENE-2751.patch, LUCENE-2751.patch
>
>
> Most tests in the search package don't care about what kind of searcher they use.
> we should randomly use MultiSearcher or ParallelMultiSearcher sometimes in tests.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Updated: (LUCENE-2751) add LuceneTestCase.newSearcher()
Posted by "Robert Muir (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Muir updated LUCENE-2751:
--------------------------------
Attachment: LUCENE-2751.patch
updated patch with more tests fixed to close their searchers, some additional debugging, and a big nasty warning instead of 1 second join if you forget to close your searcher.
> add LuceneTestCase.newSearcher()
> --------------------------------
>
> Key: LUCENE-2751
> URL: https://issues.apache.org/jira/browse/LUCENE-2751
> Project: Lucene - Java
> Issue Type: Test
> Components: Build
> Reporter: Robert Muir
> Fix For: 4.0
>
> Attachments: LUCENE-2751.patch, LUCENE-2751.patch, LUCENE-2751.patch, LUCENE-2751.patch, LUCENE-2751.patch
>
>
> Most tests in the search package don't care about what kind of searcher they use.
> we should randomly use MultiSearcher or ParallelMultiSearcher sometimes in tests.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Updated: (LUCENE-2751) add LuceneTestCase.newSearcher()
Posted by "Robert Muir (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Muir updated LUCENE-2751:
--------------------------------
Fix Version/s: 3.1
adding 3.1, given the issues we found in trunk, i think we want to know this stuff works.
> add LuceneTestCase.newSearcher()
> --------------------------------
>
> Key: LUCENE-2751
> URL: https://issues.apache.org/jira/browse/LUCENE-2751
> Project: Lucene - Java
> Issue Type: Test
> Components: Build
> Reporter: Robert Muir
> Fix For: 3.1, 4.0
>
> Attachments: LUCENE-2751.patch, LUCENE-2751.patch, LUCENE-2751.patch, LUCENE-2751.patch, LUCENE-2751.patch, LUCENE-2751.patch
>
>
> Most tests in the search package don't care about what kind of searcher they use.
> we should randomly use MultiSearcher or ParallelMultiSearcher sometimes in tests.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2751) add LuceneTestCase.newSearcher()
Posted by "Simon Willnauer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12989839#comment-12989839 ]
Simon Willnauer commented on LUCENE-2751:
-----------------------------------------
I just fixed the issues you have seen here on LUCENE-2831 all tests pass with that latest patch here.
> add LuceneTestCase.newSearcher()
> --------------------------------
>
> Key: LUCENE-2751
> URL: https://issues.apache.org/jira/browse/LUCENE-2751
> Project: Lucene - Java
> Issue Type: Test
> Components: Build
> Reporter: Robert Muir
> Assignee: Robert Muir
> Fix For: 3.1, 4.0
>
> Attachments: LUCENE-2751.patch, LUCENE-2751.patch, LUCENE-2751.patch, LUCENE-2751.patch, LUCENE-2751.patch, LUCENE-2751.patch
>
>
> Most tests in the search package don't care about what kind of searcher they use.
> we should randomly use MultiSearcher or ParallelMultiSearcher sometimes in tests.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2751) add LuceneTestCase.newSearcher()
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12980758#action_12980758 ]
Michael McCandless commented on LUCENE-2751:
--------------------------------------------
bq. I think this would be preferred before we go optimizing synchronization, because otherwise how do we know if its correct?
+1
> add LuceneTestCase.newSearcher()
> --------------------------------
>
> Key: LUCENE-2751
> URL: https://issues.apache.org/jira/browse/LUCENE-2751
> Project: Lucene - Java
> Issue Type: Test
> Components: Build
> Reporter: Robert Muir
> Fix For: 3.1, 4.0
>
> Attachments: LUCENE-2751.patch, LUCENE-2751.patch
>
>
> Most tests in the search package don't care about what kind of searcher they use.
> we should randomly use MultiSearcher or ParallelMultiSearcher sometimes in tests.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2751) add LuceneTestCase.newSearcher()
Posted by "Robert Muir (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12990319#comment-12990319 ]
Robert Muir commented on LUCENE-2751:
-------------------------------------
I think we are good to resolve this? I expect/hope for random
fails in the future that we will content with on a case by case basis!
Thanks for fixing it!
> add LuceneTestCase.newSearcher()
> --------------------------------
>
> Key: LUCENE-2751
> URL: https://issues.apache.org/jira/browse/LUCENE-2751
> Project: Lucene - Java
> Issue Type: Test
> Components: Build
> Reporter: Robert Muir
> Assignee: Robert Muir
> Fix For: 3.1, 4.0
>
> Attachments: LUCENE-2751.patch, LUCENE-2751.patch, LUCENE-2751.patch, LUCENE-2751.patch, LUCENE-2751.patch, LUCENE-2751.patch, LUCENE-2751_branch3x.patch
>
>
> Most tests in the search package don't care about what kind of searcher they use.
> we should randomly use MultiSearcher or ParallelMultiSearcher sometimes in tests.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2751) add LuceneTestCase.newSearcher()
Posted by "Robert Muir (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12980735#action_12980735 ]
Robert Muir commented on LUCENE-2751:
-------------------------------------
We should be able to actually implement this issue now right?
Implement LuceneTestCase.newSearcher(), and like the previous patch, sometimes use parallel in tests.
I think this would be preferred before we go optimizing synchronization, because otherwise how do we know if its correct?
> add LuceneTestCase.newSearcher()
> --------------------------------
>
> Key: LUCENE-2751
> URL: https://issues.apache.org/jira/browse/LUCENE-2751
> Project: Lucene - Java
> Issue Type: Test
> Components: Build
> Reporter: Robert Muir
> Fix For: 3.1, 4.0
>
> Attachments: LUCENE-2751.patch, LUCENE-2751.patch
>
>
> Most tests in the search package don't care about what kind of searcher they use.
> we should randomly use MultiSearcher or ParallelMultiSearcher sometimes in tests.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org