You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@lucene.apache.org by "Greg Miller (Jira)" <ji...@apache.org> on 2021/06/03 18:09:00 UTC
[jira] [Comment Edited] (LUCENE-9988) Address DrillSideways bug
discovered during randomized testing
[ https://issues.apache.org/jira/browse/LUCENE-9988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17356631#comment-17356631 ]
Greg Miller edited comment on LUCENE-9988 at 6/3/21, 6:08 PM:
--------------------------------------------------------------
OK, a little context and a description of the bug:
Prior to LUCENE-9944, {{DrillSidewaysQuery}} was not safe to use for concurrent search. It would share instances of {{FacetsCollector}} across threads. LUCENE-9944 modified {{DrillSidewaysQuery}} to use {{FacetsCollectorManagers instead, instantiating new FacetsCollector}} instances whenever new {{BulkScorers are created. DrillSidewaysQuery}} also keeps track of the instantiated {{FacetsCollectors and exposes them to DrillSideways}}, which is responsible for merging them all at the end.
The issue is that, if {{DrillSidewaysQuery#rewrite}} resulting in instantiating a new instance, the tracked {{FacetsCollectors weren't carried over. So DrillSideways}} would have a reference to the old query and never see the instantiated FacetsCollectors.
The actual code change to fix this is pretty simple. PR attached if someone wouldn't mind taking a look.
was (Author: gsmiller):
OK, a little context and a description of the bug:
Prior to LUCENE-9944, {{DrillSidewaysQuery}} was not safe to use for concurrent search. It would share instances of {{FacetsCollector}} across threads. LUCENE-9944 modified {{DrillSidewaysQuery}} to use {{FacetsCollectorManager}}s instead, instantiating new {{FacetsCollector}} instances whenever new {{BulkScorer}}s are created. {{DrillSidewaysQuery}} also keeps track of the instantiated {{FacetsCollector}}s and exposes them to {{DrillSideways}}, which is responsible for merging them all at the end.
The issue is that, if {{DrillSidewaysQuery#rewrite}} resulting in instantiating a new instance, the tracked {{FacetsCollector}}s weren't carried over. So {{DrillSideways}} would have a reference to the old query and never see the instantiated {{FacetsCollector}}s.
The actual code change to fix this is pretty simple. PR attached if someone wouldn't mind taking a look.
> Address DrillSideways bug discovered during randomized testing
> --------------------------------------------------------------
>
> Key: LUCENE-9988
> URL: https://issues.apache.org/jira/browse/LUCENE-9988
> Project: Lucene - Core
> Issue Type: Bug
> Components: modules/facet
> Affects Versions: main (9.0)
> Reporter: Greg Miller
> Assignee: Greg Miller
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> There appears to be a correctness bug in DrillSideways likely introduced in LUCENE-9944. Need to track it down and fix.
>
> Build: [https://ci-builds.apache.org/job/Lucene/job/Lucene-Coverage-main/62/]
> 1 tests failed.
> FAILED: org.apache.lucene.facet.TestDrillSideways.testRandom
> Error Message:
> java.lang.AssertionError
> Stack Trace:
> java.lang.AssertionError
> at __randomizedtesting.SeedInfo.seed([ADCF6881460FEE2F:DF834D8EF76F585C]:0)
> at org.junit.Assert.fail(Assert.java:87)
> at org.junit.Assert.assertTrue(Assert.java:42)
> at org.junit.Assert.assertTrue(Assert.java:53)
> at org.apache.lucene.facet.TestDrillSideways.verifyEquals(TestDrillSideways.java:1580)
> at org.apache.lucene.facet.TestDrillSideways.testRandom(TestDrillSideways.java:1159)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1754)
> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:942)
> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:978)
> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:992)
> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:44)
> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:819)
> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:470)
> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:951)
> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:836)
> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:887)
> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:898)
> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
> at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:826)
> at java.base/java.lang.Thread.run(Thread.java:834)
> Build Log:
> [...truncated 553 lines...]
> ERROR: The following test(s) have failed:
> - org.apache.lucene.facet.TestDrillSideways.testRandom (:lucene:facet)
> Test output: /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Coverage-main/lucene/facet/build/test-results/test/outputs/OUTPUT-org.apache.lucene.facet.TestDrillSideways.txt
> Reproduce with: gradlew :lucene:facet:test --tests "org.apache.lucene.facet.TestDrillSideways.testRandom" -Ptests.jvms=4 -Ptests.haltonfailure=false -Ptests.jvmargs=-XX:TieredStopAtLevel=1 -Ptests.seed=ADCF6881460FEE2F -Ptests.multiplier=2 -Ptests.badapples=false -Ptests.file.encoding=UTF-8
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org