You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Erick Erickson <er...@gmail.com> on 2019/05/30 02:44:40 UTC
CachingNaiveBayesClassifierTest failure locally
I’m pretty sure this has nothing to do with the upgrade to ZK since it’s a Lucene test case, but I ran across this and this hasn’t failed in Chris’ rollups going back to last March so I thought people might want to see it. I have the full test log if anyone’s interested
[junit4] Suite: org.apache.lucene.classification.CachingNaiveBayesClassifierTest
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=CachingNaiveBayesClassifierTest -Dtests.method=testPerformance -Dtests.seed=A9F7A8BCC21AFCBA -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=sq-AL -Dtests.timezone=America/Port-au-Prince -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 5.10s J3 | CachingNaiveBayesClassifierTest.testPerformance <<<
[junit4] > Throwable #1: java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.AssertionError
[junit4] > at __randomizedtesting.SeedInfo.seed([A9F7A8BCC21AFCBA:6E165A9EA9AEC415]:0)
[junit4] > at org.apache.lucene.classification.utils.ConfusionMatrixGenerator.getConfusionMatrix(ConfusionMatrixGenerator.java:131)
[junit4] > at org.apache.lucene.classification.CachingNaiveBayesClassifierTest.testPerformance(CachingNaiveBayesClassifierTest.java:101)
[junit4] > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] > at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] > at java.base/java.lang.reflect.Method.invoke(Method.java:566)
[junit4] > at java.base/java.lang.Thread.run(Thread.java:834)
[junit4] > Caused by: java.util.concurrent.ExecutionException: java.lang.AssertionError
[junit4] > at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
[junit4] > at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:205)
[junit4] > at org.apache.lucene.classification.utils.ConfusionMatrixGenerator.getConfusionMatrix(ConfusionMatrixGenerator.java:94)
[junit4] > ... 36 more
[junit4] > Caused by: java.lang.AssertionError
[junit4] > at java.base/java.util.HashMap$TreeNode.moveRootToFront(HashMap.java:1896)
[junit4] > at java.base/java.util.HashMap$TreeNode.putTreeVal(HashMap.java:2061)
[junit4] > at java.base/java.util.HashMap.putVal(HashMap.java:633)
[junit4] > at java.base/java.util.HashMap.putIfAbsent(HashMap.java:1057)
[junit4] > at org.apache.lucene.search.LRUQueryCache.putIfAbsent(LRUQueryCache.java:301)
[junit4] > at org.apache.lucene.search.LRUQueryCache$CachingWrapperWeight.scorerSupplier(LRUQueryCache.java:745)
[junit4] > at org.apache.lucene.search.BooleanWeight.scorerSupplier(BooleanWeight.java:361)
[junit4] > at org.apache.lucene.search.BooleanWeight.scorer(BooleanWeight.java:327)
[junit4] > at org.apache.lucene.search.Weight.bulkScorer(Weight.java:168)
[junit4] > at org.apache.lucene.search.BooleanWeight.bulkScorer(BooleanWeight.java:321)
[junit4] > at org.apache.lucene.search.LRUQueryCache$CachingWrapperWeight.cache(LRUQueryCache.java:693)
[junit4] > at org.apache.lucene.search.LRUQueryCache$CachingWrapperWeight.bulkScorer(LRUQueryCache.java:825)
[junit4] > at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:700)
[junit4] > at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:494)
[junit4] > at org.apache.lucene.classification.CachingNaiveBayesClassifier.getWordFreqForClassess(CachingNaiveBayesClassifier.java:170)
[junit4] > at org.apache.lucene.classification.CachingNaiveBayesClassifier.calculateLogLikelihood(CachingNaiveBayesClassifier.java:100)
[junit4] > at org.apache.lucene.classification.CachingNaiveBayesClassifier.assignClassNormalizedList(CachingNaiveBayesClassifier.java:83)
[junit4] > at org.apache.lucene.classification.SimpleNaiveBayesClassifier.assignClass(SimpleNaiveBayesClassifier.java:103)
[junit4] > at org.apache.lucene.classification.utils.ConfusionMatrixGenerator.lambda$getConfusionMatrix$0(ConfusionMatrixGenerator.java:94)
[junit4] > at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
[junit4] > at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
[junit4] > at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
[junit4] > ... 1 more
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene80), sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@54e63753), locale=sq-AL, timezone=America/Port-au-Prince
[junit4] 2> NOTE: Mac OS X 10.14.5 x86_64/Oracle Corporation 11.0.2 (64-bit)/cpus=12,threads=1,free=345936896,total=536870912
[junit4] 2> NOTE: All tests run in this JVM: [ConfusionMatrixGeneratorTest, CachingNaiveBayesClassifierTest]
[junit4] Completed [6/12 (1!)] on J3 in 6.65s, 4 tests, 1 error <<< FAILURES!
[junit4]
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: CachingNaiveBayesClassifierTest failure locally
Posted by Dawid Weiss <da...@gmail.com>.
I think it's this underlying standard library bug:
https://bugs.openjdk.java.net/browse/JDK-8205399
Dawid
On Thu, May 30, 2019 at 4:44 AM Erick Erickson <er...@gmail.com> wrote:
>
> I’m pretty sure this has nothing to do with the upgrade to ZK since it’s a Lucene test case, but I ran across this and this hasn’t failed in Chris’ rollups going back to last March so I thought people might want to see it. I have the full test log if anyone’s interested
>
> [junit4] Suite: org.apache.lucene.classification.CachingNaiveBayesClassifierTest
> [junit4] 2> NOTE: reproduce with: ant test -Dtestcase=CachingNaiveBayesClassifierTest -Dtests.method=testPerformance -Dtests.seed=A9F7A8BCC21AFCBA -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=sq-AL -Dtests.timezone=America/Port-au-Prince -Dtests.asserts=true -Dtests.file.encoding=UTF-8
> [junit4] ERROR 5.10s J3 | CachingNaiveBayesClassifierTest.testPerformance <<<
> [junit4] > Throwable #1: java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.AssertionError
> [junit4] > at __randomizedtesting.SeedInfo.seed([A9F7A8BCC21AFCBA:6E165A9EA9AEC415]:0)
> [junit4] > at org.apache.lucene.classification.utils.ConfusionMatrixGenerator.getConfusionMatrix(ConfusionMatrixGenerator.java:131)
> [junit4] > at org.apache.lucene.classification.CachingNaiveBayesClassifierTest.testPerformance(CachingNaiveBayesClassifierTest.java:101)
> [junit4] > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [junit4] > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> [junit4] > at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> [junit4] > at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> [junit4] > at java.base/java.lang.Thread.run(Thread.java:834)
> [junit4] > Caused by: java.util.concurrent.ExecutionException: java.lang.AssertionError
> [junit4] > at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
> [junit4] > at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:205)
> [junit4] > at org.apache.lucene.classification.utils.ConfusionMatrixGenerator.getConfusionMatrix(ConfusionMatrixGenerator.java:94)
> [junit4] > ... 36 more
> [junit4] > Caused by: java.lang.AssertionError
> [junit4] > at java.base/java.util.HashMap$TreeNode.moveRootToFront(HashMap.java:1896)
> [junit4] > at java.base/java.util.HashMap$TreeNode.putTreeVal(HashMap.java:2061)
> [junit4] > at java.base/java.util.HashMap.putVal(HashMap.java:633)
> [junit4] > at java.base/java.util.HashMap.putIfAbsent(HashMap.java:1057)
> [junit4] > at org.apache.lucene.search.LRUQueryCache.putIfAbsent(LRUQueryCache.java:301)
> [junit4] > at org.apache.lucene.search.LRUQueryCache$CachingWrapperWeight.scorerSupplier(LRUQueryCache.java:745)
> [junit4] > at org.apache.lucene.search.BooleanWeight.scorerSupplier(BooleanWeight.java:361)
> [junit4] > at org.apache.lucene.search.BooleanWeight.scorer(BooleanWeight.java:327)
> [junit4] > at org.apache.lucene.search.Weight.bulkScorer(Weight.java:168)
> [junit4] > at org.apache.lucene.search.BooleanWeight.bulkScorer(BooleanWeight.java:321)
> [junit4] > at org.apache.lucene.search.LRUQueryCache$CachingWrapperWeight.cache(LRUQueryCache.java:693)
> [junit4] > at org.apache.lucene.search.LRUQueryCache$CachingWrapperWeight.bulkScorer(LRUQueryCache.java:825)
> [junit4] > at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:700)
> [junit4] > at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:494)
> [junit4] > at org.apache.lucene.classification.CachingNaiveBayesClassifier.getWordFreqForClassess(CachingNaiveBayesClassifier.java:170)
> [junit4] > at org.apache.lucene.classification.CachingNaiveBayesClassifier.calculateLogLikelihood(CachingNaiveBayesClassifier.java:100)
> [junit4] > at org.apache.lucene.classification.CachingNaiveBayesClassifier.assignClassNormalizedList(CachingNaiveBayesClassifier.java:83)
> [junit4] > at org.apache.lucene.classification.SimpleNaiveBayesClassifier.assignClass(SimpleNaiveBayesClassifier.java:103)
> [junit4] > at org.apache.lucene.classification.utils.ConfusionMatrixGenerator.lambda$getConfusionMatrix$0(ConfusionMatrixGenerator.java:94)
> [junit4] > at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> [junit4] > at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> [junit4] > at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> [junit4] > ... 1 more
> [junit4] 2> NOTE: test params are: codec=Asserting(Lucene80), sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@54e63753), locale=sq-AL, timezone=America/Port-au-Prince
> [junit4] 2> NOTE: Mac OS X 10.14.5 x86_64/Oracle Corporation 11.0.2 (64-bit)/cpus=12,threads=1,free=345936896,total=536870912
> [junit4] 2> NOTE: All tests run in this JVM: [ConfusionMatrixGeneratorTest, CachingNaiveBayesClassifierTest]
> [junit4] Completed [6/12 (1!)] on J3 in 6.65s, 4 tests, 1 error <<< FAILURES!
> [junit4]
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: CachingNaiveBayesClassifierTest failure locally
Posted by David Smiley <da...@gmail.com>.
I've seen many cases where an IndexSearcher is created and there is no
value from the underlying query cache embedded in it. Sometimes we
remember to disable it, sometimes not. Tomasso, does
SimpleNaiveBayesClassifier benefit from that cache?
~ David Smiley
Apache Lucene/Solr Search Developer
http://www.linkedin.com/in/davidwsmiley
On Wed, May 29, 2019 at 10:44 PM Erick Erickson <er...@gmail.com>
wrote:
> I’m pretty sure this has nothing to do with the upgrade to ZK since it’s a
> Lucene test case, but I ran across this and this hasn’t failed in Chris’
> rollups going back to last March so I thought people might want to see it.
> I have the full test log if anyone’s interested
>
> [junit4] Suite:
> org.apache.lucene.classification.CachingNaiveBayesClassifierTest
> [junit4] 2> NOTE: reproduce with: ant test
> -Dtestcase=CachingNaiveBayesClassifierTest -Dtests.method=testPerformance
> -Dtests.seed=A9F7A8BCC21AFCBA -Dtests.slow=true -Dtests.badapples=true
> -Dtests.locale=sq-AL -Dtests.timezone=America/Port-au-Prince
> -Dtests.asserts=true -Dtests.file.encoding=UTF-8
> [junit4] ERROR 5.10s J3 |
> CachingNaiveBayesClassifierTest.testPerformance <<<
> [junit4] > Throwable #1: java.lang.RuntimeException:
> java.util.concurrent.ExecutionException: java.lang.AssertionError
> [junit4] > at
> __randomizedtesting.SeedInfo.seed([A9F7A8BCC21AFCBA:6E165A9EA9AEC415]:0)
> [junit4] > at
> org.apache.lucene.classification.utils.ConfusionMatrixGenerator.getConfusionMatrix(ConfusionMatrixGenerator.java:131)
> [junit4] > at
> org.apache.lucene.classification.CachingNaiveBayesClassifierTest.testPerformance(CachingNaiveBayesClassifierTest.java:101)
> [junit4] > at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> [junit4] > at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> [junit4] > at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> [junit4] > at
> java.base/java.lang.reflect.Method.invoke(Method.java:566)
> [junit4] > at java.base/java.lang.Thread.run(Thread.java:834)
> [junit4] > Caused by: java.util.concurrent.ExecutionException:
> java.lang.AssertionError
> [junit4] > at
> java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
> [junit4] > at
> java.base/java.util.concurrent.FutureTask.get(FutureTask.java:205)
> [junit4] > at
> org.apache.lucene.classification.utils.ConfusionMatrixGenerator.getConfusionMatrix(ConfusionMatrixGenerator.java:94)
> [junit4] > ... 36 more
> [junit4] > Caused by: java.lang.AssertionError
> [junit4] > at
> java.base/java.util.HashMap$TreeNode.moveRootToFront(HashMap.java:1896)
> [junit4] > at
> java.base/java.util.HashMap$TreeNode.putTreeVal(HashMap.java:2061)
> [junit4] > at
> java.base/java.util.HashMap.putVal(HashMap.java:633)
> [junit4] > at
> java.base/java.util.HashMap.putIfAbsent(HashMap.java:1057)
> [junit4] > at
> org.apache.lucene.search.LRUQueryCache.putIfAbsent(LRUQueryCache.java:301)
> [junit4] > at
> org.apache.lucene.search.LRUQueryCache$CachingWrapperWeight.scorerSupplier(LRUQueryCache.java:745)
> [junit4] > at
> org.apache.lucene.search.BooleanWeight.scorerSupplier(BooleanWeight.java:361)
> [junit4] > at
> org.apache.lucene.search.BooleanWeight.scorer(BooleanWeight.java:327)
> [junit4] > at
> org.apache.lucene.search.Weight.bulkScorer(Weight.java:168)
> [junit4] > at
> org.apache.lucene.search.BooleanWeight.bulkScorer(BooleanWeight.java:321)
> [junit4] > at
> org.apache.lucene.search.LRUQueryCache$CachingWrapperWeight.cache(LRUQueryCache.java:693)
> [junit4] > at
> org.apache.lucene.search.LRUQueryCache$CachingWrapperWeight.bulkScorer(LRUQueryCache.java:825)
> [junit4] > at
> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:700)
> [junit4] > at
> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:494)
> [junit4] > at
> org.apache.lucene.classification.CachingNaiveBayesClassifier.getWordFreqForClassess(CachingNaiveBayesClassifier.java:170)
> [junit4] > at
> org.apache.lucene.classification.CachingNaiveBayesClassifier.calculateLogLikelihood(CachingNaiveBayesClassifier.java:100)
> [junit4] > at
> org.apache.lucene.classification.CachingNaiveBayesClassifier.assignClassNormalizedList(CachingNaiveBayesClassifier.java:83)
> [junit4] > at
> org.apache.lucene.classification.SimpleNaiveBayesClassifier.assignClass(SimpleNaiveBayesClassifier.java:103)
> [junit4] > at
> org.apache.lucene.classification.utils.ConfusionMatrixGenerator.lambda$getConfusionMatrix$0(ConfusionMatrixGenerator.java:94)
> [junit4] > at
> java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> [junit4] > at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> [junit4] > at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> [junit4] > ... 1 more
> [junit4] 2> NOTE: test params are: codec=Asserting(Lucene80),
> sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@54e63753),
> locale=sq-AL, timezone=America/Port-au-Prince
> [junit4] 2> NOTE: Mac OS X 10.14.5 x86_64/Oracle Corporation 11.0.2
> (64-bit)/cpus=12,threads=1,free=345936896,total=536870912
> [junit4] 2> NOTE: All tests run in this JVM:
> [ConfusionMatrixGeneratorTest, CachingNaiveBayesClassifierTest]
> [junit4] Completed [6/12 (1!)] on J3 in 6.65s, 4 tests, 1 error <<<
> FAILURES!
> [junit4]
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>