You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@lucene.apache.org by "Chris M. Hostetter (Jira)" <ji...@apache.org> on 2020/05/08 00:38:00 UTC
[jira] [Updated] (SOLR-14467) inconsistent server errors combining
relatedness() with allBuckets:true
[ https://issues.apache.org/jira/browse/SOLR-14467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris M. Hostetter updated SOLR-14467:
--------------------------------------
Attachment: SOLR-14467_test.patch
Status: Open (was: Open)
Examples of the various (root cause) failures that can occur with the attached test patch...
{noformat}
[junit4] 2> 5351 INFO (qtp10892224-67) [n:127.0.0.1:40385_solr c:org.apache.solr.search.facet.TestCloudJSONFacetSKG_collection s:shard1 r:core_node3 x:org.apache.solr.search.facet.TestCloudJSONFacetSKG_collection_shard1_replica_n1 ] o.a.s.c.S.Request [org.apache.solr.search.facet.TestCloudJSONFacetSKG_collection_shard1_replica_n1] webapp=/solr path=/select params={df=text&distrib=false&_facet_={}&fl=id&fl=score&shards.purpose=1064964&start=0&fsv=true&back=*:*&shard.url=http://127.0.0.1:40385/solr/org.apache.solr.search.facet.TestCloudJSONFacetSKG_collection_shard1_replica_n1/&rows=0&fore=(field_4_sds:17+OR+field_7_sds:49+OR+field_10_sds:23+OR+field_2_sdsS:7+OR+field_5_sdsS:31+OR+field_6_ss:20+OR+field_4_sds:43+OR+field_14_sdsS:61+OR+field_7_sds:34)&version=2&q=(field_3_ss:44+OR+field_14_sdsS:23+OR+field_13_sds:28+OR+field_6_ss:44)&json.facet={+processEmpty:+true,+facet_1+:+{+type:terms,+field:field_14_idsS,+limit:-1,+overrequest:0,+sort:+'skg+desc',+allBuckets:true,+refine:+true,+domain:+{+query:+'*:*'+},+facet:{+processEmpty:+true,+facet_2+:+{+type:terms,+field:field_8_idsS,+limit:67,+overrequest:0,+sort:+'index+asc',+allBuckets:true,+refine:+true,+domain:+{+query:+'*:*'+},+facet:{+processEmpty:+true,+skg+:+'relatedness($fore,$back)'}}+,skg+:+'relatedness($fore,$back)'}}+,facet_3+:+{+type:terms,+field:field_2_idsS,+limit:-1,+overrequest:0,+sort:+'skg+asc',+allBuckets:true,+refine:+true,+domain:+{+query:+'*:*'+},+facet:{+processEmpty:+true,+skg+:+'relatedness($fore,$back)'}}+}&omitHeader=false&NOW=1588893509649&isShard=true&wt=javabin} hits=12 status=500 QTime=56
[junit4] 2> 5352 ERROR (qtp1523452543-62) [n:127.0.0.1:33775_solr c:org.apache.solr.search.facet.TestCloudJSONFacetSKG_collection s:shard2 r:core_node4 x:org.apache.solr.search.facet.TestCloudJSONFacetSKG_collection_shard2_replica_n2 ] o.a.s.s.HttpSolrCall null:java.lang.ArrayIndexOutOfBoundsException: Index 128 out of bounds for length 128
[junit4] 2> at org.apache.solr.search.facet.FacetFieldProcessorByHashDV.lambda$new$3(FacetFieldProcessorByHashDV.java:439)
[junit4] 2> at org.apache.solr.search.facet.RelatednessAgg$SKGSlotAcc.processSlot(RelatednessAgg.java:189)
[junit4] 2> at org.apache.solr.search.facet.RelatednessAgg$SKGSlotAcc.collect(RelatednessAgg.java:217)
[junit4] 2> at org.apache.solr.search.facet.FacetFieldProcessor$SpecialSlotAcc.collect(FacetFieldProcessor.java:772)
[junit4] 2> at org.apache.solr.search.facet.FacetFieldProcessor.collectFirstPhase(FacetFieldProcessor.java:289)
[junit4] 2> at org.apache.solr.search.facet.FacetFieldProcessorByHashDV.collectValFirstPhase(FacetFieldProcessorByHashDV.java:430)
[junit4] 2> at org.apache.solr.search.facet.FacetFieldProcessorByHashDV$5.collect(FacetFieldProcessorByHashDV.java:389)
[junit4] 2> at org.apache.solr.search.DocSetUtil.collectSortedDocSet(DocSetUtil.java:278)
[junit4] 2> at org.apache.solr.search.facet.FacetFieldProcessorByHashDV.collectDocs(FacetFieldProcessorByHashDV.java:374)
[junit4] 2> at org.apache.solr.search.facet.FacetFieldProcessorByHashDV.calcFacets(FacetFieldProcessorByHashDV.java:248)
[junit4] 2> at org.apache.solr.search.facet.FacetFieldProcessorByHashDV.process(FacetFieldProcessorByHashDV.java:215)
[junit4] 2> at org.apache.solr.search.facet.FacetRequest.process(FacetRequest.java:416)
[junit4] 2> at org.apache.solr.search.facet.FacetProcessor.processSubs(FacetProcessor.java:475)
[junit4] 2> at org.apache.solr.search.facet.FacetProcessor.fillBucket(FacetProcessor.java:432)
[junit4] 2> at org.apache.solr.search.facet.FacetQueryProcessor.process(FacetQuery.java:64)
[junit4] 2> at org.apache.solr.search.facet.FacetRequest.process(FacetRequest.java:416)
[junit4] 2> at org.apache.solr.search.facet.FacetModule.process(FacetModule.java:147)
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestCloudJSONFacetSKG -Dtests.method=testRandom -Dtests.seed=58E72A6ADD982BE0 -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=it-VA -Dtests.timezone=Etc/GMT-13 -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
[junit4] ERROR 0.11s | TestCloudJSONFacetSKG.testRandom <<<
[junit4] > Throwable #1: java.lang.RuntimeException: init query failed: {main(q=(field_6_ss:29+OR+field_8_sdsS:9+OR+field_0_ss:36+OR+field_7_sds:15+OR+field_6_ss:54+OR+field_2_sdsS:50+OR+field_4_sds:47+OR+field_8_sdsS:21)&json.facet={+processEmpty:+true,+facet_1+:+{+type:terms,+field:field_7_sds,+limit:50,+overrequest:0,+sort:+'skg+desc',+allBuckets:true,+refine:+true,+domain:+{+query:+'*:*'+},+facet:{+processEmpty:+true,+facet_2+:+{+type:terms,+field:field_14_sdsS,+limit:-1,+overrequest:0,+sort:+'count+asc',+allBuckets:true,+refine:+true,+domain:+{+query:+'*:*'+},+facet:{+processEmpty:+true,+skg+:+'relatedness($fore,$back)'}}+,skg+:+'relatedness($fore,$back)'}}+}),extra(rows=0&fore=(field_13_sds:45+OR+field_0_ss:4+OR+field_9_ss:16+OR+field_3_ss:49)&back=(field_1_sds:20+OR+field_11_sdsS:24+OR+field_12_ss:62+OR+field_12_ss:37))}: Error from server at http://127.0.0.1:42897/solr/org.apache.solr.search.facet.TestCloudJSONFacetSKG_collection: Error from server at null: Expected mime type application/octet-stream but got text/html. <html>
[junit4] > <head>
[junit4] > <meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>
[junit4] > <title>Error 500 java.lang.AssertionError</title>
[junit4] > </head>
[junit4] > <body><h2>HTTP ERROR 500 java.lang.AssertionError</h2>
[junit4] > <table>
[junit4] > <tr><th>URI:</th><td>/solr/org.apache.solr.search.facet.TestCloudJSONFacetSKG_collection_shard2_replica_n2/select</td></tr>
[junit4] > <tr><th>STATUS:</th><td>500</td></tr>
[junit4] > <tr><th>MESSAGE:</th><td>java.lang.AssertionError</td></tr>
[junit4] > <tr><th>SERVLET:</th><td>org.apache.solr.client.solrj.embedded.JettySolrRunner$Servlet404-55da7b8b</td></tr>
[junit4] > <tr><th>CAUSED BY:</th><td>java.lang.AssertionError</td></tr>
[junit4] > </table>
[junit4] > <h3>Caused by:</h3><pre>java.lang.AssertionError
[junit4] > at org.apache.lucene.index.AssertingLeafReader$AssertingSortedSetDocValues.lookupOrd(AssertingLeafReader.java:1004)
[junit4] > at org.apache.solr.search.facet.FacetFieldProcessorByArrayDV.lookupOrd(FacetFieldProcessorByArrayDV.java:182)
[junit4] > at org.apache.solr.search.facet.FacetFieldProcessorByArray.lambda$new$2(FacetFieldProcessorByArray.java:138)
[junit4] > at org.apache.solr.search.facet.RelatednessAgg$SKGSlotAcc.processSlot(RelatednessAgg.java:189)
[junit4] > at org.apache.solr.search.facet.RelatednessAgg$SKGSlotAcc.collect(RelatednessAgg.java:217)
[junit4] > at org.apache.solr.search.facet.FacetFieldProcessor$SpecialSlotAcc.collect(FacetFieldProcessor.java:772)
[junit4] > at org.apache.solr.search.facet.FacetFieldProcessorByArrayDV.collect(FacetFieldProcessorByArrayDV.java:335)
[junit4] > at org.apache.solr.search.facet.FacetFieldProcessorByArrayDV.collectDocs(FacetFieldProcessorByArrayDV.java:303)
[junit4] > at org.apache.solr.search.facet.FacetFieldProcessorByArrayDV.collectDocs(FacetFieldProcessorByArrayDV.java:171)
[junit4] > at org.apache.solr.search.facet.FacetFieldProcessorByArray.calcFacets(FacetFieldProcessorByArray.java:112)
[junit4] > at org.apache.solr.search.facet.FacetFieldProcessorByArray.process(FacetFieldProcessorByArray.java:62)
[junit4] > at org.apache.solr.search.facet.FacetRequest.process(FacetRequest.java:416)
[junit4] > at org.apache.solr.search.facet.FacetProcessor.processSubs(FacetProcessor.java:475)
[junit4] > at org.apache.solr.search.facet.FacetProcessor.fillBucket(FacetProcessor.java:432)
[junit4] > at org.apache.solr.search.facet.FacetQueryProcessor.process(FacetQuery.java:64)
[junit4] > at org.apache.solr.search.facet.FacetRequest.process(FacetRequest.java:416)
[junit4] > at org.apache.solr.search.facet.FacetModule.process(FacetModule.java:147)
[junit4] > at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:328)
[junit4] 2> java.lang.AssertionError: null
[junit4] 2> at org.apache.lucene.util.packed.PackedLongValues.get(PackedLongValues.java:105) ~[java/:?]
[junit4] 2> at org.apache.lucene.index.OrdinalMap.getFirstSegmentNumber(OrdinalMap.java:344) ~[java/:?]
[junit4] 2> at org.apache.lucene.index.MultiDocValues$MultiSortedSetDocValues.lookupOrd(MultiDocValues.java:884) ~[java/:?]
[junit4] 2> at org.apache.solr.search.facet.FacetFieldProcessorByArrayDV.lookupOrd(FacetFieldProcessorByArrayDV.java:182) ~[java/:?]
[junit4] 2> at org.apache.solr.search.facet.FacetFieldProcessorByArray.lambda$new$2(FacetFieldProcessorByArray.java:138) ~[java/:?]
[junit4] 2> at org.apache.solr.search.facet.RelatednessAgg$SKGSlotAcc.processSlot(RelatednessAgg.java:189) ~[java/:?]
[junit4] 2> at org.apache.solr.search.facet.RelatednessAgg$SKGSlotAcc.collect(RelatednessAgg.java:217) ~[java/:?]
[junit4] 2> at org.apache.solr.search.facet.FacetFieldProcessor$SpecialSlotAcc.collect(FacetFieldProcessor.java:772) ~[java/:?]
[junit4] 2> at org.apache.solr.search.facet.FacetFieldProcessorByArrayDV.collect(FacetFieldProcessorByArrayDV.java:335) ~[java/:?]
[junit4] 2> at org.apache.solr.search.facet.FacetFieldProcessorByArrayDV.collectDocs(FacetFieldProcessorByArrayDV.java:303) ~[java/:?]
[junit4] 2> at org.apache.solr.search.facet.FacetFieldProcessorByArrayDV.collectDocs(FacetFieldProcessorByArrayDV.java:171) ~[java/:?]
[junit4] 2> at org.apache.solr.search.facet.FacetFieldProcessorByArray.calcFacets(FacetFieldProcessorByArray.java:112) ~[java/:?]
[junit4] 2> at org.apache.solr.search.facet.FacetFieldProcessorByArray.process(FacetFieldProcessorByArray.java:62) ~[java/:?]
[junit4] 2> at org.apache.solr.search.facet.FacetRequest.process(FacetRequest.java:416) ~[java/:?]
[junit4] 2> at org.apache.solr.search.facet.FacetProcessor.processSubs(FacetProcessor.java:475) ~[java/:?]
[junit4] 2> at org.apache.solr.search.facet.FacetFieldProcessor.fillBucketFromSlot(FacetFieldProcessor.java:545) ~[java/:?]
[junit4] 2> at org.apache.solr.search.facet.FacetFieldProcessor.findTopSlots(FacetFieldProcessor.java:446) ~[java/:?]
[junit4] 2> at org.apache.solr.search.facet.FacetFieldProcessorByHashDV.calcFacets(FacetFieldProcessorByHashDV.java:250) ~[java/:?]
[junit4] 2> at org.apache.solr.search.facet.FacetFieldProcessorByHashDV.process(FacetFieldProcessorByHashDV.java:215) ~[java/:?]
[junit4] 2> at org.apache.solr.search.facet.FacetRequest.process(FacetRequest.java:416) ~[java/:?]
[junit4] 2> at org.apache.solr.search.facet.FacetProcessor.processSubs(FacetProcessor.java:475) ~[java/:?]
[junit4] 2> at org.apache.solr.search.facet.FacetProcessor.fillBucket(FacetProcessor.java:432) ~[java/:?]
[junit4] 2> at org.apache.solr.search.facet.FacetQueryProcessor.process(FacetQuery.java:64) ~[java/:?]
[junit4] 2> at org.apache.solr.search.facet.FacetRequest.process(FacetRequest.java:416) ~[java/:?]
[junit4] 2> at org.apache.solr.search.facet.FacetModule.process(FacetModule.java:147) ~[java/:?]
[junit4] 2> at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:328) ~[java/:?]
{noformat}
> inconsistent server errors combining relatedness() with allBuckets:true
> -----------------------------------------------------------------------
>
> Key: SOLR-14467
> URL: https://issues.apache.org/jira/browse/SOLR-14467
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Facet Module
> Reporter: Chris M. Hostetter
> Priority: Major
> Attachments: SOLR-14467_test.patch
>
>
> While working on randomized testing for SOLR-13132 i discovered a variety of different ways that JSON Faceting's "allBuckets" option can fail when combined with the "relatedness()" function.
> I haven't found a trivial way to manual reproduce this, but i have been able to trigger the failures with a trivial patch to {{TestCloudJSONFacetSKG}} which i will attach.
> Based on the nature of the failures it looks like it may have something to do with multiple segments of different sizes, and or resizing the SlotAccs ?
> The relatedness() function doesn't have much (any?) existing tests in place that leverage "allBuckets" so this is probably a bug that has always existed -- it's possible it may be excessively cumbersome to fix and we might nee/wnat to just document that incompatibility and add some code to try and detect if the user combines these options and if so fail with a 400 error?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org