You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@lucene.apache.org by "Christine Poerschke (Jira)" <ji...@apache.org> on 2019/10/23 16:18:00 UTC
[jira] [Comment Edited] (LUCENE-8996) maxScore is sometimes missing
from distributed grouped responses
[ https://issues.apache.org/jira/browse/LUCENE-8996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16957942#comment-16957942 ]
Christine Poerschke edited comment on LUCENE-8996 at 10/23/19 4:17 PM:
-----------------------------------------------------------------------
Re-opening to address (Solr) {{TestDistributedGrouping}} failure e.g. [https://jenkins.thetaphi.de/job/Lucene-Solr-8.x-Windows/512/]
{code:java}
...
[junit4] FAILURE 4.83s J0 | TestDistributedGrouping.test <<<
[junit4] > Throwable #1: junit.framework.AssertionFailedError: .grouped[a_i1].groups[0].doclist.maxScore:1.4E-45!=null
[junit4] > at __randomizedtesting.SeedInfo.seed([9532D3AC873FA0D8:1D66EC7629C3CD20]:0)
[junit4] > at junit.framework.Assert.fail(Assert.java:57)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase.compareSolrResponses(BaseDistributedSearchTestCase.java:999)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase.compareResponses(BaseDistributedSearchTestCase.java:1026)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:680)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:656)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:635)
[junit4] > at org.apache.solr.TestDistributedGrouping.test(TestDistributedGrouping.java:171)
[junit4] > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] > at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] > at java.base/java.lang.reflect.Method.invoke(Method.java:567)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsRepeatStatement.callStatement(BaseDistributedSearchTestCase.java:1108)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:1054)
[junit4] > at java.base/java.lang.Thread.run(Thread.java:830)
...
{code}
edit: further details extracted from [https://jenkins.thetaphi.de/job/Lucene-Solr-8.x-Windows/512/consoleText] file. given that the doclist is non-empty the absence of a maxScore seems unexpected?
{code:java}
...
[junit4] 2> {responseHeader={status=0,QTime=10},grouped={a_i1={matches=172,groups=[{groupValue=-987,doclist={numFound=1,start=0,maxScore=1.4E-45,docs=[SolrDocument{id=11, a_i1=-987}]}}, {groupValue=-600,doclist=...
...
[junit4] 2> {responseHeader={status=0,QTime=2},grouped={a_i1={matches=172,groups=[{groupValue=-987,doclist={numFound=1,start=0,docs=[SolrDocument{id=11, a_i1=-987}]}}, {groupValue=-600,doclist=...
...
{code}
was (Author: cpoerschke):
Re-opening to address (Solr) {{TestDistributedGrouping}} failure e.g. https://jenkins.thetaphi.de/job/Lucene-Solr-8.x-Windows/512/
{code}
...
[junit4] FAILURE 4.83s J0 | TestDistributedGrouping.test <<<
[junit4] > Throwable #1: junit.framework.AssertionFailedError: .grouped[a_i1].groups[0].doclist.maxScore:1.4E-45!=null
[junit4] > at __randomizedtesting.SeedInfo.seed([9532D3AC873FA0D8:1D66EC7629C3CD20]:0)
[junit4] > at junit.framework.Assert.fail(Assert.java:57)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase.compareSolrResponses(BaseDistributedSearchTestCase.java:999)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase.compareResponses(BaseDistributedSearchTestCase.java:1026)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:680)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:656)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:635)
[junit4] > at org.apache.solr.TestDistributedGrouping.test(TestDistributedGrouping.java:171)
[junit4] > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] > at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] > at java.base/java.lang.reflect.Method.invoke(Method.java:567)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsRepeatStatement.callStatement(BaseDistributedSearchTestCase.java:1108)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:1054)
[junit4] > at java.base/java.lang.Thread.run(Thread.java:830)
...
{code}
> maxScore is sometimes missing from distributed grouped responses
> ----------------------------------------------------------------
>
> Key: LUCENE-8996
> URL: https://issues.apache.org/jira/browse/LUCENE-8996
> Project: Lucene - Core
> Issue Type: Bug
> Affects Versions: 5.3
> Reporter: Julien Massenet
> Assignee: Christine Poerschke
> Priority: Minor
> Fix For: master (9.0), 8.3
>
> Attachments: LUCENE-8996.02.patch, LUCENE-8996.patch, LUCENE-8996.patch, lucene_6_5-GroupingMaxScore.patch, lucene_solr_5_3-GroupingMaxScore.patch, master-GroupingMaxScore.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> This issue occurs when using the grouping feature in distributed mode and sorting by score.
> Each group's {{docList}} in the response is supposed to contain a {{maxScore}} entry that hold the maximum score for that group. Using the current releases, it sometimes happens that this piece of information is not included:
> {code}
> {
> "responseHeader": {
> "status": 0,
> "QTime": 42,
> "params": {
> "sort": "score desc",
> "fl": "id,score",
> "q": "_text_:\"72\"",
> "group.limit": "2",
> "group.field": "group2",
> "group.sort": "score desc",
> "group": "true",
> "wt": "json",
> "fq": "group2:72 OR group2:45"
> }
> },
> "grouped": {
> "group2": {
> "matches": 567,
> "groups": [
> {
> "groupValue": 72,
> "doclist": {
> "numFound": 562,
> "start": 0,
> "maxScore": 2.0378063,
> "docs": [
> {
> "id": "29!26551",
> "score": 2.0378063
> },
> {
> "id": "78!11462",
> "score": 2.0298104
> }
> ]
> }
> },
> {
> "groupValue": 45,
> "doclist": {
> "numFound": 5,
> "start": 0,
> "docs": [
> {
> "id": "72!8569",
> "score": 1.8988966
> },
> {
> "id": "72!14075",
> "score": 1.5191172
> }
> ]
> }
> }
> ]
> }
> }
> }
> {code}
> Looking into the issue, it comes from the fact that if a shard does not contain a document from that group, trying to merge its {{maxScore}} with real {{maxScore}} entries from other shards is invalid (it results in NaN).
> I'm attaching a patch containing a fix.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org