[jira] [Updated] (SOLR-6780) Merging request parameters with defaults produce duplicate entries

["Hoss Man (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SOLR-6780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Hoss Man updated SOLR-6780:
---------------------------
    Attachment: SOLR-6780.patch

What an evil freaking bug.

I audited all of the usages of {{getParameterNamesIterator()}} to try and track down how severe the impacts of this issue may be -- most of them are either semi-benign (like the "echoParams" case) or not affected by the redundency (ie: code that iterats over all params looking for certain things, and then adds the assocaited names/values to a Set so they get deduped anyway)

There were 4 main areas i found where this bug could result in problematic behavior

* ExtractingRequestHandler
** "literal.\*" params will be duplicated if overridden by defaults/invariants/appends - this will result in redundent literal field=value params being added to the document.
** impact: multiple values in literal fields when not expected/desired
* FacetComponent
** "facet.\*" params will be duplicated if overridden by defaults/invariants/appends - this can result in redundent computation and identical facet.field, facet.query, or facet.range blocks in the response
** impact: wasted computation & increased response size
* SpellCheckComponent
** when "custom params" (ie: "spellcheck.\[dictionary name\].XXXX=YYYY" are used in used in defaults, appends, or invariants, it can cause redudent XXXXX=YYYY params to be used.
** when "spellcheck.collateParam.XXXX=YYYY" type params are used defaults, appends, or invariants, it can cause redundent XXXX=YYYY params to exist in the collation verification queries.
** impact: unclear to me at first glance, probably just wasted computation & increased response size
* AnalyticsComponent
** "olap.\*" params will be duplicated if overridden by defaults/invariants/appends - this can result in redundent computation
** impact: unclear to me at first glance, probably just wasted computation & increased response size

...in addition to fixing the bug & adding explicit unit tests for it, the attached patch also includes some sanity check testing for the FacetComponent & ExtractingRequestHandler situations above, to try and protect us from similar "redundency" bugs in the future.



> Merging request parameters with defaults produce duplicate entries
> ------------------------------------------------------------------
>
>                 Key: SOLR-6780
>                 URL: https://issues.apache.org/jira/browse/SOLR-6780
>             Project: Solr
>          Issue Type: Bug
>    Affects Versions: 4.1, 5.0, Trunk
>            Reporter: Alexandre Rafalovitch
>              Labels: parameters
>         Attachments: SOLR-6780.patch
>
>
> When a parameter (e.g. echoParams) is specified and overrides the default on the handler, it actually generates two entries for that key with the same value. 
> Most of the time it is just a confusion and not an issue, however, some components will do the work twice. For example faceting component as described in http://search-lucene.com/m/QTPaSlFUQ1/duplicate
> It may also be connected to SOLR-6369
> The cause seems to be the interplay between *DefaultSolrParams#getParameterNamesIterator()* which just returns param names in sequence and *SolrParams#toNamedList()* which uses the first (override then default) value for each key, without deduplication.
> It's easily reproducible in trunk against schemaless example with 
> bq. curl "http://localhost:8983/solr/schemaless/select?indent=true&echoParams=all"
> I've also spot checked it and it seems to be reproducible back to Solr 4.1.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org