[jira] [Commented] (SOLR-4656) Add hl.maxMultiValuedToExamine to limit the number of multiValued entries examined while highlighting

["David Smiley (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SOLR-4656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14394644#comment-14394644 ] 

David Smiley commented on SOLR-4656:
------------------------------------

While working on SOLR-6692 I noticed this again, and I'm wondering two things:
* Is the semantics of maxMultiValuedToMatch intentional with respect to that it counts snippets (i.e. fragments), as opposed to values?  It's unfortunate the parameter name doesn't make this clear, which is suggestive that it counts values (maxMultiValuedToExamine counts values).  There's a difference when hl.snippets isn't 1.
* I don't believe mvToMatch should be decremented when bestTextFragment.getScore() is <= 0 since _there actually was no match_.  This can happen often when hl.preserveMulti=true.  I think this is a bug.

I can fix but I'd like your thoughts, [~erickerickson].

> Add hl.maxMultiValuedToExamine to limit the number of multiValued entries examined while highlighting
> -----------------------------------------------------------------------------------------------------
>
>                 Key: SOLR-4656
>                 URL: https://issues.apache.org/jira/browse/SOLR-4656
>             Project: Solr
>          Issue Type: Improvement
>          Components: highlighter
>    Affects Versions: 4.3, Trunk
>            Reporter: Erick Erickson
>            Assignee: Erick Erickson
>            Priority: Minor
>             Fix For: 4.3, Trunk
>
>         Attachments: SOLR-4656-4x.patch, SOLR-4656-4x.patch, SOLR-4656-trunk.patch, SOLR-4656.patch
>
>
> I'm looking at an admittedly pathological case of many, many entries in a multiValued field, and trying to implement a way to limit the number examined, analogous to maxAnalyzedChars, see the patch.
> Along the way, I noticed that we do what looks like unnecessary copying of the fields to be examined. We call Document.getFields, which copies all of the fields and values to the returned array. Then we copy all of those to another array, converting them to Strings. Then we actually examine them. a> this doesn't seem very efficient and b> reduces the benefit from limiting the number of mv values examined.
> So the attached does two things:
> 1> attempts to fix this
> 2> implements hl.maxMultiValuedToExamine
> I'd _really_ love it if someone who knows the highlighting code takes a peek at the fix to see if I've messed things up, the changes are actually pretty minimal.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org