You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Ann B <al...@gmail.com> on 2015/08/18 21:31:17 UTC

Stem Words Highlighted - Keyword Not Highlighted

Question:

Can I configure solr to highlight the keyword also?  The search results are
correct, but the highlighting is not complete.

*

Example:

Keyword: stocks

Request: (I only provided the url parameters below.)

hl=true&
hl.fl=spell&
hl.simple.pre=%5BHIGHLIGHT%5D&
hl.simple.post=%5B%2FHIGHLIGHT%5D&
hl.snippets=3&
hl.fragsize=70&
hl.mergeContiguous=true&

fl=item_id%2Cscore&

qf=tm_body%3Avalue%5E1.0&
qf=tm_title%5E13.0&

fq=im_field_webresource_category%3A%226013%22&
fq=index_id%3Atest&

start=0&rows=10&facet=true&facet.sort=count&facet.limit=10&facet.mincount=1&facet.missing=false&facet.field=im_field_webresource_category&f.im_field_webresource_category.facet.limit=50&

wt=json&json.nl=map&

q=%22stocks%22

*

Response:

"highlighting":{
"test-49904":{"spell":[
"Includes free access to [HIGHLIGHT]stock[/HIGHLIGHT] charts and
instruction about using [HIGHLIGHT]stock[/HIGHLIGHT] charts in technical
analysis of stocks. Paid subscriptions provide access to more
information."]},...

*

Details:

Tokenizer:  <tokenizer class="solr.WhitespaceTokenizerFactory"/>

Filters:

<filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt"
ignoreCase="true" expand="true"/> <filter class="solr.StopFilterFactory"
ignoreCase="true" words="stopwords.txt" enablePositionIncrements="true"/>
<filter class="solr.WordDelimiterFilterFactory" protected="protwords.txt"
generateWordParts="1" generateNumberParts="1" catenateWords="0"
catenateNumbers="0" catenateAll="0" splitOnCaseChange="1"
preserveOriginal="1"/> <filter class="solr.LengthFilterFactory" min="2"
max="100"/> <filter class="solr.LowerCaseFilterFactory"/> <filter
class="solr.*SnowballPorterFilterFactory*" language="English"
protected="protwords.txt"/> <filter
class="solr.RemoveDuplicatesTokenFilterFactory"/>

I think I'm using the Standard Highlighter.

I’m using the Drupal 7 search api solr configuration files without
modification.


Thank you,

Ann

Re: Stem Words Highlighted - Keyword Not Highlighted

Posted by Ann B <al...@gmail.com>.
I removed the highlight specification before going to production.  The
results have the expected words.

Example: (search term stocks)

Includes free access to stock charts and instruction about using stock
charts in technical analysis of stocks. Paid subscriptions provide access
to more information.





On Tue, Aug 18, 2015 at 12:31 PM, Ann B <al...@gmail.com> wrote:

> Question:
>
> Can I configure solr to highlight the keyword also?  The search results
> are correct, but the highlighting is not complete.
>
> *
>
> Example:
>
> Keyword: stocks
>
> Request: (I only provided the url parameters below.)
>
> hl=true&
> hl.fl=spell&
> hl.simple.pre=%5BHIGHLIGHT%5D&
> hl.simple.post=%5B%2FHIGHLIGHT%5D&
> hl.snippets=3&
> hl.fragsize=70&
> hl.mergeContiguous=true&
>
> fl=item_id%2Cscore&
>
> qf=tm_body%3Avalue%5E1.0&
> qf=tm_title%5E13.0&
>
> fq=im_field_webresource_category%3A%226013%22&
> fq=index_id%3Atest&
>
>
> start=0&rows=10&facet=true&facet.sort=count&facet.limit=10&facet.mincount=1&facet.missing=false&facet.field=im_field_webresource_category&f.im_field_webresource_category.facet.limit=50&
>
> wt=json&json.nl=map&
>
> q=%22stocks%22
>
> *
>
> Response:
>
> "highlighting":{
> "test-49904":{"spell":[
> "Includes free access to [HIGHLIGHT]stock[/HIGHLIGHT] charts and
> instruction about using [HIGHLIGHT]stock[/HIGHLIGHT] charts in technical
> analysis of stocks. Paid subscriptions provide access to more
> information."]},...
>
> *
>
> Details:
>
> Tokenizer:  <tokenizer class="solr.WhitespaceTokenizerFactory"/>
>
> Filters:
>
> <filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt"
> ignoreCase="true" expand="true"/> <filter class="solr.StopFilterFactory"
> ignoreCase="true" words="stopwords.txt" enablePositionIncrements="true"/>
> <filter class="solr.WordDelimiterFilterFactory" protected="protwords.txt"
> generateWordParts="1" generateNumberParts="1" catenateWords="0"
> catenateNumbers="0" catenateAll="0" splitOnCaseChange="1"
> preserveOriginal="1"/> <filter class="solr.LengthFilterFactory" min="2"
> max="100"/> <filter class="solr.LowerCaseFilterFactory"/> <filter
> class="solr.*SnowballPorterFilterFactory*" language="English"
> protected="protwords.txt"/> <filter
> class="solr.RemoveDuplicatesTokenFilterFactory"/>
>
> I think I'm using the Standard Highlighter.
>
> I’m using the Drupal 7 search api solr configuration files without
> modification.
>
>
> Thank you,
>
> Ann
>
>
>
>
>
>
>
>

Re: Stem Words Highlighted - Keyword Not Highlighted

Posted by Ann B <al...@gmail.com>.
Thank you Jack.  The field I was passing to Solr actually uses the
following:

Tokenizer:  StandardTokenizerFactory

Filters:

StopFilterFactory
LengthFilterFactory
LowerCaseFilterFactory
RemoveDuplicatesTokenFilterFactory

Once I passed in the correct field that uses the white space tokenizer and
the
WordDelimiterFilterFactory, all is well.


On Thu, Oct 29, 2015 at 8:16 AM, Jack Krupansky <ja...@gmail.com>
wrote:

> Did you index the data before adding the word delimiter filter? The white
> space tokenizer preserves the period after "stocks.", but the WDF should
> remove it. The period is likely interfering with stemming.
>
> Are your filters the same for index time and query time?
>
> -- Jack Krupansky
>
> On Tue, Aug 18, 2015 at 3:31 PM, Ann B <al...@gmail.com> wrote:
>
> > Question:
> >
> > Can I configure solr to highlight the keyword also?  The search results
> are
> > correct, but the highlighting is not complete.
> >
> > *
> >
> > Example:
> >
> > Keyword: stocks
> >
> > Request: (I only provided the url parameters below.)
> >
> > hl=true&
> > hl.fl=spell&
> > hl.simple.pre=%5BHIGHLIGHT%5D&
> > hl.simple.post=%5B%2FHIGHLIGHT%5D&
> > hl.snippets=3&
> > hl.fragsize=70&
> > hl.mergeContiguous=true&
> >
> > fl=item_id%2Cscore&
> >
> > qf=tm_body%3Avalue%5E1.0&
> > qf=tm_title%5E13.0&
> >
> > fq=im_field_webresource_category%3A%226013%22&
> > fq=index_id%3Atest&
> >
> >
> >
> start=0&rows=10&facet=true&facet.sort=count&facet.limit=10&facet.mincount=1&facet.missing=false&facet.field=im_field_webresource_category&f.im_field_webresource_category.facet.limit=50&
> >
> > wt=json&json.nl=map&
> >
> > q=%22stocks%22
> >
> > *
> >
> > Response:
> >
> > "highlighting":{
> > "test-49904":{"spell":[
> > "Includes free access to [HIGHLIGHT]stock[/HIGHLIGHT] charts and
> > instruction about using [HIGHLIGHT]stock[/HIGHLIGHT] charts in technical
> > analysis of stocks. Paid subscriptions provide access to more
> > information."]},...
> >
> > *
> >
> > Details:
> >
> > Tokenizer:  <tokenizer class="solr.WhitespaceTokenizerFactory"/>
> >
> > Filters:
> >
> > <filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt"
> > ignoreCase="true" expand="true"/> <filter class="solr.StopFilterFactory"
> > ignoreCase="true" words="stopwords.txt" enablePositionIncrements="true"/>
> > <filter class="solr.WordDelimiterFilterFactory" protected="protwords.txt"
> > generateWordParts="1" generateNumberParts="1" catenateWords="0"
> > catenateNumbers="0" catenateAll="0" splitOnCaseChange="1"
> > preserveOriginal="1"/> <filter class="solr.LengthFilterFactory" min="2"
> > max="100"/> <filter class="solr.LowerCaseFilterFactory"/> <filter
> > class="solr.*SnowballPorterFilterFactory*" language="English"
> > protected="protwords.txt"/> <filter
> > class="solr.RemoveDuplicatesTokenFilterFactory"/>
> >
> > I think I'm using the Standard Highlighter.
> >
> > I’m using the Drupal 7 search api solr configuration files without
> > modification.
> >
> >
> > Thank you,
> >
> > Ann
> >
>

Re: Stem Words Highlighted - Keyword Not Highlighted

Posted by Jack Krupansky <ja...@gmail.com>.
Did you index the data before adding the word delimiter filter? The white
space tokenizer preserves the period after "stocks.", but the WDF should
remove it. The period is likely interfering with stemming.

Are your filters the same for index time and query time?

-- Jack Krupansky

On Tue, Aug 18, 2015 at 3:31 PM, Ann B <al...@gmail.com> wrote:

> Question:
>
> Can I configure solr to highlight the keyword also?  The search results are
> correct, but the highlighting is not complete.
>
> *
>
> Example:
>
> Keyword: stocks
>
> Request: (I only provided the url parameters below.)
>
> hl=true&
> hl.fl=spell&
> hl.simple.pre=%5BHIGHLIGHT%5D&
> hl.simple.post=%5B%2FHIGHLIGHT%5D&
> hl.snippets=3&
> hl.fragsize=70&
> hl.mergeContiguous=true&
>
> fl=item_id%2Cscore&
>
> qf=tm_body%3Avalue%5E1.0&
> qf=tm_title%5E13.0&
>
> fq=im_field_webresource_category%3A%226013%22&
> fq=index_id%3Atest&
>
>
> start=0&rows=10&facet=true&facet.sort=count&facet.limit=10&facet.mincount=1&facet.missing=false&facet.field=im_field_webresource_category&f.im_field_webresource_category.facet.limit=50&
>
> wt=json&json.nl=map&
>
> q=%22stocks%22
>
> *
>
> Response:
>
> "highlighting":{
> "test-49904":{"spell":[
> "Includes free access to [HIGHLIGHT]stock[/HIGHLIGHT] charts and
> instruction about using [HIGHLIGHT]stock[/HIGHLIGHT] charts in technical
> analysis of stocks. Paid subscriptions provide access to more
> information."]},...
>
> *
>
> Details:
>
> Tokenizer:  <tokenizer class="solr.WhitespaceTokenizerFactory"/>
>
> Filters:
>
> <filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt"
> ignoreCase="true" expand="true"/> <filter class="solr.StopFilterFactory"
> ignoreCase="true" words="stopwords.txt" enablePositionIncrements="true"/>
> <filter class="solr.WordDelimiterFilterFactory" protected="protwords.txt"
> generateWordParts="1" generateNumberParts="1" catenateWords="0"
> catenateNumbers="0" catenateAll="0" splitOnCaseChange="1"
> preserveOriginal="1"/> <filter class="solr.LengthFilterFactory" min="2"
> max="100"/> <filter class="solr.LowerCaseFilterFactory"/> <filter
> class="solr.*SnowballPorterFilterFactory*" language="English"
> protected="protwords.txt"/> <filter
> class="solr.RemoveDuplicatesTokenFilterFactory"/>
>
> I think I'm using the Standard Highlighter.
>
> I’m using the Drupal 7 search api solr configuration files without
> modification.
>
>
> Thank you,
>
> Ann
>