You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Jed Glazner <jg...@beyondoblivion.com> on 2010/09/08 19:26:29 UTC

Solr Highlighting Question

Thanks for taking time to read through this.  I'm using a checkout from

the solr 3.x branch

My problem is with the highlighter and wildcards

I can get the highlighter to work with wild cards just fine, the problem
is that  solr is returning the term matched, when what I want it to do
is highlight the chars in the term that were matched.


Example:

http://192.168.1.75:8983/solr/music/select?indent=on&q=name_title:wel*&qt=beyond&hl=true&hl.fl=name_title&f.name_title.hl.usePhraseHighlighter=true&f.name_title.hl.highlightMultiTerm=true

The results that come back look like this:

<em>Welcome</em>   to the Jungle

What I want them to look like is this:
<em>Wel</em>come to the Jungle

   From what I gathered by searching the archives is that solr 1.1 used to
do this... Is there a way to get that functionality?

Thanks!


Re: Solr Highlighting Question

Posted by Jed Glazner <jg...@beyondoblivion.com>.
Anybody?

On 09/08/2010 11:26 AM, Jed Glazner wrote:
> Thanks for taking time to read through this.  I'm using a checkout from
>
> the solr 3.x branch
>
> My problem is with the highlighter and wildcards
>
> I can get the highlighter to work with wild cards just fine, the problem
> is that  solr is returning the term matched, when what I want it to do
> is highlight the chars in the term that were matched.
>
>
> Example:
>
> http://192.168.1.75:8983/solr/music/select?indent=on&q=name_title:wel*&qt=beyond&hl=true&hl.fl=name_title&f.name_title.hl.usePhraseHighlighter=true&f.name_title.hl.highlightMultiTerm=true
>
> The results that come back look like this:
>
> <em>Welcome</em>    to the Jungle
>
> What I want them to look like is this:
> <em>Wel</em>come to the Jungle
>
>     From what I gathered by searching the archives is that solr 1.1 used to
> do this... Is there a way to get that functionality?
>
> Thanks!
>
>    


-- 
Jed Glazner (801-360-0181)

This email and its attachments (if any) are for the sole use of the 
intended recipient, and may contain private, confidential, and 
privileged material. Any review, copying, or distribution of this email, 
its attachments or the information contained herein is strictly 
prohibited. If you are not the intended recipient, please contact the 
sender immediately and permanently delete the original and any copies of 
this email and any attachments.


Re: Solr Highlighting Question

Posted by Koji Sekiguchi <ko...@r.email.ne.jp>.
  (10/09/25 8:07), Jed Glazner wrote:
> Hi Koji,
>
> I'm trying to get the FVH to work per your suggestion, but I think I must have
> something misconfigured...
>
> Here is the field def in my schema.xml:
>
> <fieldType name="nGram" class="solr.TextField">
> <analyzer>
> <tokenizer class="solr.LowerCaseTokenizerFactory"/>
> <filter class="solr.EdgeNGramFilterFactory" minGramSize="2" maxGramSize="15"
> side="front"/>
> </analyzer>
> </fieldType>
> <field name="grams" type="nGram" indexed="true" stored="false"
> multiValued="false" termVectors="true" termPositions="true" termOffsets="true"/>
>
> Then here is my request handler in solrconfig.xml:
>
> <requestHandler name="beyond" class="solr.SearchHandler" default="true">
> <lst name="defaults">
> <str name="defType">edismax</str>
> <str name="qf">name_title^3 plain^6 grams^1 soundsLike^1</str>
> <!-- automatic phrase! -->
> <!-- uncomment this to enable phrase query. it's disabled now because i saw a
> thread that said FVH didn't work with MPQ -->
> <!--<str name="pf">name_title^6 plain^12</str>-->
> <str name="tie">0.1</str>
> </lst>
> </requestHandler>
>
> But when I try the following query:
>
> http://192.168.1.75:8983/solr/music/select?q=beyo&qt=beyond&hl=true&fl.hl=grams&f.grams.hl.useFastVectorHighlighter=true
>
> There are hits, and results, but the highligher section is empty:
> <lst name="highlighting">
> <lst name="artist.artist.241"/>
> <lst name="artist.artist.1616"/>
> <lst name="track.track.7022"/>
> <lst name="album.album.1775"/>
> <lst name="album.album.3128"/>
> <lst name="album.album.4590"/>
> <lst name="album.album.6951"/>
> <lst name="album.album.9626"/>
> <lst name="artist.artist.10498"/>
> <lst name="artist.artist.29631"/>
> </lst>
>
> Any Ideas?
> On 09/08/2010 07:54 PM, Koji Sekiguchi wrote:
> >     (10/09/09 2:26), Jed Glazner wrote:
> >
> >>  Thanks for taking time to read through this.  I'm using a checkout from
> >>
> >>  the solr 3.x branch
> >>
> >>  My problem is with the highlighter and wildcards
> >>
> >>  I can get the highlighter to work with wild cards just fine, the problem
> >>  is that  solr is returning the term matched, when what I want it to do
> >>  is highlight the chars in the term that were matched.
> >>
> >>
> >>  Example:
> >>
> >>  http://192.168.1.75:8983/solr/music/select?indent=on&q=name_title:wel*&qt=beyond&hl=true&hl.fl=name_title&f.name_title.hl.usePhraseHighlighter=true&f.name_title.hl.highlightMultiTerm=true
> >>
> >>
> >>  The results that come back look like this:
> >>
> >>  <em>Welcome</em>     to the Jungle
> >>
> >>  What I want them to look like is this:
> >>  <em>Wel</em>come to the Jungle
> >>
> >>      From what I gathered by searching the archives is that solr 1.1 used to
> >>  do this... Is there a way to get that functionality?
> >>
> >>  Thanks!
> >>
> >>
> >>
> >  Try to use FastVectorHighlighter on n-gram field for highlighting problem...
> >  But FVH cannot process wildcard query. So you should query wel instead of
> >  wel*. Then this makes you got unwanted hit like vo<em>wel</em>.
> >  I don't think there is a solution for both of them with OOTB today.
> >
> >  There is a JIRA issue, but no patches there:
> >
> >  https://issues.apache.org/jira/browse/SOLR-1926
> >
> >  Koji
> >
> >
>
Hi Jed,

Unfortunately, FVH doesn't support variable-length-gram tokens.
For example, this should work:

<tokenizer class="solr.NGramTokenizerFactory" minGramSize="2" maxGramSize="2"/>

But this may not work:

<tokenizer class="solr.NGramTokenizerFactory" minGramSize="2" maxGramSize="4"/>

Can you try constant-length-gram tokens?

Thanks,

Koji

-- 
http://www.rondhuit.com/en/


Re: Solr Highlighting Question

Posted by Jed Glazner <jg...@beyondoblivion.com>.
Hi Koji,

I'm trying to get the FVH to work per your suggestion, but I think I 
must have something misconfigured...

Here is the field def in my schema.xml:

<fieldType name="nGram" class="solr.TextField">
<analyzer>
<tokenizer class="solr.LowerCaseTokenizerFactory"/>
<filter class="solr.EdgeNGramFilterFactory" minGramSize="2" 
maxGramSize="15" side="front"/>
</analyzer>
</fieldType>
<field name="grams" type="nGram" indexed="true" stored="false" 
multiValued="false" termVectors="true" termPositions="true" 
termOffsets="true"/>

Then here is my request handler in solrconfig.xml:

<requestHandler name="beyond" class="solr.SearchHandler" default="true">
<lst name="defaults">
<str name="defType">edismax</str>
<str name="qf">name_title^3 plain^6 grams^1 soundsLike^1</str>
<!-- automatic phrase! -->
<!-- uncomment this to enable phrase query.  it's disabled now because i 
saw a thread that said FVH didn't work with MPQ -->
<!--<str name="pf">name_title^6 plain^12</str>-->
<str name="tie">0.1</str>
</lst>
</requestHandler>

But when I try the following query:

http://192.168.1.75:8983/solr/music/select?q=beyo&qt=beyond&hl=true&fl.hl=grams&f.grams.hl.useFastVectorHighlighter=true

There are hits, and results, but the highligher section is empty:
<lst name="highlighting">
<lst name="artist.artist.241"/>
<lst name="artist.artist.1616"/>
<lst name="track.track.7022"/>
<lst name="album.album.1775"/>
<lst name="album.album.3128"/>
<lst name="album.album.4590"/>
<lst name="album.album.6951"/>
<lst name="album.album.9626"/>
<lst name="artist.artist.10498"/>
<lst name="artist.artist.29631"/>
</lst>

Any Ideas?
On 09/08/2010 07:54 PM, Koji Sekiguchi wrote:
>    (10/09/09 2:26), Jed Glazner wrote:
>    
>> Thanks for taking time to read through this.  I'm using a checkout from
>>
>> the solr 3.x branch
>>
>> My problem is with the highlighter and wildcards
>>
>> I can get the highlighter to work with wild cards just fine, the problem
>> is that  solr is returning the term matched, when what I want it to do
>> is highlight the chars in the term that were matched.
>>
>>
>> Example:
>>
>> http://192.168.1.75:8983/solr/music/select?indent=on&q=name_title:wel*&qt=beyond&hl=true&hl.fl=name_title&f.name_title.hl.usePhraseHighlighter=true&f.name_title.hl.highlightMultiTerm=true
>>
>>
>> The results that come back look like this:
>>
>> <em>Welcome</em>    to the Jungle
>>
>> What I want them to look like is this:
>> <em>Wel</em>come to the Jungle
>>
>>    From what I gathered by searching the archives is that solr 1.1 used to
>> do this... Is there a way to get that functionality?
>>
>> Thanks!
>>
>>
>>      
> Try to use FastVectorHighlighter on n-gram field for highlighting problem...
> But FVH cannot process wildcard query. So you should query wel instead of
> wel*. Then this makes you got unwanted hit like vo<em>wel</em>.
> I don't think there is a solution for both of them with OOTB today.
>
> There is a JIRA issue, but no patches there:
>
> https://issues.apache.org/jira/browse/SOLR-1926
>
> Koji
>
>    


-- 
Jed Glazner (801-360-0181)

This email and its attachments (if any) are for the sole use of the 
intended recipient, and may contain private, confidential, and 
privileged material. Any review, copying, or distribution of this email, 
its attachments or the information contained herein is strictly 
prohibited. If you are not the intended recipient, please contact the 
sender immediately and permanently delete the original and any copies of 
this email and any attachments.


Re: Solr Highlighting Question

Posted by Koji Sekiguchi <ko...@r.email.ne.jp>.
  (10/09/09 2:26), Jed Glazner wrote:
> Thanks for taking time to read through this.  I'm using a checkout from
>
> the solr 3.x branch
>
> My problem is with the highlighter and wildcards
>
> I can get the highlighter to work with wild cards just fine, the problem
> is that  solr is returning the term matched, when what I want it to do
> is highlight the chars in the term that were matched.
>
>
> Example:
>
> http://192.168.1.75:8983/solr/music/select?indent=on&q=name_title:wel*&qt=beyond&hl=true&hl.fl=name_title&f.name_title.hl.usePhraseHighlighter=true&f.name_title.hl.highlightMultiTerm=true 
>
>
> The results that come back look like this:
>
> <em>Welcome</em>   to the Jungle
>
> What I want them to look like is this:
> <em>Wel</em>come to the Jungle
>
>   From what I gathered by searching the archives is that solr 1.1 used to
> do this... Is there a way to get that functionality?
>
> Thanks!
>
>
Try to use FastVectorHighlighter on n-gram field for highlighting problem...
But FVH cannot process wildcard query. So you should query wel instead of
wel*. Then this makes you got unwanted hit like vo<em>wel</em>.
I don't think there is a solution for both of them with OOTB today.

There is a JIRA issue, but no patches there:

https://issues.apache.org/jira/browse/SOLR-1926

Koji

-- 
http://www.rondhuit.com/en/