You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by 12rad <pr...@gmail.com> on 2012/05/21 02:56:11 UTC
Not able to use the highlighting feature! Want to return snippets
of text
I indexed a structured pdf document in Solr. The problem is when I search
for a simple string - I get the entire content field as the response! I
don't know how to change that.
My requirement is that, lets say I search for "metadata" it should give me
"*Metadata*Discussion . . . 4 matches ... make sure that Tika users have a
chance to get to all of the* metadata* created and/or extracted by Tika. ==
Original Problem == The original inspiration for this page was a Tika ...
10.7k - rev: 2 (current) last modified: 2010-08-02 18:09:45 "
But it gives me the whole document!- the entire string that was indexed. It
seems like Lucene can only tell me in which field it occurred, not where in
the field it occurred
I posted the document like this
curl
"http://localhost:8983/solr/update/extract?stream.file=/home/Desktop/DOCUMENTS/T.pdf&stream.contentType=application/pdf&literal.id=DOC_N&commit=true&captureAttr=true"
A query of *:* gives me the entire content of the document indexed in the
<text> field And any search also returns the same thing.
Any help will be greatly appreciated!
Any help will be greatly appreciated!!
--
View this message in context: http://lucene.472066.n3.nabble.com/Not-able-to-use-the-highlighting-feature-Want-to-return-snippets-of-text-tp3985012.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Not able to use the highlighting feature! Want to return
snippets of text. Urgent!!
Posted by 12rad <pr...@gmail.com>.
Also,
the response just returns
<lst name="highlighting">
<lst name="DOC_N"/>
</lst>
That is the name of the document.
--
View this message in context: http://lucene.472066.n3.nabble.com/Not-able-to-use-the-highlighting-feature-Want-to-return-snippets-of-text-Urgent-tp3985012p3985013.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Not able to use the highlighting feature! Want to return
snippets of text
Posted by 12rad <pr...@gmail.com>.
That worked!
Thanks!
I did <str name="hl.simple.pre"> </str>
<str name="hl.simple.post"> </str>
--
View this message in context: http://lucene.472066.n3.nabble.com/Not-able-to-use-the-highlighting-feature-Want-to-return-snippets-of-text-Urgent-tp3985012p3985507.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Not able to use the highlighting feature! Want to return snippets
of text
Posted by Rahul Warawdekar <ra...@gmail.com>.
Hi,
I believe, in your "colored" fragmentsBuilder definition, you have not
mentioned anything in your pre and post tags and that may be the reason
that you are getting snippets of text, without highlighting.
Please refer http://wiki.apache.org/solr/HighlightingParameters and check
the "hl.fragmentsBuilder" section.
Try specifying the pre and post tags with information as mentioned below.
(same as wiki link above)
<!-- multi-colored tag FragmentsBuilder -->
<fragmentsBuilder name="colored"
class="org.apache.solr.highlight.ScoreOrderFragmentsBuilder">
<lst name="defaults">
<str name="hl.tag.pre"><![CDATA[
<b style="background:yellow">,<b style="background:lawgreen">,
<b style="background:aquamarine">,<b style="background:magenta">,
<b style="background:palegreen">,<b style="background:coral">,
<b style="background:wheat">,<b style="background:khaki">,
<b style="background:lime">,<b
style="background:deepskyblue">]]></str>
<str name="hl.tag.post"><![CDATA[</b>]]></str>
</lst>
</fragmentsBuilder>
On Mon, May 21, 2012 at 3:52 PM, 12rad <pr...@gmail.com> wrote:
> For the fragListBuilder
> it's
> <fragListBuilder name="simple"
> default="true"
> class="solr.highlight.SimpleFragListBuilder"/>
>
> fragment builder is
> <fragmentsBuilder name="colored"
> class="solr.highlight.ScoreOrderFragmentsBuilder">
> <lst name="defaults">
> <str name="hl.tag.pre"></str>
> <str name="hl.tag.post"></str>
> </lst>
> </fragmentsBuilder>
>
>
> <fragmenter name="regex"
> class="solr.highlight.RegexFragmenter">
> <lst name="defaults">
>
> <int name="hl.fragsize">70</int>
>
> <float name="hl.regex.slop">0.5</float>
>
> <str name="hl.regex.pattern">[-\w ,/\n\"']{20,200}</str>
> </lst>
> </fragmenter>
>
>
> Thanks!
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Not-able-to-use-the-highlighting-feature-Want-to-return-snippets-of-text-Urgent-tp3985012p3985212.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
--
Thanks and Regards
Rahul A. Warawdekar
Re: Not able to use the highlighting feature! Want to return
snippets of text
Posted by 12rad <pr...@gmail.com>.
For the fragListBuilder
it's
<fragListBuilder name="simple"
default="true"
class="solr.highlight.SimpleFragListBuilder"/>
fragment builder is
<fragmentsBuilder name="colored"
class="solr.highlight.ScoreOrderFragmentsBuilder">
<lst name="defaults">
<str name="hl.tag.pre"></str>
<str name="hl.tag.post"></str>
</lst>
</fragmentsBuilder>
<fragmenter name="regex"
class="solr.highlight.RegexFragmenter">
<lst name="defaults">
<int name="hl.fragsize">70</int>
<float name="hl.regex.slop">0.5</float>
<str name="hl.regex.pattern">[-\w ,/\n\"']{20,200}</str>
</lst>
</fragmenter>
Thanks!
--
View this message in context: http://lucene.472066.n3.nabble.com/Not-able-to-use-the-highlighting-feature-Want-to-return-snippets-of-text-Urgent-tp3985012p3985212.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Not able to use the highlighting feature! Want to return snippets
of text
Posted by Rahul Warawdekar <ra...@gmail.com>.
Hi,
Can you please provide the definitions of the following 3 objects from your
solrconfig.xml ?
<str name ="hl.fragListBuilder">simple</str>
<str name ="hl.fragmentsBuilder">colored</str>
<str name="hl.fragmenter">regex</str>
For eg,
the "simple" hl.fragListBuilder should be defined as mentioned below in
your solrconfig.xml
<fragListBuilder name="simple"
class="org.apache.solr.highlight.SimpleFragListBuilder" default="true"/>
On Mon, May 21, 2012 at 2:06 PM, 12rad <pr...@gmail.com> wrote:
> The field I am trying to highlight is stored.
>
>
> <field name="text" type="text_en" required="false" compressed="false"
> omitNorms="false"
> indexed="true" stored="true" multiValued="true" termVectors="true"
> termPositions="true"
> termOffsets="true"/>
>
>
> In the searchHandler i've set the parameters as follows:
>
> <str name="hl">on</str>
> <str name="hl.fl">text</str>
> <str name ="hl.snippets">5</str>
> <str name="hl.fragsize">1000</str>
> <str name="hl.maxAnalyzedChars">510000</str>
> <str name="hl.requireFieldMatch">true</str>
> <str name="hl.fragmenter">regex</str>
> <str name ="hl.fragListBuilder">simple</str>
> <str name ="hl.fragmentsBuilder">colored</str>
> <str name="hl.phraseLimit">1000</str>
> <str name="hl.usePhraseHighlighter">true</str>
> <str name="hl.highlightMultiTerm">true</str>
> <str name ="hl.useFastVectorHighligher">true</str>
>
>
> I still don't see any highlighting. I've managed to get snippets of text
> but
> the actual word is not highlighted. I don't know where I am going wrong?
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Not-able-to-use-the-highlighting-feature-Want-to-return-snippets-of-text-Urgent-tp3985012p3985174.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
--
Thanks and Regards
Rahul A. Warawdekar
Re: Not able to use the highlighting feature! Want to return
snippets of text
Posted by 12rad <pr...@gmail.com>.
The field I am trying to highlight is stored.
<field name="text" type="text_en" required="false" compressed="false"
omitNorms="false"
indexed="true" stored="true" multiValued="true" termVectors="true"
termPositions="true"
termOffsets="true"/>
In the searchHandler i've set the parameters as follows:
<str name="hl">on</str>
<str name="hl.fl">text</str>
<str name ="hl.snippets">5</str>
<str name="hl.fragsize">1000</str>
<str name="hl.maxAnalyzedChars">510000</str>
<str name="hl.requireFieldMatch">true</str>
<str name="hl.fragmenter">regex</str>
<str name ="hl.fragListBuilder">simple</str>
<str name ="hl.fragmentsBuilder">colored</str>
<str name="hl.phraseLimit">1000</str>
<str name="hl.usePhraseHighlighter">true</str>
<str name="hl.highlightMultiTerm">true</str>
<str name ="hl.useFastVectorHighligher">true</str>
I still don't see any highlighting. I've managed to get snippets of text but
the actual word is not highlighted. I don't know where I am going wrong?
--
View this message in context: http://lucene.472066.n3.nabble.com/Not-able-to-use-the-highlighting-feature-Want-to-return-snippets-of-text-Urgent-tp3985012p3985174.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Not able to use the highlighting feature! Want to return snippets of text
Posted by Jack Krupansky <ja...@basetechnology.com>.
Take a look at the "/browse" request handler in the example solrconfig.xml
and compare how it does highlighting to what you are doing. There are a lot
of little details, so maybe even one might be missing.
Also, you can only highlight stored fields, so make sure that "text" is
stored. In the Solr example it is not stored and not intended to be stored,
and highlighting should be performed using some other field containing the
text as a stored field.
-- Jack Krupansky
-----Original Message-----
From: 12rad
Sent: Sunday, May 20, 2012 11:26 PM
To: solr-user@lucene.apache.org
Subject: Re: Not able to use the highlighting feature! Want to return
snippets of text
My query parameters are this:
text:abstract&hl=true&hl.fl=text&f.text.hl.snippets=2&f.text.hl.fragsize=200&debugQuery=true
I still get the entire string as the result in the
<lst name="highlighting"> tag.
--
View this message in context:
http://lucene.472066.n3.nabble.com/Not-able-to-use-the-highlighting-feature-Want-to-return-snippets-of-text-Urgent-tp3985012p3985022.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Not able to use the highlighting feature! Want to return snippets of text
Posted by Ahmet Arslan <io...@yahoo.com>.
> text:abstract&hl=true&hl.fl=text&f.text.hl.snippets=2&f.text.hl.fragsize=200&debugQuery=true
Three things to check:
1-) See your text field declared as suitable for highlighting.
http://wiki.apache.org/solr/FieldOptionsByUseCase
2-) Increase hl.maxAnalyzedChars=Integer.MAX
3-) Increase <maxFieldLength>Integer.MAX</maxFieldLength>
For some reason (complex analysis etc) snippets cannot be always generated. For this cases consider using hl.alternateField and hl.maxAlternateFieldLength
Re: Not able to use the highlighting feature! Want to return
snippets of text
Posted by 12rad <pr...@gmail.com>.
My query parameters are this:
text:abstract&hl=true&hl.fl=text&f.text.hl.snippets=2&f.text.hl.fragsize=200&debugQuery=true
I still get the entire string as the result in the
<lst name="highlighting"> tag.
--
View this message in context: http://lucene.472066.n3.nabble.com/Not-able-to-use-the-highlighting-feature-Want-to-return-snippets-of-text-Urgent-tp3985012p3985022.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Not able to use the highlighting feature! Want to return snippets of text
Posted by Jack Krupansky <ja...@basetechnology.com>.
Read up on highlighting here:
http://wiki.apache.org/solr/HighlightingParameters
And then look at solrconfig.xml for how it is used there.
-- Jack Krupansky
-----Original Message-----
From: 12rad
Sent: Sunday, May 20, 2012 8:56 PM
To: solr-user@lucene.apache.org
Subject: Not able to use the highlighting feature! Want to return snippets
of text
I indexed a structured pdf document in Solr. The problem is when I search
for a simple string - I get the entire content field as the response! I
don't know how to change that.
My requirement is that, lets say I search for "metadata" it should give me
"*Metadata*Discussion . . . 4 matches ... make sure that Tika users have a
chance to get to all of the* metadata* created and/or extracted by Tika. ==
Original Problem == The original inspiration for this page was a Tika ...
10.7k - rev: 2 (current) last modified: 2010-08-02 18:09:45 "
But it gives me the whole document!- the entire string that was indexed. It
seems like Lucene can only tell me in which field it occurred, not where in
the field it occurred
I posted the document like this
curl
"http://localhost:8983/solr/update/extract?stream.file=/home/Desktop/DOCUMENTS/T.pdf&stream.contentType=application/pdf&literal.id=DOC_N&commit=true&captureAttr=true"
A query of *:* gives me the entire content of the document indexed in the
<text> field And any search also returns the same thing.
Any help will be greatly appreciated!
Any help will be greatly appreciated!!
--
View this message in context:
http://lucene.472066.n3.nabble.com/Not-able-to-use-the-highlighting-feature-Want-to-return-snippets-of-text-tp3985012.html
Sent from the Solr - User mailing list archive at Nabble.com.