You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Paul Tomblin <pt...@xcski.com> on 2009/09/11 00:19:59 UTC
Highlighting in SolrJ?
Can somebody point me to some sample code for using highlighting in
SolrJ? I understand the highlighted versions of the field comes in a
separate NamedList? How does that work?
--
http://www.linkedin.com/in/paultomblin
Re: Highlighting in SolrJ?
Posted by Shalin Shekhar Mangar <sh...@gmail.com>.
Thanks Jay!
On Sat, Sep 12, 2009 at 10:03 PM, Jay Hill <ja...@gmail.com> wrote:
> Will do Shalin.
>
> -Jay
> http://www.lucidimagination.com
>
>
> On Fri, Sep 11, 2009 at 9:23 PM, Shalin Shekhar Mangar <
> shalinmangar@gmail.com> wrote:
>
> > Jay, it would be great if you can add this example to the Solrj wiki:
> >
> > http://wiki.apache.org/solr/Solrj
> >
> > On Fri, Sep 11, 2009 at 5:15 AM, Jay Hill <ja...@gmail.com>
> wrote:
> >
> > > Set up the query like this to highlight a field named "content":
> > >
> > > SolrQuery query = new SolrQuery();
> > > query.setQuery("foo");
> > >
> > > query.setHighlight(true).setHighlightSnippets(1); //set other params
> > as
> > > needed
> > > query.setParam("hl.fl", "content");
> > >
> > > QueryResponse queryResponse =getSolrServer().query(query);
> > >
> > > Then to get back the highlight results you need something like this:
> > >
> > > Iterator<SolrDocument> iter = queryResponse.getResults();
> > >
> > > while (iter.hasNext()) {
> > > SolrDocument resultDoc = iter.next();
> > >
> > > String content = (String) resultDoc.getFieldValue("content"));
> > > String id = (String) resultDoc.getFieldValue("id"); //id is the
> > > uniqueKey field
> > >
> > > if (queryResponse.getHighlighting().get(id) != null) {
> > > List<String> highightSnippets =
> > > queryResponse.getHighlighting().get(id).get("content");
> > > }
> > > }
> > >
> > > Hope that gets you what you need.
> > >
> > > -Jay
> > > http://www.lucidimagination.com
> > >
> > > On Thu, Sep 10, 2009 at 3:19 PM, Paul Tomblin <pt...@xcski.com>
> > wrote:
> > >
> > > > Can somebody point me to some sample code for using highlighting in
> > > > SolrJ? I understand the highlighted versions of the field comes in a
> > > > separate NamedList? How does that work?
> > > >
> > > > --
> > > > http://www.linkedin.com/in/paultomblin
> > > >
> > >
> >
> >
> >
> > --
> > Regards,
> > Shalin Shekhar Mangar.
> >
>
--
Regards,
Shalin Shekhar Mangar.
Re: Highlighting in SolrJ?
Posted by Jay Hill <ja...@gmail.com>.
Will do Shalin.
-Jay
http://www.lucidimagination.com
On Fri, Sep 11, 2009 at 9:23 PM, Shalin Shekhar Mangar <
shalinmangar@gmail.com> wrote:
> Jay, it would be great if you can add this example to the Solrj wiki:
>
> http://wiki.apache.org/solr/Solrj
>
> On Fri, Sep 11, 2009 at 5:15 AM, Jay Hill <ja...@gmail.com> wrote:
>
> > Set up the query like this to highlight a field named "content":
> >
> > SolrQuery query = new SolrQuery();
> > query.setQuery("foo");
> >
> > query.setHighlight(true).setHighlightSnippets(1); //set other params
> as
> > needed
> > query.setParam("hl.fl", "content");
> >
> > QueryResponse queryResponse =getSolrServer().query(query);
> >
> > Then to get back the highlight results you need something like this:
> >
> > Iterator<SolrDocument> iter = queryResponse.getResults();
> >
> > while (iter.hasNext()) {
> > SolrDocument resultDoc = iter.next();
> >
> > String content = (String) resultDoc.getFieldValue("content"));
> > String id = (String) resultDoc.getFieldValue("id"); //id is the
> > uniqueKey field
> >
> > if (queryResponse.getHighlighting().get(id) != null) {
> > List<String> highightSnippets =
> > queryResponse.getHighlighting().get(id).get("content");
> > }
> > }
> >
> > Hope that gets you what you need.
> >
> > -Jay
> > http://www.lucidimagination.com
> >
> > On Thu, Sep 10, 2009 at 3:19 PM, Paul Tomblin <pt...@xcski.com>
> wrote:
> >
> > > Can somebody point me to some sample code for using highlighting in
> > > SolrJ? I understand the highlighted versions of the field comes in a
> > > separate NamedList? How does that work?
> > >
> > > --
> > > http://www.linkedin.com/in/paultomblin
> > >
> >
>
>
>
> --
> Regards,
> Shalin Shekhar Mangar.
>
Re: Highlighting in SolrJ?
Posted by Shalin Shekhar Mangar <sh...@gmail.com>.
Jay, it would be great if you can add this example to the Solrj wiki:
http://wiki.apache.org/solr/Solrj
On Fri, Sep 11, 2009 at 5:15 AM, Jay Hill <ja...@gmail.com> wrote:
> Set up the query like this to highlight a field named "content":
>
> SolrQuery query = new SolrQuery();
> query.setQuery("foo");
>
> query.setHighlight(true).setHighlightSnippets(1); //set other params as
> needed
> query.setParam("hl.fl", "content");
>
> QueryResponse queryResponse =getSolrServer().query(query);
>
> Then to get back the highlight results you need something like this:
>
> Iterator<SolrDocument> iter = queryResponse.getResults();
>
> while (iter.hasNext()) {
> SolrDocument resultDoc = iter.next();
>
> String content = (String) resultDoc.getFieldValue("content"));
> String id = (String) resultDoc.getFieldValue("id"); //id is the
> uniqueKey field
>
> if (queryResponse.getHighlighting().get(id) != null) {
> List<String> highightSnippets =
> queryResponse.getHighlighting().get(id).get("content");
> }
> }
>
> Hope that gets you what you need.
>
> -Jay
> http://www.lucidimagination.com
>
> On Thu, Sep 10, 2009 at 3:19 PM, Paul Tomblin <pt...@xcski.com> wrote:
>
> > Can somebody point me to some sample code for using highlighting in
> > SolrJ? I understand the highlighted versions of the field comes in a
> > separate NamedList? How does that work?
> >
> > --
> > http://www.linkedin.com/in/paultomblin
> >
>
--
Regards,
Shalin Shekhar Mangar.
Re: Highlighting in SolrJ?
Posted by Jay Hill <ja...@gmail.com>.
Try adding this param: &hl.fragsize=30000
(obviously set the fragsize to whatever very high number you need for your
largest doc.)
-Jay
On Fri, Sep 11, 2009 at 7:54 AM, Paul Tomblin <pt...@xcski.com> wrote:
> What I want is the whole text of that field with every instance of the
> search term high lighted, even if the search term only occurs in the
> first line of a 300 page field. I'm not sure if mergeContinuous will
> do that, or if it will miss everything after the last line that
> contains the search term.
>
> On Fri, Sep 11, 2009 at 10:42 AM, Jay Hill <ja...@gmail.com> wrote:
> > It's really just a matter of what you're intentions are. There are an
> awful
> > lot of highlighting params and so highlighting is very flexible and
> > customizable. Regarding snippets, as an example Google presents two
> snippets
> > in results, which is fairly common. I'd recommend doing a lot of
> > experimenting by changing the params on the query string to get what you
> > want, and then setting them up in SolrJ. The example I sent was intended
> to
> > be a generic starting point and mostly just to show how to set
> highlighting
> > params and how to get back a List of highlighting results.
> >
> > -Jay
> > http://www.lucidimagination.com
> >
> >
> > On Thu, Sep 10, 2009 at 5:40 PM, Paul Tomblin <pt...@xcski.com>
> wrote:
> >
> >> If I set snippets to 99999 and "mergeContinuous" to true, will I get
> >> the entire contents of the field with all the search terms replaced?
> >> I don't see what good it would be just getting one line out of the
> >> whole field as a snippet.
> >>
> >> On Thu, Sep 10, 2009 at 7:45 PM, Jay Hill <ja...@gmail.com>
> wrote:
> >> > Set up the query like this to highlight a field named "content":
> >> >
> >> > SolrQuery query = new SolrQuery();
> >> > query.setQuery("foo");
> >> >
> >> > query.setHighlight(true).setHighlightSnippets(1); //set other
> params
> >> as
> >> > needed
> >> > query.setParam("hl.fl", "content");
> >> >
> >> > QueryResponse queryResponse =getSolrServer().query(query);
> >> >
> >> > Then to get back the highlight results you need something like this:
> >> >
> >> > Iterator<SolrDocument> iter = queryResponse.getResults();
> >> >
> >> > while (iter.hasNext()) {
> >> > SolrDocument resultDoc = iter.next();
> >> >
> >> > String content = (String) resultDoc.getFieldValue("content"));
> >> > String id = (String) resultDoc.getFieldValue("id"); //id is the
> >> > uniqueKey field
> >> >
> >> > if (queryResponse.getHighlighting().get(id) != null) {
> >> > List<String> highightSnippets =
> >> > queryResponse.getHighlighting().get(id).get("content");
> >> > }
> >> > }
> >> >
> >> > Hope that gets you what you need.
> >> >
> >> > -Jay
> >> > http://www.lucidimagination.com
> >> >
> >> > On Thu, Sep 10, 2009 at 3:19 PM, Paul Tomblin <pt...@xcski.com>
> >> wrote:
> >> >
> >> >> Can somebody point me to some sample code for using highlighting in
> >> >> SolrJ? I understand the highlighted versions of the field comes in a
> >> >> separate NamedList? How does that work?
> >> >>
> >> >> --
> >> >> http://www.linkedin.com/in/paultomblin
> >> >>
> >> >
> >>
> >>
> >>
> >> --
> >> http://www.linkedin.com/in/paultomblin
> >>
> >
>
>
>
> --
> http://www.linkedin.com/in/paultomblin
>
Re: Highlighting in SolrJ?
Posted by Paul Tomblin <pt...@xcski.com>.
What I want is the whole text of that field with every instance of the
search term high lighted, even if the search term only occurs in the
first line of a 300 page field. I'm not sure if mergeContinuous will
do that, or if it will miss everything after the last line that
contains the search term.
On Fri, Sep 11, 2009 at 10:42 AM, Jay Hill <ja...@gmail.com> wrote:
> It's really just a matter of what you're intentions are. There are an awful
> lot of highlighting params and so highlighting is very flexible and
> customizable. Regarding snippets, as an example Google presents two snippets
> in results, which is fairly common. I'd recommend doing a lot of
> experimenting by changing the params on the query string to get what you
> want, and then setting them up in SolrJ. The example I sent was intended to
> be a generic starting point and mostly just to show how to set highlighting
> params and how to get back a List of highlighting results.
>
> -Jay
> http://www.lucidimagination.com
>
>
> On Thu, Sep 10, 2009 at 5:40 PM, Paul Tomblin <pt...@xcski.com> wrote:
>
>> If I set snippets to 99999 and "mergeContinuous" to true, will I get
>> the entire contents of the field with all the search terms replaced?
>> I don't see what good it would be just getting one line out of the
>> whole field as a snippet.
>>
>> On Thu, Sep 10, 2009 at 7:45 PM, Jay Hill <ja...@gmail.com> wrote:
>> > Set up the query like this to highlight a field named "content":
>> >
>> > SolrQuery query = new SolrQuery();
>> > query.setQuery("foo");
>> >
>> > query.setHighlight(true).setHighlightSnippets(1); //set other params
>> as
>> > needed
>> > query.setParam("hl.fl", "content");
>> >
>> > QueryResponse queryResponse =getSolrServer().query(query);
>> >
>> > Then to get back the highlight results you need something like this:
>> >
>> > Iterator<SolrDocument> iter = queryResponse.getResults();
>> >
>> > while (iter.hasNext()) {
>> > SolrDocument resultDoc = iter.next();
>> >
>> > String content = (String) resultDoc.getFieldValue("content"));
>> > String id = (String) resultDoc.getFieldValue("id"); //id is the
>> > uniqueKey field
>> >
>> > if (queryResponse.getHighlighting().get(id) != null) {
>> > List<String> highightSnippets =
>> > queryResponse.getHighlighting().get(id).get("content");
>> > }
>> > }
>> >
>> > Hope that gets you what you need.
>> >
>> > -Jay
>> > http://www.lucidimagination.com
>> >
>> > On Thu, Sep 10, 2009 at 3:19 PM, Paul Tomblin <pt...@xcski.com>
>> wrote:
>> >
>> >> Can somebody point me to some sample code for using highlighting in
>> >> SolrJ? I understand the highlighted versions of the field comes in a
>> >> separate NamedList? How does that work?
>> >>
>> >> --
>> >> http://www.linkedin.com/in/paultomblin
>> >>
>> >
>>
>>
>>
>> --
>> http://www.linkedin.com/in/paultomblin
>>
>
--
http://www.linkedin.com/in/paultomblin
Re: Highlighting in SolrJ?
Posted by Jay Hill <ja...@gmail.com>.
It's really just a matter of what you're intentions are. There are an awful
lot of highlighting params and so highlighting is very flexible and
customizable. Regarding snippets, as an example Google presents two snippets
in results, which is fairly common. I'd recommend doing a lot of
experimenting by changing the params on the query string to get what you
want, and then setting them up in SolrJ. The example I sent was intended to
be a generic starting point and mostly just to show how to set highlighting
params and how to get back a List of highlighting results.
-Jay
http://www.lucidimagination.com
On Thu, Sep 10, 2009 at 5:40 PM, Paul Tomblin <pt...@xcski.com> wrote:
> If I set snippets to 99999 and "mergeContinuous" to true, will I get
> the entire contents of the field with all the search terms replaced?
> I don't see what good it would be just getting one line out of the
> whole field as a snippet.
>
> On Thu, Sep 10, 2009 at 7:45 PM, Jay Hill <ja...@gmail.com> wrote:
> > Set up the query like this to highlight a field named "content":
> >
> > SolrQuery query = new SolrQuery();
> > query.setQuery("foo");
> >
> > query.setHighlight(true).setHighlightSnippets(1); //set other params
> as
> > needed
> > query.setParam("hl.fl", "content");
> >
> > QueryResponse queryResponse =getSolrServer().query(query);
> >
> > Then to get back the highlight results you need something like this:
> >
> > Iterator<SolrDocument> iter = queryResponse.getResults();
> >
> > while (iter.hasNext()) {
> > SolrDocument resultDoc = iter.next();
> >
> > String content = (String) resultDoc.getFieldValue("content"));
> > String id = (String) resultDoc.getFieldValue("id"); //id is the
> > uniqueKey field
> >
> > if (queryResponse.getHighlighting().get(id) != null) {
> > List<String> highightSnippets =
> > queryResponse.getHighlighting().get(id).get("content");
> > }
> > }
> >
> > Hope that gets you what you need.
> >
> > -Jay
> > http://www.lucidimagination.com
> >
> > On Thu, Sep 10, 2009 at 3:19 PM, Paul Tomblin <pt...@xcski.com>
> wrote:
> >
> >> Can somebody point me to some sample code for using highlighting in
> >> SolrJ? I understand the highlighted versions of the field comes in a
> >> separate NamedList? How does that work?
> >>
> >> --
> >> http://www.linkedin.com/in/paultomblin
> >>
> >
>
>
>
> --
> http://www.linkedin.com/in/paultomblin
>
Re: Highlighting in SolrJ?
Posted by Paul Tomblin <pt...@xcski.com>.
If I set snippets to 99999 and "mergeContinuous" to true, will I get
the entire contents of the field with all the search terms replaced?
I don't see what good it would be just getting one line out of the
whole field as a snippet.
On Thu, Sep 10, 2009 at 7:45 PM, Jay Hill <ja...@gmail.com> wrote:
> Set up the query like this to highlight a field named "content":
>
> SolrQuery query = new SolrQuery();
> query.setQuery("foo");
>
> query.setHighlight(true).setHighlightSnippets(1); //set other params as
> needed
> query.setParam("hl.fl", "content");
>
> QueryResponse queryResponse =getSolrServer().query(query);
>
> Then to get back the highlight results you need something like this:
>
> Iterator<SolrDocument> iter = queryResponse.getResults();
>
> while (iter.hasNext()) {
> SolrDocument resultDoc = iter.next();
>
> String content = (String) resultDoc.getFieldValue("content"));
> String id = (String) resultDoc.getFieldValue("id"); //id is the
> uniqueKey field
>
> if (queryResponse.getHighlighting().get(id) != null) {
> List<String> highightSnippets =
> queryResponse.getHighlighting().get(id).get("content");
> }
> }
>
> Hope that gets you what you need.
>
> -Jay
> http://www.lucidimagination.com
>
> On Thu, Sep 10, 2009 at 3:19 PM, Paul Tomblin <pt...@xcski.com> wrote:
>
>> Can somebody point me to some sample code for using highlighting in
>> SolrJ? I understand the highlighted versions of the field comes in a
>> separate NamedList? How does that work?
>>
>> --
>> http://www.linkedin.com/in/paultomblin
>>
>
--
http://www.linkedin.com/in/paultomblin
Re: Highlighting in SolrJ?
Posted by Paul Tomblin <pt...@xcski.com>.
Thanks to Jay, I have my code doing what I need it to do. If anybody
cares, this is my code:
SolrQuery query = new SolrQuery();
query.setQuery(searchTerm);
query.addFilterQuery(Chunk.SOLR_KEY_CONCEPT + ":" + concept);
query.addFilterQuery(Chunk.SOLR_KEY_CATEGORY + ":" + category);
if (maxChunks > 0)
query.setRows(maxChunks);
// Set highlighting fields
query.setHighlight(true);
query.setHighlightFragsize(0);
query.addHighlightField(Chunk.SOLR_KEY_TEXT);
query.setHighlightSnippets(1);
query.setHighlightSimplePre("<b>");
query.setHighlightSimplePost("</b>");
QueryResponse resp = solrChunkServer.query(query);
SolrDocumentList docs = resp.getResults();
retCode = new ArrayList<Chunk>(docs.size());
for (SolrDocument doc : docs)
{
LOG.debug("got doc " + doc);
Chunk chunk = new Chunk(doc);
// retrieve highlighting
List<String> highlights =
resp.getHighlighting().get(chunk.getId()).get(Chunk.SOLR_KEY_TEXT);
if (highlights != null && highlights.size() > 0)
chunk.setHighlighted(highlights.get(0));
retCode.add(chunk);
}
--
http://www.linkedin.com/in/paultomblin
Re: Highlighting in SolrJ?
Posted by Jay Hill <ja...@gmail.com>.
Set up the query like this to highlight a field named "content":
SolrQuery query = new SolrQuery();
query.setQuery("foo");
query.setHighlight(true).setHighlightSnippets(1); //set other params as
needed
query.setParam("hl.fl", "content");
QueryResponse queryResponse =getSolrServer().query(query);
Then to get back the highlight results you need something like this:
Iterator<SolrDocument> iter = queryResponse.getResults();
while (iter.hasNext()) {
SolrDocument resultDoc = iter.next();
String content = (String) resultDoc.getFieldValue("content"));
String id = (String) resultDoc.getFieldValue("id"); //id is the
uniqueKey field
if (queryResponse.getHighlighting().get(id) != null) {
List<String> highightSnippets =
queryResponse.getHighlighting().get(id).get("content");
}
}
Hope that gets you what you need.
-Jay
http://www.lucidimagination.com
On Thu, Sep 10, 2009 at 3:19 PM, Paul Tomblin <pt...@xcski.com> wrote:
> Can somebody point me to some sample code for using highlighting in
> SolrJ? I understand the highlighted versions of the field comes in a
> separate NamedList? How does that work?
>
> --
> http://www.linkedin.com/in/paultomblin
>