You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Jahangir Anwari (JIRA)" <ji...@apache.org> on 2011/06/27 18:01:48 UTC
[jira] [Issue Comment Edited] (LUCENE-3243) FastVectorHighlighter -
add position offset to FieldPhraseList.WeightedPhraseInfo.Toffs
[ https://issues.apache.org/jira/browse/LUCENE-3243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13055610#comment-13055610 ]
Jahangir Anwari edited comment on LUCENE-3243 at 6/27/11 4:01 PM:
------------------------------------------------------------------
Hi Koji,
Sorry for not elaborating more on our requirements and our implementation. Basically for every search result we needed the position(word offset) information of the search hits in the document. On the search result page, this position offsets information was embedded in the search result links. When the user clicked on a search link, at the target page using javascript and the position offset information we would highlight the search terms.
To return the position offset information along with the highlighted snippet we created a CustomSolrHighlihter(attached). Depending on the type of query the custom highlighter returns the position offsets information.
# Non-phrase query: Using FieldTermStack we return the term position offset for the terms in the query.
# Phrase query: Using the WeightedFragInfo.fragInfos we return the term position offset for the terms in the query.
But currently the Toffs(Term offsets) class only stores the start and end offset and so we updated it so that it would store the position information as well.
Answers to your questions:
* *What is the position offset? Isn't it just a position?*
Yes, it is just the position.
* *Why is the position offset String?*
Since for phrase queries(e.g. "divine knowledge") the position-gap between terms == 1, WeightedPhraseInfo would only store the startOffset(i.e 12) of the first term of the phrase terms and the endOffset(i.e. 29) of the phrase terms.
{code}
[startOffset, endOffset]
"divine knowledge": [(12,29)]
{code}
But as we needed position information(i.e. 5,6) of all the terms it required storing the position of the terms of a phrase query as a String.
{code}
[startOffset, endOffset, positions]
"divine knowledge": [(12,29, [5,6])]
{code}
* *Why do you need setPositionOffset()?*
setPositionOffset() is used to store the positions of consecutive terms of a phrase query. For every terms of the phrase query it just appends the argument position to the current position(i.e. [5,6]).
Example output:
{code}
<lst name="/book/title/pg15">
<arr name="para">
<str>un of <strong class="highlight">divine knowledge</strong> and understanding, and become the recipients of a grace that is infinite and </str>
</arr>
<str name="positionOffsets">80,81,118,119</str>
</lst>
{code}
P.S. In order to able to override doHighlightingByFastVectorHighlighter() method in CustomSolrHighlighter we had to change the access modifier for alternateField() and getSolrFragmentsBuilder() to protected.
was (Author: janwari):
Hi Koji,
Sorry for not elaborating more on our requirements and our implementation. Basically for every search result we needed the position(word offset) information of the search hits in the document. On the search result page, this position offsets information was embedded in the search result links. When the user clicked on a search link, at the target page using javascript and the position offset information we would highlight the search terms.
To return the position offset information along with the highlighted snippet we created a CustomSolrHighlihter(attached). Depending on the type of query the custom highlighter returns the position offsets information.
# Non-phrase query: Using FieldTermStack we return the term position offset for the terms in the query.
# Phrase query: Using the WeightedFragInfo.fragInfos we return the term position offset for the terms in the query.
But currently the Toffs(Term offsets) class only stores the start and end offset and so we updated it so that it would store the position information as well.
Answers to your questions:
* *What is the position offset? Isn't it just a position?*
Yes, it is just the position.
* *Why is the position offset String?*
Since for phrase queries(e.g. "divine knowledge") the position-gap between terms == 1, WeightedPhraseInfo would only store the startOffset(i.e 12) of the first term of the phrase terms and the endOffset(i.e. 29) of the phrase terms.
{code}
[startOffset, endOffset]
"divine knowledge": [(12,29)]
{code}
But as we needed position information(i.e. 5,6) of all the terms it required storing the position of the terms of a phrase query as a String.
{code}
[startOffset, endOffset, positions]
"divine knowledge": [(12,29, [5,6])]
{code}
* *Why do you need setPositionOffset()?*
setPositionOffset() is used to store the positions of consecutive terms of a phrase query. For every terms of the phrase query it just appends the argument position to the current position(i.e. [5,6]).
P.S. In order to able to override doHighlightingByFastVectorHighlighter() method in CustomSolrHighlighter we had to change the access modifier for alternateField() and getSolrFragmentsBuilder() to protected.
> FastVectorHighlighter - add position offset to FieldPhraseList.WeightedPhraseInfo.Toffs
> ---------------------------------------------------------------------------------------
>
> Key: LUCENE-3243
> URL: https://issues.apache.org/jira/browse/LUCENE-3243
> Project: Lucene - Java
> Issue Type: Improvement
> Components: modules/highlighter
> Affects Versions: 3.2
> Environment: Lucene 3.2
> Reporter: Jahangir Anwari
> Priority: Minor
> Labels: feature, lucene
> Attachments: CustomSolrHighlighter.java, LUCENE-3243.patch.diff
>
>
> Needed to return position offsets along with highlighted snippets when using FVH for highlighting.
> Using the ([LUCENE-3141|https://issues.apache.org/jira/browse/LUCENE-3141]) patch I was able to get the fragInfo for a particular Phrase search. Currently the Toffs(Term offsets) class only stores the start and end offset.
> To get the position offset, I added the position offset information in Toffs and FieldPhraseList class.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org