You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-dev@lucene.apache.org by "Tomer Gabel (JIRA)" <ji...@apache.org> on 2008/02/25 12:06:51 UTC
[jira] Created: (SOLR-488) Solr does not generate highlights when
uniqueId field is not defined in the schema
Solr does not generate highlights when uniqueId field is not defined in the schema
----------------------------------------------------------------------------------
Key: SOLR-488
URL: https://issues.apache.org/jira/browse/SOLR-488
Project: Solr
Issue Type: Improvement
Components: highlighter
Affects Versions: 1.2
Environment: Windows Vista Business (x86, x64), latest Ubuntu server, Apache Tomcat 6.0.14
Reporter: Tomer Gabel
Attachments: linkfield.HighlightingUtils.patch
Solr does not generate highlights when there is no uniqueId field defined in the schema. I believe the reason for this is that it's very difficult to modify or extend the XmlWriter behavior, which is why highlights reside in their own "section" in the response XML and subsequently need to be "linked" to their respective documents via the uniqueId field.
Our schema does not define a uniqueId for various reasons but we still need highlights; the solution we came up with was to provide a user-definable "link field," which is the document field whose value resides in the {{<lst name="215">}} elements in the generated output. I will presently attach a patch which adds a "hl.link" query parameter, which takes a field name and uses that as the "link field." If the parameter is not specified the original behavior is used, so backwards compatibility is maintained.
As an aside, we've found this technique to be useful because our custom handlers add a lot of information to each document, and the way the response writer is implemented makes it nigh impossible to add information to any specific document within the response. I should probably open an issue which calls to reimplement this aspect of Solr.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-488) Solr does not generate highlights when
uniqueId field is not defined in the schema
Posted by "Tomer Gabel (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12572407#action_12572407 ]
Tomer Gabel commented on SOLR-488:
----------------------------------
Hi Mike,
We use a derivative of 1.2 here and have only just started looking at 1.3, however because of our development schedule I doubt we'll be able to safely use the trunk within the allotted time. Regardless, in 1.2 the highlights section is either missing or empty (can't remember which); if the order is guaranteed the same as the results this is probably good enough.
As for the query-time link field, it merely seemed like a path of least resistance to getting the functionality I need while maintaining backwards compatibility and with minimum intervention. I can't really think of a scenario where this will change (for the same schema) between requests.
Finally, the unique key is insufficient because we simply don't use one. Due to various performance and design goals we prefer to handle duplicates on our own with an external system before even going to Solr.
> Solr does not generate highlights when uniqueId field is not defined in the schema
> ----------------------------------------------------------------------------------
>
> Key: SOLR-488
> URL: https://issues.apache.org/jira/browse/SOLR-488
> Project: Solr
> Issue Type: Improvement
> Components: highlighter
> Affects Versions: 1.2
> Environment: Windows Vista Business (x86, x64), latest Ubuntu server, Apache Tomcat 6.0.14
> Reporter: Tomer Gabel
> Attachments: linkfield.HighlightingUtils.patch
>
>
> Solr does not generate highlights when there is no uniqueId field defined in the schema. I believe the reason for this is that it's very difficult to modify or extend the XmlWriter behavior, which is why highlights reside in their own "section" in the response XML and subsequently need to be "linked" to their respective documents via the uniqueId field.
> Our schema does not define a uniqueId for various reasons but we still need highlights; the solution we came up with was to provide a user-definable "link field," which is the document field whose value resides in the {{<lst name="215">}} elements in the generated output. I will presently attach a patch which adds a "hl.link" query parameter, which takes a field name and uses that as the "link field." If the parameter is not specified the original behavior is used, so backwards compatibility is maintained.
> As an aside, we've found this technique to be useful because our custom handlers add a lot of information to each document, and the way the response writer is implemented makes it nigh impossible to add information to any specific document within the response. I should probably open an issue which calls to reimplement this aspect of Solr.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-488) Solr does not generate highlights when
uniqueId field is not defined in the schema
Posted by "Tomer Gabel (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tomer Gabel updated SOLR-488:
-----------------------------
Attachment: linkfield.HighlightingUtils.patch
Patch to HighlightingUtils.java: adds a "link field" parameter which allows highlight generation even when uniqueId is not defined in the schema.
> Solr does not generate highlights when uniqueId field is not defined in the schema
> ----------------------------------------------------------------------------------
>
> Key: SOLR-488
> URL: https://issues.apache.org/jira/browse/SOLR-488
> Project: Solr
> Issue Type: Improvement
> Components: highlighter
> Affects Versions: 1.2
> Environment: Windows Vista Business (x86, x64), latest Ubuntu server, Apache Tomcat 6.0.14
> Reporter: Tomer Gabel
> Attachments: linkfield.HighlightingUtils.patch
>
>
> Solr does not generate highlights when there is no uniqueId field defined in the schema. I believe the reason for this is that it's very difficult to modify or extend the XmlWriter behavior, which is why highlights reside in their own "section" in the response XML and subsequently need to be "linked" to their respective documents via the uniqueId field.
> Our schema does not define a uniqueId for various reasons but we still need highlights; the solution we came up with was to provide a user-definable "link field," which is the document field whose value resides in the {{<lst name="215">}} elements in the generated output. I will presently attach a patch which adds a "hl.link" query parameter, which takes a field name and uses that as the "link field." If the parameter is not specified the original behavior is used, so backwards compatibility is maintained.
> As an aside, we've found this technique to be useful because our custom handlers add a lot of information to each document, and the way the response writer is implemented makes it nigh impossible to add information to any specific document within the response. I should probably open an issue which calls to reimplement this aspect of Solr.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-488) Solr does not generate highlights when
uniqueId field is not defined in the schema
Posted by "Mike Klaas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12573080#action_12573080 ]
Mike Klaas commented on SOLR-488:
---------------------------------
Perhaps I should have been more clear: it is very unlikely that there will be a bugfix release for 1.2, so the best that can be done is to look at the closed issues in JIRA for the highlighter to find the change that fixed the bug in the current version.
As for uniqueKey, it is about more than finding duplicates at document insertion time. It is also a unique identifier for each document used for federated search, highlighting, and other places (exactly what you are looking for). If you want to turn off Solr's duplicate detection, just add allowDups="true" to the <update/> you send to solr.
> Solr does not generate highlights when uniqueId field is not defined in the schema
> ----------------------------------------------------------------------------------
>
> Key: SOLR-488
> URL: https://issues.apache.org/jira/browse/SOLR-488
> Project: Solr
> Issue Type: Improvement
> Components: highlighter
> Affects Versions: 1.2
> Environment: Windows Vista Business (x86, x64), latest Ubuntu server, Apache Tomcat 6.0.14
> Reporter: Tomer Gabel
> Attachments: linkfield.HighlightingUtils.patch
>
>
> Solr does not generate highlights when there is no uniqueId field defined in the schema. I believe the reason for this is that it's very difficult to modify or extend the XmlWriter behavior, which is why highlights reside in their own "section" in the response XML and subsequently need to be "linked" to their respective documents via the uniqueId field.
> Our schema does not define a uniqueId for various reasons but we still need highlights; the solution we came up with was to provide a user-definable "link field," which is the document field whose value resides in the {{<lst name="215">}} elements in the generated output. I will presently attach a patch which adds a "hl.link" query parameter, which takes a field name and uses that as the "link field." If the parameter is not specified the original behavior is used, so backwards compatibility is maintained.
> As an aside, we've found this technique to be useful because our custom handlers add a lot of information to each document, and the way the response writer is implemented makes it nigh impossible to add information to any specific document within the response. I should probably open an issue which calls to reimplement this aspect of Solr.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (SOLR-488) Solr does not generate highlights when
uniqueId field is not defined in the schema
Posted by "Mike Klaas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mike Klaas resolved SOLR-488.
-----------------------------
Resolution: Fixed
Fix Version/s: 1.3
Assignee: Mike Klaas
The principal issue is fixed in 1.3.
Thanks Tomer.
> Solr does not generate highlights when uniqueId field is not defined in the schema
> ----------------------------------------------------------------------------------
>
> Key: SOLR-488
> URL: https://issues.apache.org/jira/browse/SOLR-488
> Project: Solr
> Issue Type: Improvement
> Components: highlighter
> Affects Versions: 1.2
> Environment: Windows Vista Business (x86, x64), latest Ubuntu server, Apache Tomcat 6.0.14
> Reporter: Tomer Gabel
> Assignee: Mike Klaas
> Fix For: 1.3
>
> Attachments: linkfield.HighlightingUtils.patch
>
>
> Solr does not generate highlights when there is no uniqueId field defined in the schema. I believe the reason for this is that it's very difficult to modify or extend the XmlWriter behavior, which is why highlights reside in their own "section" in the response XML and subsequently need to be "linked" to their respective documents via the uniqueId field.
> Our schema does not define a uniqueId for various reasons but we still need highlights; the solution we came up with was to provide a user-definable "link field," which is the document field whose value resides in the {{<lst name="215">}} elements in the generated output. I will presently attach a patch which adds a "hl.link" query parameter, which takes a field name and uses that as the "link field." If the parameter is not specified the original behavior is used, so backwards compatibility is maintained.
> As an aside, we've found this technique to be useful because our custom handlers add a lot of information to each document, and the way the response writer is implemented makes it nigh impossible to add information to any specific document within the response. I should probably open an issue which calls to reimplement this aspect of Solr.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-488) Solr does not generate highlights when
uniqueId field is not defined in the schema
Posted by "Mike Klaas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12572312#action_12572312 ]
Mike Klaas commented on SOLR-488:
---------------------------------
Thanks for the patch!
Tomer, I just tested this scenario under trunk (1.3dev), and Solr _does_ highlight documents if no uniqueKey is specified. It is true that the highlight data is not labeled, but it is guaranteed to appear in the same order as the documents in the <results> section, so they are identifiable.
Why do you propose a query-time "link" field? Do you need to be able to vary the field from request to request?
Also, it would be helpful to know why uniqueKey is not sufficient for this purpose.
> Solr does not generate highlights when uniqueId field is not defined in the schema
> ----------------------------------------------------------------------------------
>
> Key: SOLR-488
> URL: https://issues.apache.org/jira/browse/SOLR-488
> Project: Solr
> Issue Type: Improvement
> Components: highlighter
> Affects Versions: 1.2
> Environment: Windows Vista Business (x86, x64), latest Ubuntu server, Apache Tomcat 6.0.14
> Reporter: Tomer Gabel
> Attachments: linkfield.HighlightingUtils.patch
>
>
> Solr does not generate highlights when there is no uniqueId field defined in the schema. I believe the reason for this is that it's very difficult to modify or extend the XmlWriter behavior, which is why highlights reside in their own "section" in the response XML and subsequently need to be "linked" to their respective documents via the uniqueId field.
> Our schema does not define a uniqueId for various reasons but we still need highlights; the solution we came up with was to provide a user-definable "link field," which is the document field whose value resides in the {{<lst name="215">}} elements in the generated output. I will presently attach a patch which adds a "hl.link" query parameter, which takes a field name and uses that as the "link field." If the parameter is not specified the original behavior is used, so backwards compatibility is maintained.
> As an aside, we've found this technique to be useful because our custom handlers add a lot of information to each document, and the way the response writer is implemented makes it nigh impossible to add information to any specific document within the response. I should probably open an issue which calls to reimplement this aspect of Solr.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-488) Solr does not generate highlights when
uniqueId field is not defined in the schema
Posted by "Tomer Gabel (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12577479#action_12577479 ]
Tomer Gabel commented on SOLR-488:
----------------------------------
I'll try the solution you've mentioned; worst case we'll use the custom-tailored solution... feel free to close the bug. If I still see this an issue with 1.3 I'll reopen it or post another one.
> Solr does not generate highlights when uniqueId field is not defined in the schema
> ----------------------------------------------------------------------------------
>
> Key: SOLR-488
> URL: https://issues.apache.org/jira/browse/SOLR-488
> Project: Solr
> Issue Type: Improvement
> Components: highlighter
> Affects Versions: 1.2
> Environment: Windows Vista Business (x86, x64), latest Ubuntu server, Apache Tomcat 6.0.14
> Reporter: Tomer Gabel
> Attachments: linkfield.HighlightingUtils.patch
>
>
> Solr does not generate highlights when there is no uniqueId field defined in the schema. I believe the reason for this is that it's very difficult to modify or extend the XmlWriter behavior, which is why highlights reside in their own "section" in the response XML and subsequently need to be "linked" to their respective documents via the uniqueId field.
> Our schema does not define a uniqueId for various reasons but we still need highlights; the solution we came up with was to provide a user-definable "link field," which is the document field whose value resides in the {{<lst name="215">}} elements in the generated output. I will presently attach a patch which adds a "hl.link" query parameter, which takes a field name and uses that as the "link field." If the parameter is not specified the original behavior is used, so backwards compatibility is maintained.
> As an aside, we've found this technique to be useful because our custom handlers add a lot of information to each document, and the way the response writer is implemented makes it nigh impossible to add information to any specific document within the response. I should probably open an issue which calls to reimplement this aspect of Solr.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.