You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-dev@lucene.apache.org by "Grant Ingersoll (JIRA)" <ji...@apache.org> on 2009/08/18 14:50:15 UTC
[jira] Created: (SOLR-1367) Provide Callback mechanism for
SolrPluginUtils.docListToSolrDocumentList
Provide Callback mechanism for SolrPluginUtils.docListToSolrDocumentList
------------------------------------------------------------------------
Key: SOLR-1367
URL: https://issues.apache.org/jira/browse/SOLR-1367
Project: Solr
Issue Type: Improvement
Reporter: Grant Ingersoll
Assignee: Grant Ingersoll
Priority: Minor
Fix For: 1.4
Sometimes SearchComponents need to modify the DocList. Often times this requires two loops over the (Solr)DocList: once to convert the DocList to SolrDocList and then once to iterate the SolrDocList for the Search Component. This can be seen in SOLR-773, for example.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1367) Provide Callback mechanism for
SolrPluginUtils.docListToSolrDocumentList
Posted by "Yonik Seeley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12744748#action_12744748 ]
Yonik Seeley commented on SOLR-1367:
------------------------------------
I'm puzzled wondering why this is an improvement.
New:
{code}
SolrDocumentList sdocs = docListToSolrDocumentList(dlist,...,
new SolrDocumentModifier() {
void process(SolrDocument doc) {
doc.add("foo",1);
}
},
...);
{code}
Old:
{code}
SolrDocumentList sdocs = docListToSolrDocumentList(dlist,...,...);
for (SolrDocument doc : sdocs) {
d.add("foo",1);
}
{code}
> Provide Callback mechanism for SolrPluginUtils.docListToSolrDocumentList
> ------------------------------------------------------------------------
>
> Key: SOLR-1367
> URL: https://issues.apache.org/jira/browse/SOLR-1367
> Project: Solr
> Issue Type: Improvement
> Reporter: Grant Ingersoll
> Assignee: Grant Ingersoll
> Priority: Minor
> Fix For: 1.4
>
> Attachments: SOLR-1367.patch
>
>
> Sometimes SearchComponents need to modify the DocList. Often times this requires two loops over the (Solr)DocList: once to convert the DocList to SolrDocList and then once to iterate the SolrDocList for the Search Component. This can be seen in SOLR-773, for example.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1367) Provide Callback mechanism for
SolrPluginUtils.docListToSolrDocumentList
Posted by "Grant Ingersoll (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12745024#action_12745024 ]
Grant Ingersoll commented on SOLR-1367:
---------------------------------------
Just because something isn't used internally by Solr, doesn't mean it isn't useful to Solr users. Why have Solr users write the same thing over and over again? It is perfectly reasonable for Solr to provide end user APIs.
> Provide Callback mechanism for SolrPluginUtils.docListToSolrDocumentList
> ------------------------------------------------------------------------
>
> Key: SOLR-1367
> URL: https://issues.apache.org/jira/browse/SOLR-1367
> Project: Solr
> Issue Type: Improvement
> Reporter: Grant Ingersoll
> Assignee: Grant Ingersoll
> Priority: Minor
> Fix For: 1.4
>
> Attachments: SOLR-1367.patch
>
>
> Sometimes SearchComponents need to modify the DocList. Often times this requires two loops over the (Solr)DocList: once to convert the DocList to SolrDocList and then once to iterate the SolrDocList for the Search Component. This can be seen in SOLR-773, for example.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1367) Provide Callback mechanism for
SolrPluginUtils.docListToSolrDocumentList
Posted by "Erik Hatcher (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12745029#action_12745029 ]
Erik Hatcher commented on SOLR-1367:
------------------------------------
Just because it is useful in one situation does not mean it should be something the Solr committers must maintain either. I'm not convinced that adding fields to a document (in this way) is the right way to add metadata to the response, personally. I'm only one opinion out there though, but I'm -0.5 on this being in Solr 1.4.
> Provide Callback mechanism for SolrPluginUtils.docListToSolrDocumentList
> ------------------------------------------------------------------------
>
> Key: SOLR-1367
> URL: https://issues.apache.org/jira/browse/SOLR-1367
> Project: Solr
> Issue Type: Improvement
> Reporter: Grant Ingersoll
> Assignee: Grant Ingersoll
> Priority: Minor
> Fix For: 1.4
>
> Attachments: SOLR-1367.patch
>
>
> Sometimes SearchComponents need to modify the DocList. Often times this requires two loops over the (Solr)DocList: once to convert the DocList to SolrDocList and then once to iterate the SolrDocList for the Search Component. This can be seen in SOLR-773, for example.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (SOLR-1367) Provide Callback mechanism for
SolrPluginUtils.docListToSolrDocumentList
Posted by "Grant Ingersoll (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Grant Ingersoll resolved SOLR-1367.
-----------------------------------
Resolution: Won't Fix
Reverted
> Provide Callback mechanism for SolrPluginUtils.docListToSolrDocumentList
> ------------------------------------------------------------------------
>
> Key: SOLR-1367
> URL: https://issues.apache.org/jira/browse/SOLR-1367
> Project: Solr
> Issue Type: Improvement
> Reporter: Grant Ingersoll
> Assignee: Grant Ingersoll
> Priority: Minor
> Fix For: 1.4
>
> Attachments: SOLR-1367.patch
>
>
> Sometimes SearchComponents need to modify the DocList. Often times this requires two loops over the (Solr)DocList: once to convert the DocList to SolrDocList and then once to iterate the SolrDocList for the Search Component. This can be seen in SOLR-773, for example.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (SOLR-1367) Provide Callback mechanism for
SolrPluginUtils.docListToSolrDocumentList
Posted by "Grant Ingersoll (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Grant Ingersoll resolved SOLR-1367.
-----------------------------------
Resolution: Fixed
Committed revision 805528.
> Provide Callback mechanism for SolrPluginUtils.docListToSolrDocumentList
> ------------------------------------------------------------------------
>
> Key: SOLR-1367
> URL: https://issues.apache.org/jira/browse/SOLR-1367
> Project: Solr
> Issue Type: Improvement
> Reporter: Grant Ingersoll
> Assignee: Grant Ingersoll
> Priority: Minor
> Fix For: 1.4
>
> Attachments: SOLR-1367.patch
>
>
> Sometimes SearchComponents need to modify the DocList. Often times this requires two loops over the (Solr)DocList: once to convert the DocList to SolrDocList and then once to iterate the SolrDocList for the Search Component. This can be seen in SOLR-773, for example.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1367) Provide Callback mechanism for
SolrPluginUtils.docListToSolrDocumentList
Posted by "Noble Paul (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12745032#action_12745032 ]
Noble Paul commented on SOLR-1367:
----------------------------------
IMHO the performance improvement is almost none with this enhancement. And it is cleaner the old way.
> Provide Callback mechanism for SolrPluginUtils.docListToSolrDocumentList
> ------------------------------------------------------------------------
>
> Key: SOLR-1367
> URL: https://issues.apache.org/jira/browse/SOLR-1367
> Project: Solr
> Issue Type: Improvement
> Reporter: Grant Ingersoll
> Assignee: Grant Ingersoll
> Priority: Minor
> Fix For: 1.4
>
> Attachments: SOLR-1367.patch
>
>
> Sometimes SearchComponents need to modify the DocList. Often times this requires two loops over the (Solr)DocList: once to convert the DocList to SolrDocList and then once to iterate the SolrDocList for the Search Component. This can be seen in SOLR-773, for example.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1367) Provide Callback mechanism for
SolrPluginUtils.docListToSolrDocumentList
Posted by "Yonik Seeley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12744788#action_12744788 ]
Yonik Seeley commented on SOLR-1367:
------------------------------------
It seems in this case that iteration is more straight-forward and easier to understand, and offers the same performance.
> Provide Callback mechanism for SolrPluginUtils.docListToSolrDocumentList
> ------------------------------------------------------------------------
>
> Key: SOLR-1367
> URL: https://issues.apache.org/jira/browse/SOLR-1367
> Project: Solr
> Issue Type: Improvement
> Reporter: Grant Ingersoll
> Assignee: Grant Ingersoll
> Priority: Minor
> Fix For: 1.4
>
> Attachments: SOLR-1367.patch
>
>
> Sometimes SearchComponents need to modify the DocList. Often times this requires two loops over the (Solr)DocList: once to convert the DocList to SolrDocList and then once to iterate the SolrDocList for the Search Component. This can be seen in SOLR-773, for example.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1367) Provide Callback mechanism for
SolrPluginUtils.docListToSolrDocumentList
Posted by "Erik Hatcher (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12745019#action_12745019 ]
Erik Hatcher commented on SOLR-1367:
------------------------------------
My opinion on this... this code should not be included in Solr yet. It can easily be added as custom utility or just a couple of lines of code into custom components. Until the components that need such utilities are added to Solr and until we agree on a cleaner way to add metadata to a document, I vote we revert this and not include it in Solr 1.4.
> Provide Callback mechanism for SolrPluginUtils.docListToSolrDocumentList
> ------------------------------------------------------------------------
>
> Key: SOLR-1367
> URL: https://issues.apache.org/jira/browse/SOLR-1367
> Project: Solr
> Issue Type: Improvement
> Reporter: Grant Ingersoll
> Assignee: Grant Ingersoll
> Priority: Minor
> Fix For: 1.4
>
> Attachments: SOLR-1367.patch
>
>
> Sometimes SearchComponents need to modify the DocList. Often times this requires two loops over the (Solr)DocList: once to convert the DocList to SolrDocList and then once to iterate the SolrDocList for the Search Component. This can be seen in SOLR-773, for example.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1367) Provide Callback mechanism for
SolrPluginUtils.docListToSolrDocumentList
Posted by "Grant Ingersoll (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Grant Ingersoll updated SOLR-1367:
----------------------------------
Attachment: SOLR-1367.patch
Simple fix for this. Marked the interface as changeable pending some other fixes later.
> Provide Callback mechanism for SolrPluginUtils.docListToSolrDocumentList
> ------------------------------------------------------------------------
>
> Key: SOLR-1367
> URL: https://issues.apache.org/jira/browse/SOLR-1367
> Project: Solr
> Issue Type: Improvement
> Reporter: Grant Ingersoll
> Assignee: Grant Ingersoll
> Priority: Minor
> Fix For: 1.4
>
> Attachments: SOLR-1367.patch
>
>
> Sometimes SearchComponents need to modify the DocList. Often times this requires two loops over the (Solr)DocList: once to convert the DocList to SolrDocList and then once to iterate the SolrDocList for the Search Component. This can be seen in SOLR-773, for example.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1367) Provide Callback mechanism for
SolrPluginUtils.docListToSolrDocumentList
Posted by "Yonik Seeley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12744826#action_12744826 ]
Yonik Seeley commented on SOLR-1367:
------------------------------------
bq. What's hard to understand about a callback mechanism?
Callbacks are useful when you need to execute your own snippet of code in a different context - that's not the case here.
bq. Also, why would two loops over the same list be the same performance as one?
Because the iteration part is irrelevant compared to what else is done during the iteration. Iteration can be as simple as a pointer dereference. The callback approach has null pointer check whether to invoke the modifier or not... and that gets done even if no modifier will be used (but that will also be in the noise).
bq. Now multiply the need to do this for every query and I think it would add up.
It won't - it won't even be measurable.
bq. The whole use case is very much a special case. Most people in most situations don't need to do this.
Which would seem to argue against adding code to try and do it.
That said, docListToSolrDocList isn't even used in Solr anywhere, so I don't care too much - it can be deprecated/removed later. Solr itself shouldn't use this method since it's not streamable. Allowing custom modifications while preserving streamability *does* call for callbacks... but not in 1.4
> Provide Callback mechanism for SolrPluginUtils.docListToSolrDocumentList
> ------------------------------------------------------------------------
>
> Key: SOLR-1367
> URL: https://issues.apache.org/jira/browse/SOLR-1367
> Project: Solr
> Issue Type: Improvement
> Reporter: Grant Ingersoll
> Assignee: Grant Ingersoll
> Priority: Minor
> Fix For: 1.4
>
> Attachments: SOLR-1367.patch
>
>
> Sometimes SearchComponents need to modify the DocList. Often times this requires two loops over the (Solr)DocList: once to convert the DocList to SolrDocList and then once to iterate the SolrDocList for the Search Component. This can be seen in SOLR-773, for example.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1367) Provide Callback mechanism for
SolrPluginUtils.docListToSolrDocumentList
Posted by "Grant Ingersoll (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12744818#action_12744818 ]
Grant Ingersoll commented on SOLR-1367:
---------------------------------------
What's hard to understand about a callback mechanism? It's akin to FileFilter for File.listFiles or similar things.
Also, why would two loops over the same list be the same performance as one? I suppose in the rows=10 or so case, it is in the noise, but, but if someone were getting deeper results than say 10, the Modifier approach would be faster. Now multiply the need to do this for every query and I think it would add up.
The whole use case is very much a special case. Most people in most situations don't need to do this. But when you do need to do it, you do it all the time and you want it to be fast. That being said, people can implement it themselves. I just figured since the docListToSolrDocList method was already there, it would be a time saver.
> Provide Callback mechanism for SolrPluginUtils.docListToSolrDocumentList
> ------------------------------------------------------------------------
>
> Key: SOLR-1367
> URL: https://issues.apache.org/jira/browse/SOLR-1367
> Project: Solr
> Issue Type: Improvement
> Reporter: Grant Ingersoll
> Assignee: Grant Ingersoll
> Priority: Minor
> Fix For: 1.4
>
> Attachments: SOLR-1367.patch
>
>
> Sometimes SearchComponents need to modify the DocList. Often times this requires two loops over the (Solr)DocList: once to convert the DocList to SolrDocList and then once to iterate the SolrDocList for the Search Component. This can be seen in SOLR-773, for example.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1367) Provide Callback mechanism for
SolrPluginUtils.docListToSolrDocumentList
Posted by "Grant Ingersoll (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12745013#action_12745013 ]
Grant Ingersoll commented on SOLR-1367:
---------------------------------------
bq. That said, docListToSolrDocList isn't even used in Solr anywhere, so I don't care too much - it can be deprecated/removed later. Solr itself shouldn't use this method since it's not streamable. Allowing custom modifications while preserving streamability does call for callbacks... but not in 1.4
Right, it is likely used by a very small subset of people who: write their own SearchComp and modify the doclist. LocalSolr is one such case. If you look at PJ's patch on SOLR-773, you will see it in use. I think Ryan put it in, so he may have use case for it too. It is _definitely_ not needed by Solr itself. The modifier thing is also likely a temp fix until pseudo-fields and pluggable doc stores are in place.
> Provide Callback mechanism for SolrPluginUtils.docListToSolrDocumentList
> ------------------------------------------------------------------------
>
> Key: SOLR-1367
> URL: https://issues.apache.org/jira/browse/SOLR-1367
> Project: Solr
> Issue Type: Improvement
> Reporter: Grant Ingersoll
> Assignee: Grant Ingersoll
> Priority: Minor
> Fix For: 1.4
>
> Attachments: SOLR-1367.patch
>
>
> Sometimes SearchComponents need to modify the DocList. Often times this requires two loops over the (Solr)DocList: once to convert the DocList to SolrDocList and then once to iterate the SolrDocList for the Search Component. This can be seen in SOLR-773, for example.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1367) Provide Callback mechanism for
SolrPluginUtils.docListToSolrDocumentList
Posted by "Grant Ingersoll (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12745091#action_12745091 ]
Grant Ingersoll commented on SOLR-1367:
---------------------------------------
OK, I will revert.
> Provide Callback mechanism for SolrPluginUtils.docListToSolrDocumentList
> ------------------------------------------------------------------------
>
> Key: SOLR-1367
> URL: https://issues.apache.org/jira/browse/SOLR-1367
> Project: Solr
> Issue Type: Improvement
> Reporter: Grant Ingersoll
> Assignee: Grant Ingersoll
> Priority: Minor
> Fix For: 1.4
>
> Attachments: SOLR-1367.patch
>
>
> Sometimes SearchComponents need to modify the DocList. Often times this requires two loops over the (Solr)DocList: once to convert the DocList to SolrDocList and then once to iterate the SolrDocList for the Search Component. This can be seen in SOLR-773, for example.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1367) Provide Callback mechanism for
SolrPluginUtils.docListToSolrDocumentList
Posted by "Grant Ingersoll (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12744784#action_12744784 ]
Grant Ingersoll commented on SOLR-1367:
---------------------------------------
B/c docListToSolrDocList also requires looping over the DocList. So, in the old case, you've iterated the DocList twice. In the new way, you only iterated it once.
> Provide Callback mechanism for SolrPluginUtils.docListToSolrDocumentList
> ------------------------------------------------------------------------
>
> Key: SOLR-1367
> URL: https://issues.apache.org/jira/browse/SOLR-1367
> Project: Solr
> Issue Type: Improvement
> Reporter: Grant Ingersoll
> Assignee: Grant Ingersoll
> Priority: Minor
> Fix For: 1.4
>
> Attachments: SOLR-1367.patch
>
>
> Sometimes SearchComponents need to modify the DocList. Often times this requires two loops over the (Solr)DocList: once to convert the DocList to SolrDocList and then once to iterate the SolrDocList for the Search Component. This can be seen in SOLR-773, for example.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Reopened: (SOLR-1367) Provide Callback mechanism for
SolrPluginUtils.docListToSolrDocumentList
Posted by "Erik Hatcher (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Erik Hatcher reopened SOLR-1367:
--------------------------------
Reopened to consider reverting.
> Provide Callback mechanism for SolrPluginUtils.docListToSolrDocumentList
> ------------------------------------------------------------------------
>
> Key: SOLR-1367
> URL: https://issues.apache.org/jira/browse/SOLR-1367
> Project: Solr
> Issue Type: Improvement
> Reporter: Grant Ingersoll
> Assignee: Grant Ingersoll
> Priority: Minor
> Fix For: 1.4
>
> Attachments: SOLR-1367.patch
>
>
> Sometimes SearchComponents need to modify the DocList. Often times this requires two loops over the (Solr)DocList: once to convert the DocList to SolrDocList and then once to iterate the SolrDocList for the Search Component. This can be seen in SOLR-773, for example.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.