You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Erick Erickson (JIRA)" <ji...@apache.org> on 2016/01/11 20:04:39 UTC
[jira] [Commented] (SOLR-8220) Read field from docValues for non
stored fields
[ https://issues.apache.org/jira/browse/SOLR-8220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15092491#comment-15092491 ]
Erick Erickson commented on SOLR-8220:
--------------------------------------
Hmmm, one implication that I just realized (yeah, I'm slow sometimes). In addition to the output from a DV field being reordered, identical values are collapsed since it's a SortedSet under the covers, right? So if I have a MultiValued DocValues field and put in "memory", "memory", "memory", then all I get back from the DV field is one copy of "memory".
Since the guarantee that multiValued fields return data in the same order inserted when returning the Stored value is not true of returning DV values, this is perhaps no biggie. IMO it does deserve a comment when we do the docs for this though.
> Read field from docValues for non stored fields
> -----------------------------------------------
>
> Key: SOLR-8220
> URL: https://issues.apache.org/jira/browse/SOLR-8220
> Project: Solr
> Issue Type: Improvement
> Reporter: Keith Laban
> Assignee: Shalin Shekhar Mangar
> Attachments: SOLR-8220-5x.patch, SOLR-8220-branch_5x.patch, SOLR-8220-ishan.patch, SOLR-8220-ishan.patch, SOLR-8220-ishan.patch, SOLR-8220-ishan.patch, SOLR-8220.patch, SOLR-8220.patch, SOLR-8220.patch, SOLR-8220.patch, SOLR-8220.patch, SOLR-8220.patch, SOLR-8220.patch, SOLR-8220.patch, SOLR-8220.patch, SOLR-8220.patch, SOLR-8220.patch, SOLR-8220.patch, SOLR-8220.patch, SOLR-8220.patch, SOLR-8220.patch, SOLR-8220.patch, SOLR-8220.patch, SOLR-8220.patch, SOLR-8220.patch, SOLR-8220.patch, SOLR-8220.patch, SOLR-8220.patch, SOLR-8220.patch, SOLR-8220.patch, SOLR-8220.patch, SOLR-8220.patch, SOLR-8220.patch, SOLR-8220.patch
>
>
> Many times a value will be both stored="true" and docValues="true" which requires redundant data to be stored on disk. Since reading from docValues is both efficient and a common practice (facets, analytics, streaming, etc), reading values from docValues when a stored version of the field does not exist would be a valuable disk usage optimization.
> The only caveat with this that I can see would be for multiValued fields as they would always be returned sorted in the docValues approach. I believe this is a fair compromise.
> I've done a rough implementation for this as a field transform, but I think it should live closer to where stored fields are loaded in the SolrIndexSearcher.
> Two open questions/observations:
> 1) There doesn't seem to be a standard way to read values for docValues, facets, analytics, streaming, etc, all seem to be doing their own ways, perhaps some of this logic should be centralized.
> 2) What will the API behavior be? (Below is my proposed implementation)
> Parameters for fl:
> - fl="docValueField"
> -- return field from docValue if the field is not stored and in docValues, if the field is stored return it from stored fields
> - fl="*"
> -- return only stored fields
> - fl="+"
> -- return stored fields and docValue fields
> 2a - would be easiest implementation and might be sufficient for a first pass. 2b - is current behavior
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org