[jira] [Commented] (SOLR-10286) Declare a field as "large", don't keep value in the document cache

["David Smiley (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SOLR-10286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15926119#comment-15926119 ] 

David Smiley commented on SOLR-10286:
-------------------------------------

One small change I should do is to have the SolrIndexSearcher.doc code only do it's large field handling if either there is a DocumentCache, or if lazy field loading is enabled.  If neither are true, there's no point to the special LargeLazyField.

One idea I rejected early (in parent issue SOLR-10117) I should mention here is that it's not realistic to use the LargeLazyField dynamically based on the actual value size.  If the value is huge, then it's effectively too late -- you've already born the cost of fetching it (to observe its size) and putting it on the heap.  So we have to say we're going to do this always for certain fields.  SOLR-10255 included an idea of putting a special marker stored value to give this code a heads-up that certain values on a particular document are large, thus allowing the decision to be made dynamically.  That could be added in the future still.

> Declare a field as "large", don't keep value in the document cache
> ------------------------------------------------------------------
>
>                 Key: SOLR-10286
>                 URL: https://issues.apache.org/jira/browse/SOLR-10286
>             Project: Solr
>          Issue Type: Improvement
>      Security Level: Public(Default Security Level. Issues are Public) 
>            Reporter: David Smiley
>            Assignee: David Smiley
>         Attachments: SOLR-10286_large_fields.patch
>
>
> (part of umbrella issue SOLR-10117)
> This adds a field to be declared as "large" in the schema.  In the {{SolrIndexSearcher.doc(...)}} handling, these fields are lazily fetched from Lucene.  Unlike {{LazyDocument.LazyField}}, it's not cached after first-use unless the value is "small" < 512KB by default.  "large" can only be used when its stored="true" and multiValued="false" and the field is otherwise compatible (basically not a numeric field) -- you'll get a helpful exception if it's unsupported. BinaryField is not yet supported at this time; it could be in the future.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org