[jira] [Commented] (SOLR-18251) Add Solr schema support for DocValues skip lists

["Vadim Kirilchuk (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SOLR-18251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18086812#comment-18086812 ] 

Vadim Kirilchuk commented on SOLR-18251:
----------------------------------------

Created a PR for this, it's about 80% ready, so I want to discuss a few things and get a feedback.
 * Any suggestions for DocValuesFieldUtil package? Currently it's adjacent to Field classes and I don't like it much.
 * The DocValuesSkipListTest, I need some guidance on what we want to test besides checking FieldInfo docValuesSkipIndexType. I left some basic range querying for some basic fields, but if we want comprehensive test it should query for all the field types.
 ** I also wonder what to do with ICUCollationField as it sits in separate module, do we need a separate test there, or do we want to bring it into DocValuesSkipListTest somehow?
 * SchemaField.calcProps I introduced a guard against setting docValues=false but docValuesSkipList=true. However, I was not able to find any negative test cases for any of the guards, do we need that?
 * Anything else to test?

Looking forward to suggestions.

> Add Solr schema support for DocValues skip lists
> ------------------------------------------------
>
>                 Key: SOLR-18251
>                 URL: https://issues.apache.org/jira/browse/SOLR-18251
>             Project: Solr
>          Issue Type: Task
>          Components: Schema and Analysis
>    Affects Versions: 10.1
>            Reporter: Sanjay Dutt
>            Priority: Major
>              Labels: pull-request-available
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> Lucene introduced optional skip lists on top of DocValues in apache/lucene#13449. The feature is exposed through the {{DocValuesSkipper}} abstraction and is controlled by a Lucene {{FieldType}} flag.
> Solr currently exposes DocValues through schema configuration, but does not provide a schema-level option to enable this DocValues skip list support.
> This issue proposes adding an opt-in Solr schema option, such as:
> {code:java}
> docValuesSkipList="true"{code}
> Example:
> {code:java}
> <field name="price"
>        type="plong"
>        indexed="false"
>        docValues="true"
>        docValuesSkipList="true" />{code}
> The Lucene PR notes that this can be efficient when the index is sorted and the field belongs to the index sorting. 
> The actual benefit depends on data layout and query pattern, so this should be exposed as an opt-in feature.
> Only compatible DocValues field types should allow this option. Based on the Lucene PR, this applies to DocValues types such as {{{}NUMERIC{}}}, {{{}SORTED_NUMERIC{}}}, {{{}SORTED{}}}, and {{{}SORTED_SET{}}}. 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org