DocValues usage

[Poorna Murali <po...@gmail.com>]

Hi,

We are planning to introduce docValues and reduce the usage of fieldcache
in our application.

I have a few doubts on these. Please help me to clarify them.

1) Will there be a performance impact if we use docValues as it does I/O
read? Especially at the time of index generation?

2) Are there any list of recommended fields/ category of fields that are
suitable for docValues?

3) There is a field named "id" which is used as unique key in solr.
Although it's not used for faceting or sorting, it still comes up in
fieldcache and occupies lot of memory. Please help me understand how this
id field came in field cache.

4) I understand that by looking to field cache in solr admin page, we will
get the  current entries in it. But, is it possible to get the whole list
of attributes that can go into field cache?

Thanks in advance,
Poorna

Re: DocValues usage

[Mikhail Khludnev <mk...@apache.org>]

Hi, Poorna.
Pls check inline below.

On Tue, May 31, 2022 at 4:21 PM Poorna Murali <po...@gmail.com>
wrote:

> Hi,
>
> We are planning to introduce docValues and reduce the usage of fieldcache
> in our application.
>
> I have a few doubts on these. Please help me to clarify them.
>
> 1) Will there be a performance impact if we use docValues as it does I/O
> read? Especially at the time of index generation?
>
Sure, it needs to write more files during indexing. Then, query
performance depends on particular IO. It definitely make sense to limit jvm
heap leaving some RAM free for memory mapping index files.


>
> 2) Are there any list of recommended fields/ category of fields that are
> suitable for docValues?
>
Short enums and numbers. Texts (in terms of analysis) are not supported
anyway.

>
> 3) There is a field named "id" which is used as unique key in solr.
> Although it's not used for faceting or sorting, it still comes up in
> fieldcache and occupies lot of memory. Please help me understand how this
> id field came in field cache.
>
Some internal routines can trigger its loading. I'm afraid only debugger
can give a definite answer.  Maybe declaring it as uninvertible=false  can
find a code path, but I'm not sure.

>
> 4) I understand that by looking to field cache in solr admin page, we will
> get the  current entries in it. But, is it possible to get the whole list
> of attributes that can go into field cache?
>
I don't think it's possible, since field cache is lazy/runtime. ie one can
trigger new field caching by requesting a facet over it.

>
> Thanks in advance,
> Poorna
>
Welcome!


-- 
Sincerely yours
Mikhail Khludnev