FieldCache?

[William Bell <bi...@gmail.com>]

So the FieldCache was removed from Solr 5.

What is the implication of this? Should we move all facets to DocValues
when we have high cardinality (lots of values) ? Are we adding it back?

Other ideas to improve performance?

>From Mike M:

FieldCache is gone (moved to a dedicated UninvertingReader in the miscmodule).
This means when you intend to sort on a field, you should index that field
using doc values, which is much faster and less heap consuming than
FieldCache.

-- 
Bill Bell
billnbell@gmail.com
cell 720-256-8076

Re: FieldCache?

[Alessandro Benedetti <be...@gmail.com>]

For completeness this is the related issue :

https://issues.apache.org/jira/browse/SOLR-8096

Cheers

2015-10-06 11:21 GMT+01:00 Alessandro Benedetti <be...@gmail.com>
:

> We should make some precision here,
> When dealing with faceting , there are currently 2 main approaches :
>
> 1) *Enum Algorithm* - best for low cardinality value fields, it is based
> on retrieving the term enum for all the terms in the index, and then
> intersecting the related posting list with the query result set
>
> 2) *Un-Inverting Algorithms* - Best for high cardinality value fields, it
> is based on uninverting the index, checking the value for each query result
> document field, and counting the occurrences
>
> Within the 2nd approach :
>
> 2a) *Doc Values * - It is better for dynamic indexes, built at indexing
> time and stored on the disk ( setting the specific attribute for the field)
> OR calculated at runtime thanks to the UnInvertedReader that will uninvert
> to an in-memory structure that looks like DocValues
>
> 2b) *Uninverted field* - for index that changes less frequently . After
> the removal related :
> Only per segment field caches are available, which means you should be
> able to use it using the fcs algorithm.
>
> This should be the current situation,
> I will take a look into details and let you know if I understood something
> wrong.
>
> Cheers
>
>
>
> 2015-10-06 5:03 GMT+01:00 William Bell <bi...@gmail.com>:
>
>> So the FieldCache was removed from Solr 5.
>>
>> What is the implication of this? Should we move all facets to DocValues
>> when we have high cardinality (lots of values) ? Are we adding it back?
>>
>> Other ideas to improve performance?
>>
>> From Mike M:
>>
>> FieldCache is gone (moved to a dedicated UninvertingReader in the
>> miscmodule).
>> This means when you intend to sort on a field, you should index that field
>> using doc values, which is much faster and less heap consuming than
>> FieldCache.
>>
>> --
>> Bill Bell
>> billnbell@gmail.com
>> cell 720-256-8076
>>
>
>
>
> --
> --------------------------
>
> Benedetti Alessandro
> Visiting card - http://about.me/alessandro_benedetti
> Blog - http://alexbenedetti.blogspot.co.uk
>
> "Tyger, tyger burning bright
> In the forests of the night,
> What immortal hand or eye
> Could frame thy fearful symmetry?"
>
> William Blake - Songs of Experience -1794 England
>



-- 
--------------------------

Benedetti Alessandro
Visiting card - http://about.me/alessandro_benedetti
Blog - http://alexbenedetti.blogspot.co.uk

"Tyger, tyger burning bright
In the forests of the night,
What immortal hand or eye
Could frame thy fearful symmetry?"

William Blake - Songs of Experience -1794 England

Re: FieldCache?

[Alessandro Benedetti <be...@gmail.com>]

We should make some precision here,
When dealing with faceting , there are currently 2 main approaches :

1) *Enum Algorithm* - best for low cardinality value fields, it is based on
retrieving the term enum for all the terms in the index, and then
intersecting the related posting list with the query result set

2) *Un-Inverting Algorithms* - Best for high cardinality value fields, it
is based on uninverting the index, checking the value for each query result
document field, and counting the occurrences

Within the 2nd approach :

2a) *Doc Values * - It is better for dynamic indexes, built at indexing
time and stored on the disk ( setting the specific attribute for the field)
OR calculated at runtime thanks to the UnInvertedReader that will uninvert
to an in-memory structure that looks like DocValues

2b) *Uninverted field* - for index that changes less frequently . After the
removal related :
Only per segment field caches are available, which means you should be able
to use it using the fcs algorithm.

This should be the current situation,
I will take a look into details and let you know if I understood something
wrong.

Cheers



2015-10-06 5:03 GMT+01:00 William Bell <bi...@gmail.com>:

> So the FieldCache was removed from Solr 5.
>
> What is the implication of this? Should we move all facets to DocValues
> when we have high cardinality (lots of values) ? Are we adding it back?
>
> Other ideas to improve performance?
>
> From Mike M:
>
> FieldCache is gone (moved to a dedicated UninvertingReader in the
> miscmodule).
> This means when you intend to sort on a field, you should index that field
> using doc values, which is much faster and less heap consuming than
> FieldCache.
>
> --
> Bill Bell
> billnbell@gmail.com
> cell 720-256-8076
>



-- 
--------------------------

Benedetti Alessandro
Visiting card - http://about.me/alessandro_benedetti
Blog - http://alexbenedetti.blogspot.co.uk

"Tyger, tyger burning bright
In the forests of the night,
What immortal hand or eye
Could frame thy fearful symmetry?"

William Blake - Songs of Experience -1794 England

Re: FieldCache?

[Chris Hostetter <ho...@fucit.org>]

: What is the implication of this? Should we move all facets to DocValues
: when we have high cardinality (lots of values) ? Are we adding it back?

1) Using DocValues is almost certainly a good idea moving forward for 
situations where the FieldCache was used in the past.

: FieldCache is gone (moved to a dedicated UninvertingReader in the miscmodule).

2) Solr implicitly uses the UninvertingReader under the covers on your 
behalf automatically in cases where FieldCache is not available.



-Hoss
http://www.lucidworks.com/