You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@solr.apache.org by Dominique Bejean <do...@eolya.fr> on 2022/12/06 14:13:16 UTC
FieldCache and _version_field
Hi,
One of my customers has a huge collection (1.5 billion docs across 14
shards).
All fields are correctly configured in order to enable docValues
except _version_. They are still using the old configuration with
indexed=true instead of docValues and hence _version_ populate FiledCache
in JVM heap (several Gb).
They need to reindex for various reasons including this one but this can't
be done before several weeks due to the complexity to handle full
reindexing and continuous indexing at the same time.
Why is FieldCache populated with _version_ field as it isn't explicitly
used in sort, facet, grouping and function ?
I guess Solr needs this internally.
Is there a workaround in order to avoid FiledCache to be populated by
_version_ field waiting to reindex ?
Regards
Dominique
Re: FieldCache and _version_field
Posted by Dominique Bejean <do...@eolya.fr>.
Hi Mikhail,
Thank you for the response.
More details. Solr is version 7.7.0 and collection replicas are TLOG.
I will check, but I don't think atomic updates are required.
Regards
Dominique
Le mar. 6 déc. 2022 à 15:43, Mikhail Khludnev <mk...@apache.org> a écrit :
> Hello, Dominique.
> I suppose it's used for updates and specifically in
> AtomicUpdateProcessorFactory
> and UpdateLog. Presumably, if that cluster can live without atomic updates,
> you can try to drop them out of update chain.
>
> On Tue, Dec 6, 2022 at 5:14 PM Dominique Bejean <dominique.bejean@eolya.fr
> >
> wrote:
>
> > Hi,
> >
> > One of my customers has a huge collection (1.5 billion docs across 14
> > shards).
> > All fields are correctly configured in order to enable docValues
> > except _version_. They are still using the old configuration with
> > indexed=true instead of docValues and hence _version_ populate FiledCache
> > in JVM heap (several Gb).
> >
> > They need to reindex for various reasons including this one but this
> can't
> > be done before several weeks due to the complexity to handle full
> > reindexing and continuous indexing at the same time.
> >
> > Why is FieldCache populated with _version_ field as it isn't explicitly
> > used in sort, facet, grouping and function ?
> > I guess Solr needs this internally.
> >
> > Is there a workaround in order to avoid FiledCache to be populated by
> > _version_ field waiting to reindex ?
> >
> > Regards
> >
> > Dominique
> >
>
>
> --
> Sincerely yours
> Mikhail Khludnev
>
Re: FieldCache and _version_field
Posted by Mikhail Khludnev <mk...@apache.org>.
Hello, Dominique.
I suppose it's used for updates and specifically in
AtomicUpdateProcessorFactory
and UpdateLog. Presumably, if that cluster can live without atomic updates,
you can try to drop them out of update chain.
On Tue, Dec 6, 2022 at 5:14 PM Dominique Bejean <do...@eolya.fr>
wrote:
> Hi,
>
> One of my customers has a huge collection (1.5 billion docs across 14
> shards).
> All fields are correctly configured in order to enable docValues
> except _version_. They are still using the old configuration with
> indexed=true instead of docValues and hence _version_ populate FiledCache
> in JVM heap (several Gb).
>
> They need to reindex for various reasons including this one but this can't
> be done before several weeks due to the complexity to handle full
> reindexing and continuous indexing at the same time.
>
> Why is FieldCache populated with _version_ field as it isn't explicitly
> used in sort, facet, grouping and function ?
> I guess Solr needs this internally.
>
> Is there a workaround in order to avoid FiledCache to be populated by
> _version_ field waiting to reindex ?
>
> Regards
>
> Dominique
>
--
Sincerely yours
Mikhail Khludnev