Adding docValues in schema - Solr Cloud 4.8.1

[Vincenzo D'Amore <v....@gmail.com>]

Dear Solr Gurus :),

I would like to add docValues to few fields definition in production.

I first tried in a test environment during partial reindexing and it seems
have no effect, (i.e. no real benefits with small number of documents to
reindex, 30% of total).

So I have to wait a full reindexing in order to be sure all documents have
fields with docValues enabled.

Given that during the partial reindex I suppose that new documents have
fields with docValues enabled and old documents not. And given that I would
change the production configuration seamless, i.e. without to be forced to
full reindex everything.

The question is: what I have to avoid adding docValues to field definition
to an existing production collection, or may the partial reindexing switch
the collection to an inconsistent situation (or critical)?
In other words, should I full reindex immediately after the schema change
and collection reload?

Best regards,
Vincenzo


-- 
Vincenzo D'Amore
email: v.damore@gmail.com
skype: free.dev
mobile: +39 349 8513251

Re: Adding docValues in schema - Solr Cloud 4.8.1

[Vincenzo D'Amore <v....@gmail.com>]

Thanks Shawn,

just to confirm your claim.
Following your suggestion I have double checked my queries with grouping
and faceting.
Faceting and grouping became empty immediately after I have added docValues
.

Thanks again for your support,
Vincenzo

On Fri, Apr 15, 2016 at 4:31 PM, Shawn Heisey <ap...@elyograg.org> wrote:

> On 4/15/2016 7:42 AM, Vincenzo D'Amore wrote:
> > I would like to add docValues to few fields definition in production.
> >
> > I first tried in a test environment during partial reindexing and it
> seems
> > have no effect, (i.e. no real benefits with small number of documents to
> > reindex, 30% of total).
>
> Adding docValues requires a full reindex.  If your index meets the
> criteria for Atomic Updates, then you could do an atomic update on every
> document, but either way, you're going to be indexing every document again.
>
> The problem with not reindexing is that certain Solr features will
> switch to use docValues if the schema says the field has them ... but
> until you reindex, your existing documents will not actually contain
> docValues, so those features will not work on the majority of your
> index.  Those features will *not* fall back to indexed data if the
> schema says docValues="true".
>
> The list of features that won't work right without a reindex is the list
> of features that benefit from docValues (sorting, faceting, grouping),
> so usually there's no reason to add docValues unless you're using at
> least one of those features.
>
> Thanks,
> Shawn
>
>


-- 
Vincenzo D'Amore
email: v.damore@gmail.com
skype: free.dev
mobile: +39 349 8513251

Re: Adding docValues in schema - Solr Cloud 4.8.1

[Shawn Heisey <ap...@elyograg.org>]

On 4/15/2016 7:42 AM, Vincenzo D'Amore wrote:
> I would like to add docValues to few fields definition in production.
>
> I first tried in a test environment during partial reindexing and it seems
> have no effect, (i.e. no real benefits with small number of documents to
> reindex, 30% of total).

Adding docValues requires a full reindex.  If your index meets the
criteria for Atomic Updates, then you could do an atomic update on every
document, but either way, you're going to be indexing every document again.

The problem with not reindexing is that certain Solr features will
switch to use docValues if the schema says the field has them ... but
until you reindex, your existing documents will not actually contain
docValues, so those features will not work on the majority of your
index.  Those features will *not* fall back to indexed data if the
schema says docValues="true".

The list of features that won't work right without a reindex is the list
of features that benefit from docValues (sorting, faceting, grouping),
so usually there's no reason to add docValues unless you're using at
least one of those features.

Thanks,
Shawn