Impact of changing precisionStep on already indexed content

[Jaroslaw Rozanski <so...@jarekrozanski.com>]

Hi all,


What is the impact of changing "precisionStep" without re-indexing
document preceding the change?

Scenario:

Assume you have index with field:
<field name="event" type="tdate" indexed="true" stored="true"/>
<fieldType name="tdate" class="solr.TrieDateField" precisionStep="6"
positionIncrementGap="0"/>

So `tdate` of precision 6. Now let's assume you were to change `tdate`
to be of precision 8.

According to [1] the change will affect how many terms will be stored in
index after type change is in place.

But will that affect queries and sorting if index contains documents
with date precisionStep 6 and 8 at the same time?

Simple test indicates no difference (performance aside) but maybe you
folks are aware of an edge case here.



[1]
https://lucene.apache.org/core/5_5_0/core/org/apache/lucene/search/NumericRangeQuery.html




Cheers,
Jarek

Re: Impact of changing precisionStep on already indexed content

[Jaroslaw Rozanski <me...@jarekrozanski.eu>]

Apologies for dup. Please ignore.

On 26/05/17 18:00, Jaroslaw Rozanski wrote:
> Hi all,
> 
> 
> What is the impact of changing "precisionStep" without re-indexing
> document preceding the change?
> 
> Scenario:
> 
> Assume you have index with field:
> <field name="event" type="tdate" indexed="true" stored="true"/>
> <fieldType name="tdate" class="solr.TrieDateField" precisionStep="6"
> positionIncrementGap="0"/>
> 
> So `tdate` of precision 6. Now let's assume you were to change `tdate`
> to be of precision 8.
> 
> According to [1] the change will affect how many terms will be stored in
> index after type change is in place.
> 
> But will that affect queries and sorting if index contains documents
> with date precisionStep 6 and 8 at the same time?
> 
> Simple test indicates no difference (performance aside) but maybe you
> folks are aware of an edge case here.
> 
> 
> 
> [1]
> https://lucene.apache.org/core/5_5_0/core/org/apache/lucene/search/NumericRangeQuery.html
> 
> 
> 
> 
> Cheers,
> Jarek
> 

-- 
Jaroslaw Rozanski | e: me@jarekrozanski.eu

Re: Impact of changing precisionStep on already indexed content

[Shawn Heisey <ap...@elyograg.org>]

On 5/26/2017 11:00 AM, Jaroslaw Rozanski wrote:
> What is the impact of changing "precisionStep" without re-indexing
> document preceding the change?
>
> Scenario:
>
> Assume you have index with field:
> <field name="event" type="tdate" indexed="true" stored="true"/>
> <fieldType name="tdate" class="solr.TrieDateField" precisionStep="6"
> positionIncrementGap="0"/>
>
> So `tdate` of precision 6. Now let's assume you were to change `tdate`
> to be of precision 8.
>
> According to [1] the change will affect how many terms will be stored in
> index after type change is in place.
>
> But will that affect queries and sorting if index contains documents
> with date precisionStep 6 and 8 at the same time?

Some queries could mysteriously stop working correctly, in a way that
might take a long time to notice.  I'm pretty sure that the
precisionStep is used in the query process, to speed up the queries
based on the step indicated in the schema. The values searched first (at
higher tiers) could be way off what's correct for the documents indexed
with a different step.  At query time, Solr can only go by what's in the
schema, it doesn't look at the index when it is putting its queries
together.

At a minimum, this could mean you won't get some results that match your
query, particularly for range queries, but if the planets align just
right, I can also imagine it including at least one document that
shouldn't be there.

I've got no idea about how it might affect sorting.  My instinct says
that it won't affect sorting, but I do not actually know.

You should definitely completely reindex.  This is probably one
situation where deleting the entire index and restarting is NOT
required, but if it were me, that is the way I'd do it.

Thanks,
Shawn