omitNorms for short searchable fields and ID field

[Chaula Ganatra <cg...@asite.com>]

Hello

We have a use case with very large index split on 2 shards and 2 replicas. Each shard has around 200GB data.
We want to reduce our index size and for which we tried to do omit Norms for all the fields. We have also done it for the ID field and short searchable fields.
We have observed that it has reduced the size to almost 50% . So we would like to go with this setting.
Will it have any impact on performance or functionally? Please note we do not have any large text field for searching. But we have many different fields (dynamic fields) in our index.

Regards
Chaula

[CC Award Winners!]

Re: omitNorms for short searchable fields and ID field

[Atita Arora <at...@gmail.com>]

Hi Chaula,

Omitnorms are basically used for index time boost & field length
normalization saying that I meant when you do omitNorms=true for any field
it stops storing additional stats regarding terms , length , boosts etc for
that field and hence drastically reduces the size of index.
It infact is advisable to turn this to true for small fields like you
mentioned you have in your schema.
It should not impact adversely on performance as long as you dont have a
high field length / index boost related operations in your use case.

-Atita

On Fri, Aug 25, 2017 at 12:19 PM, Chaula Ganatra <cg...@asite.com> wrote:

> Hello
>
> We have a use case with very large index split on 2 shards and 2 replicas.
> Each shard has around 200GB data.
> We want to reduce our index size and for which we tried to do omit Norms
> for all the fields. We have also done it for the ID field and short
> searchable fields.
> We have observed that it has reduced the size to almost 50% . So we would
> like to go with this setting.
> Will it have any impact on performance or functionally? Please note we do
> not have any large text field for searching. But we have many different
> fields (dynamic fields) in our index.
>
> Regards
> Chaula
>
> [CC Award Winners!]
>
>