Storage of indexed and stored fields (Space and Performance)

[Rajnish kamboj <ra...@gmail.com>]

Hi



How are indexed and stored fields treated by Lucene w.r.t space and
performance?

Is there any performance hit with stored fields which are indexed?



Lucene Version: 5.3.1



Assumption:

Stored fields are just simple strings (not huge documents)



Example:

Data: [101, Gold]; [102, Silver]; [103, Gold]

Additional Data: Stored and indexed as well: Partition1, Partition2,
Schema1, Version1 etc. depending on data



Index:

Gold: 101 (Partition1, Schema1, Version1) , 103 (Partition2, Schema1,
Version1)

Silver: 102 (Partition1, Schema1, Version1)

Partition1: 101 (Partition1, Schema1, Version1), 102 (Partition1, Schema1,
Version1)

Partition2: 103 (Partition1, Schema1, Version1)

Schema1: 101 (Partition1, Schema1, Version1), 102 (Partition1, Schema1,
Version1), 103 (Partition1, Schema1, Version1)

Version1: 101 (Partition1, Schema1, Version1), 102 (Partition1, Schema1,
Version1), 103 (Partition1, Schema1, Version1)




Is it how the index will look like? i.e. stored fields will be replicated
with each data field?





Thanks & Regards

Rajnish

Re: Storage of indexed and stored fields (Space and Performance)

[Erick Erickson <er...@gmail.com>]

Stored data is kept in separate segment files (*.fdt and *.fdx). As
such they have no measurable impact on query time. All the data for
executing searches is kept in other extensions in each segment and
accessed separately.

Adding stored data does increase the size on disk by roughly 50% of
the number of bytes stored (i.e. if I have a field with 128 bytes, the
_stored_ portion of the data will occupy roughly 64 bytes) and will
add some I/O but by and large the effects can be ignored.

Best,
Erick

On Thu, Mar 15, 2018 at 6:48 AM, Rajnish kamboj
<ra...@gmail.com> wrote:
> Hi
>
>
>
> How are indexed and stored fields treated by Lucene w.r.t space and
> performance?
>
> Is there any performance hit with stored fields which are indexed?
>
>
>
> Lucene Version: 5.3.1
>
>
>
> Assumption:
>
> Stored fields are just simple strings (not huge documents)
>
>
>
> Example:
>
> Data: [101, Gold]; [102, Silver]; [103, Gold]
>
> Additional Data: Stored and indexed as well: Partition1, Partition2,
> Schema1, Version1 etc. depending on data
>
>
>
> Index:
>
> Gold: 101 (Partition1, Schema1, Version1) , 103 (Partition2, Schema1,
> Version1)
>
> Silver: 102 (Partition1, Schema1, Version1)
>
> Partition1: 101 (Partition1, Schema1, Version1), 102 (Partition1, Schema1,
> Version1)
>
> Partition2: 103 (Partition1, Schema1, Version1)
>
> Schema1: 101 (Partition1, Schema1, Version1), 102 (Partition1, Schema1,
> Version1), 103 (Partition1, Schema1, Version1)
>
> Version1: 101 (Partition1, Schema1, Version1), 102 (Partition1, Schema1,
> Version1), 103 (Partition1, Schema1, Version1)
>
>
>
>
> Is it how the index will look like? i.e. stored fields will be replicated
> with each data field?
>
>
>
>
>
> Thanks & Regards
>
> Rajnish

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org