Atomic updates and "stored"

[Mark Robinson <ma...@gmail.com>]

Hi,

I have some 150 fields in my schema out of which about 100 are dynamic
fields which I am not storing (stored="false").
In case I need to do an atomic update to one or two fields which belong to
the stored list of fields, do I need to change my dynamic fields (100 or so
now not "stored") to stored="true"?

If so wouldn't it considerably increase index size and affect performance
in the negative?

Is there any way currently to do partial/ atomic updates to one or two
fields (which I will make stored="true") without having to make my now
stored="false" fields to stored="true" just
to accommodate atomic updates.

Could some one pls give your suggestions.

Thanks!
Mark.

Re: Atomic updates and "stored"

[Mark Robinson <ma...@gmail.com>]

Thanks Eric!

Best,
Mark

On Mon, May 23, 2016 at 1:35 PM, Erick Erickson <er...@gmail.com>
wrote:

> Yes, currently when using Atomic updates _all_ fields
> have to be stored, except the _destinations_ of copyField
> directives.
>
> Yes, it will make your index bigger. The affects on speed are
> probably minimal though. The stored data is in your *.fdt and
> *.fdx segments files and are not referenced only to pull
> the top N docs back, they're not referenced for _search_ at all.
>
> Coming Real Soon will be updateable DocValues, which may
> be what you really need.
>
> Best,
> Erick
>
> On Mon, May 23, 2016 at 6:13 AM, Mark Robinson <ma...@gmail.com>
> wrote:
> > Hi,
> >
> > I have some 150 fields in my schema out of which about 100 are dynamic
> > fields which I am not storing (stored="false").
> > In case I need to do an atomic update to one or two fields which belong
> to
> > the stored list of fields, do I need to change my dynamic fields (100 or
> so
> > now not "stored") to stored="true"?
> >
> > If so wouldn't it considerably increase index size and affect performance
> > in the negative?
> >
> > Is there any way currently to do partial/ atomic updates to one or two
> > fields (which I will make stored="true") without having to make my now
> > stored="false" fields to stored="true" just
> > to accommodate atomic updates.
> >
> > Could some one pls give your suggestions.
> >
> > Thanks!
> > Mark.
>

Re: Atomic updates and "stored"

[Erick Erickson <er...@gmail.com>]

Yes, currently when using Atomic updates _all_ fields
have to be stored, except the _destinations_ of copyField
directives.

Yes, it will make your index bigger. The affects on speed are
probably minimal though. The stored data is in your *.fdt and
*.fdx segments files and are not referenced only to pull
the top N docs back, they're not referenced for _search_ at all.

Coming Real Soon will be updateable DocValues, which may
be what you really need.

Best,
Erick

On Mon, May 23, 2016 at 6:13 AM, Mark Robinson <ma...@gmail.com> wrote:
> Hi,
>
> I have some 150 fields in my schema out of which about 100 are dynamic
> fields which I am not storing (stored="false").
> In case I need to do an atomic update to one or two fields which belong to
> the stored list of fields, do I need to change my dynamic fields (100 or so
> now not "stored") to stored="true"?
>
> If so wouldn't it considerably increase index size and affect performance
> in the negative?
>
> Is there any way currently to do partial/ atomic updates to one or two
> fields (which I will make stored="true") without having to make my now
> stored="false" fields to stored="true" just
> to accommodate atomic updates.
>
> Could some one pls give your suggestions.
>
> Thanks!
> Mark.