frequently update field

[sara hajili <ha...@gmail.com>]

hi
i am new in solr, i face to a problem and need any solution to solve that.
i have a field that this field need to update frequently.
"image i need to index all post of member of a social app"
in this case i need to store and index all posts field like caption ,
image, title,comments ,etc
but question is about some field like
"like_count,repost_count,comment_count" this field frequenly changed and i
need to update that but other like caption ,title are not as the same of
like count field.
so what is the best solution to handle this frequntly update..
i found that in solr 4 people used external file.
but now in solr 5.x i see that atomic update appear.
atomic update is substitute of extenal file?and what is best approach in
this case?
"i really worry about cost of re indexing docs when update like count"

Re: frequently update field

[Upayavira <uv...@odoko.co.uk>]

On Sat, Sep 5, 2015, at 06:16 AM, sara hajili wrote:
> hi
> i am new in solr, i face to a problem and need any solution to solve
> that.
> i have a field that this field need to update frequently.
> "image i need to index all post of member of a social app"
> in this case i need to store and index all posts field like caption ,
> image, title,comments ,etc
> but question is about some field like
> "like_count,repost_count,comment_count" this field frequenly changed and
> i
> need to update that but other like caption ,title are not as the same of
> like count field.
> so what is the best solution to handle this frequntly update..
> i found that in solr 4 people used external file.
> but now in solr 5.x i see that atomic update appear.
> atomic update is substitute of extenal file?and what is best approach in
> this case?
> "i really worry about cost of re indexing docs when update like count"

You are right to worry about the re-indexing cost.

You need to think about what you want to do with the field once it is
indexed.

Do you want to filter based upon it? Do you want to score based upon it?
Do you want to get the value back in a search result?

The first two can be done, the third hasn't yet been implemented in
Solr. 

Upayavira

Re: frequently update field

[sara hajili <ha...@gmail.com>]

tnx for comparison of  external file and atomic update

On Sat, Sep 5, 2015 at 6:53 AM, Jack Krupansky <ja...@gmail.com>
wrote:

> The standard recommendation is to create a proof of concept implementation
> and see how well it performs.
>
> The external file approach is intended for bulk update, such as when the
> pricing for many products will be updated all at once.
>
> Atomic update is recommended for incremental updates.
>
> Atomic update does depend of setting all source fields to stored since the
> entire document must be first read from the stored values before updating
> the selected fields.
>
> If storing all source fields is prohibitive, then they must be stored in an
> external database so that the full documents can be reindexed when updating
> is required.
>
> As with any database question, the first thing you must do is identify your
> access patterns - how much data will you be updating and with what
> frequency.
>
> Generally, atomic update is recommended when only a small fraction of the
> data will be updated in some relatively small interval of time, such as
> hundreds of documents per hour or dozens of documents per minute, or a
> handful per second.
>
>
> -- Jack Krupansky
>
> On Sat, Sep 5, 2015 at 1:16 AM, sara hajili <ha...@gmail.com> wrote:
>
> > hi
> > i am new in solr, i face to a problem and need any solution to solve
> that.
> > i have a field that this field need to update frequently.
> > "image i need to index all post of member of a social app"
> > in this case i need to store and index all posts field like caption ,
> > image, title,comments ,etc
> > but question is about some field like
> > "like_count,repost_count,comment_count" this field frequenly changed and
> i
> > need to update that but other like caption ,title are not as the same of
> > like count field.
> > so what is the best solution to handle this frequntly update..
> > i found that in solr 4 people used external file.
> > but now in solr 5.x i see that atomic update appear.
> > atomic update is substitute of extenal file?and what is best approach in
> > this case?
> > "i really worry about cost of re indexing docs when update like count"
> >
>

Re: frequently update field

[Jack Krupansky <ja...@gmail.com>]

The standard recommendation is to create a proof of concept implementation
and see how well it performs.

The external file approach is intended for bulk update, such as when the
pricing for many products will be updated all at once.

Atomic update is recommended for incremental updates.

Atomic update does depend of setting all source fields to stored since the
entire document must be first read from the stored values before updating
the selected fields.

If storing all source fields is prohibitive, then they must be stored in an
external database so that the full documents can be reindexed when updating
is required.

As with any database question, the first thing you must do is identify your
access patterns - how much data will you be updating and with what
frequency.

Generally, atomic update is recommended when only a small fraction of the
data will be updated in some relatively small interval of time, such as
hundreds of documents per hour or dozens of documents per minute, or a
handful per second.


-- Jack Krupansky

On Sat, Sep 5, 2015 at 1:16 AM, sara hajili <ha...@gmail.com> wrote:

> hi
> i am new in solr, i face to a problem and need any solution to solve that.
> i have a field that this field need to update frequently.
> "image i need to index all post of member of a social app"
> in this case i need to store and index all posts field like caption ,
> image, title,comments ,etc
> but question is about some field like
> "like_count,repost_count,comment_count" this field frequenly changed and i
> need to update that but other like caption ,title are not as the same of
> like count field.
> so what is the best solution to handle this frequntly update..
> i found that in solr 4 people used external file.
> but now in solr 5.x i see that atomic update appear.
> atomic update is substitute of extenal file?and what is best approach in
> this case?
> "i really worry about cost of re indexing docs when update like count"
>