Solr DIH - Small index still take time?

[Spadez <ja...@hotmail.com>]

Hi,

This is the situation, I have two sources of data in my dataimport handler,
one is huge, the other is tiny:

Source A: 10-20 records
Source B: 50,000,000 records

I was wondering what happens if I was to do a DIH just on Source A every 10
mins, and only run the DIH on source B every 24 hours.

Would running my DIH on Source A be extremely quick, because the data we are
importing is small, or would it still be time consuming, because it would
have to rebuild the index of the entire SOLR (i.e 50,000,010 records).

Thank you!



--
View this message in context: http://lucene.472066.n3.nabble.com/Solr-DIH-Small-index-still-take-time-tp4065582.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Solr DIH - Small index still take time?

[Alexandre Rafalovitch <ar...@gmail.com>]

That should work. Just watch out for (set value of)
preImportDeleteQuery. Otherwise, when you do full import you may
accidentally delete items from the other set.

Regards,
   Alex.
Personal blog: http://blog.outerthoughts.com/
LinkedIn: http://www.linkedin.com/in/alexandrerafalovitch
- Time is the quality of nature that keeps events from happening all
at once. Lately, it doesn't seem to be working.  (Anonymous  - via GTD
book)


On Thu, May 23, 2013 at 6:25 AM, Spadez <ja...@hotmail.com> wrote:
> Hi,
>
> This is the situation, I have two sources of data in my dataimport handler,
> one is huge, the other is tiny:
>
> Source A: 10-20 records
> Source B: 50,000,000 records
>
> I was wondering what happens if I was to do a DIH just on Source A every 10
> mins, and only run the DIH on source B every 24 hours.
>
> Would running my DIH on Source A be extremely quick, because the data we are
> importing is small, or would it still be time consuming, because it would
> have to rebuild the index of the entire SOLR (i.e 50,000,010 records).
>
> Thank you!
>
>
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/Solr-DIH-Small-index-still-take-time-tp4065582.html
> Sent from the Solr - User mailing list archive at Nabble.com.