solr parallel update and total indexing Issue

[~$alpha` <la...@gmail.com>]

There is a bis issue in solr parallel update and total indexing

    Total Import syntax (working)
    dataimport?command=full-import&commit=true&optimize=true 

    Update syntax(working)
    solr/update?softCommit=true' -H 'Content-type:application/json' -d
'[{"id":"1870719","column":{"set":11}}]'


Issue: If both are run in parallel, then commit in b/w take place.

Example: i have 10k in total indexes.... i fire an solr query to update 1000
records and in between i fire a total import(full indexer).... what's
happening is that in between commit is taken place... i.e untill total
indexer runs i got limited records(1000).

How to solve this ?



--
View this message in context: http://lucene.472066.n3.nabble.com/solr-parallel-update-and-total-indexing-Issue-tp4131935.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: solr parallel update and total indexing Issue

[Erick Erickson <er...@gmail.com>]

try not setting softCommit=true, that's going to take the current
state of your index and make it visible. If your DIH process has
deleted all your records, then that's the "current state".

Personally I wouldn't try to mix-n-match like this, the results will
take forever to get right. If you absolutely must do something like
this, I'd use collection aliasing to rebuild my index in a different
collection then switch from the old to new one in a controlled
fashion.

Best,
Erick

On Thu, Apr 17, 2014 at 11:37 PM, ~$alpha` <la...@gmail.com> wrote:
> There is a bis issue in solr parallel update and total indexing
>
>     Total Import syntax (working)
>     dataimport?command=full-import&commit=true&optimize=true
>
>     Update syntax(working)
>     solr/update?softCommit=true' -H 'Content-type:application/json' -d
> '[{"id":"1870719","column":{"set":11}}]'
>
>
> Issue: If both are run in parallel, then commit in b/w take place.
>
> Example: i have 10k in total indexes.... i fire an solr query to update 1000
> records and in between i fire a total import(full indexer).... what's
> happening is that in between commit is taken place... i.e untill total
> indexer runs i got limited records(1000).
>
> How to solve this ?
>
>
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/solr-parallel-update-and-total-indexing-Issue-tp4131935.html
> Sent from the Solr - User mailing list archive at Nabble.com.