Indexing fails with partially done

[neotorand <ne...@gmail.com>]

with Solrcloud What happens if indexing is partially completed and ensemble 
goes down.What are the ways to Resume.In one of the scenario i am using 3 ZK 
Node in ensemble.Lets say i am indexing 5 million data and i have partially 
indexed the data and ZK ensemble goes down. What should be the best approach 
for handling such scenario 

Regards 
Neo 



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: Indexing fails with partially done

[Emir Arnautović <em...@sematext.com>]

Hi Neo,
My DIH knowledge is a bit rusty, but I think that in best case, depending on your queries you might be able to use delta update to “resume” indexing, but it is likely that you cannot do that.

Emir
--
Monitoring - Log Management - Alerting - Anomaly Detection
Solr & Elasticsearch Consulting Support Training - http://sematext.com/



> On 11 Apr 2018, at 15:34, neotorand <ne...@gmail.com> wrote:
> 
> Thanks Emir
> with context to DIH do we have any Resume mechanism?
> 
> Regards
> Neo
> 
> 
> 
> 
> --
> Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: Indexing fails with partially done

[neotorand <ne...@gmail.com>]

Thanks Emir
with context to DIH do we have any Resume mechanism?

Regards
Neo




--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: Indexing fails with partially done

[Emir Arnautović <em...@sematext.com>]

Hi,
First of all, the purpose of having 3 ZK in ensemble is to minimise the chances of loosing quorum. With 3 ZK, you can loose one and still have operable ZK ensemble. You should monitor it and alert if you loose one ZK and react to alert before you loose the second one. If you are wondering how to monitor it, you can check out Sematext’s performance monitor: https://sematext.com/spm/ <https://sematext.com/spm/>
If you loose the second ZK, then you either ignored the alert or you are really unlucky person or you have something terribly wrong in your setup.

Anyway, if that happens, there is nothing in Solr that keeps track what is indexed and what is not. You need to track that on your client side. It can be loosely tracking by some timestamp or strictly tracked by acknowledging every indexed document. In some cases it might be the simplest if you do full reindexing.

HTH,
Emir
--
Monitoring - Log Management - Alerting - Anomaly Detection
Solr & Elasticsearch Consulting Support Training - http://sematext.com/



> On 11 Apr 2018, at 14:46, neotorand <ne...@gmail.com> wrote:
> 
> with Solrcloud What happens if indexing is partially completed and ensemble 
> goes down.What are the ways to Resume.In one of the scenario i am using 3 ZK 
> Node in ensemble.Lets say i am indexing 5 million data and i have partially 
> indexed the data and ZK ensemble goes down. What should be the best approach 
> for handling such scenario 
> 
> Regards 
> Neo 
> 
> 
> 
> --
> Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: Indexing fails with partially done

[Shawn Heisey <ap...@elyograg.org>]

On 4/11/2018 6:46 AM, neotorand wrote:
> with Solrcloud What happens if indexing is partially completed and ensemble 
> goes down.What are the ways to Resume.In one of the scenario i am using 3 ZK 
> Node in ensemble.Lets say i am indexing 5 million data and i have partially 
> indexed the data and ZK ensemble goes down. What should be the best approach 
> for handling such scenario 

If you lose quorum on your ZK ensemble, SolrCloud will go read-only, and
indexing will not be possible.  Solr will remember the last clusterstate
that the ZK database held and will attempt to keep serving queries using
that information.  Whether or not that succeeds will depend on whether
your Solr servers stay operational.

If you restart a Solr server while ZK has no quorum, then I'm not sure
what happens.  It probably doesn't start correctly and may need another
restart after ZK is restored.

Thanks,
Shawn