You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by ch...@opensgi.com on 2013/06/03 23:32:33 UTC

Inconsistent Full import document index counts.

Hello All,
 
I've been working on a 2-shard SolrCloud instance with several million documents, and the import process has recently begun to miss documents as they are added to the underlying Postgres database. There are no glaring failures in the log files (all SEVERE and WARNING level errors in the log are from malformed queries). To ensure that it is not an issue with my delta-import query, I've tried running full imports to no avail. Strangely, when I modify my data-import query to only search for a specific id that is missed in the full-import, all of the relevant documents are indexed. Any ideas for possible causes of missed document imports in long-running full-imports?
 
Thanks,
Chris Donaher
 
 

Re: Inconsistent Full import document index counts.

Posted by Erick Erickson <er...@gmail.com>.
Hmmm, could it be you aren't sending the docs to Solr?

I'm guessing your delta-import is somehow missing the
records in the select, but that's just a guess.

FWIW,
Erick

On Mon, Jun 3, 2013 at 5:32 PM,  <ch...@opensgi.com> wrote:
>
> Hello All,
>
> I've been working on a 2-shard SolrCloud instance with several million documents, and the import process has recently begun to miss documents as they are added to the underlying Postgres database. There are no glaring failures in the log files (all SEVERE and WARNING level errors in the log are from malformed queries). To ensure that it is not an issue with my delta-import query, I've tried running full imports to no avail. Strangely, when I modify my data-import query to only search for a specific id that is missed in the full-import, all of the relevant documents are indexed. Any ideas for possible causes of missed document imports in long-running full-imports?
>
> Thanks,
> Chris Donaher
>
>