You are viewing a plain text version of this content. The canonical link for it is here.

Posted to solr-user@lucene.apache.org by Srinivas Kashyap <sr...@tradestonesoftware.com> on 2018/05/02 13:29:54 UTC

count mismatch: number of records indexed

Hi,

I have standalone solr index server 5.2.1 and have a core with 15 fields(all indexed and stored).

Through DIH I'm indexing the data (around 65million records). The index process took 6hours to complete. But after the completion when I checked through Solr admin query console(*:*), numfound is only 41 thousand records. Am I missing some configuration to index all records?

Physical memory: 16GB
JVM memory: 4GB

Thanks,
Srinivas

Re: count mismatch: number of records indexed

Posted by Erick Erickson <er...@gmail.com>.

And if you _do_ have a uniqueKey ("id" by default), subsequent records
will overwrite older records with the same key.

The tip from Annameneni is the first thing I'd try though, make sure
you've issued a commit.

Best,
Erick

On Wed, May 2, 2018 at 7:09 AM, ANNAMANENI RAVEENDRA
<a....@gmail.com> wrote:
> Possible cases can be
>
> If you don’t have unique key then there are high chances that you will see
> less data
> Try hard commit or check your commit times (hard/soft)
>
>
> On Wed, May 2, 2018 at 9:30 AM Srinivas Kashyap <
> srinivas@tradestonesoftware.com> wrote:
>
>> Hi,
>>
>> I have standalone solr index server 5.2.1 and have a core with 15
>> fields(all indexed and stored).
>>
>> Through DIH I'm indexing the data (around 65million records). The index
>> process took 6hours to complete. But after the completion when I checked
>> through Solr admin query console(*:*), numfound is only 41 thousand
>> records. Am I missing some configuration to index all records?
>>
>> Physical memory: 16GB
>> JVM memory: 4GB
>>
>> Thanks,
>> Srinivas
>>

Re: count mismatch: number of records indexed

Posted by ANNAMANENI RAVEENDRA <a....@gmail.com>.

Possible cases can be

If you don’t have unique key then there are high chances that you will see
less data
Try hard commit or check your commit times (hard/soft)


On Wed, May 2, 2018 at 9:30 AM Srinivas Kashyap <
srinivas@tradestonesoftware.com> wrote:

> Hi,
>
> I have standalone solr index server 5.2.1 and have a core with 15
> fields(all indexed and stored).
>
> Through DIH I'm indexing the data (around 65million records). The index
> process took 6hours to complete. But after the completion when I checked
> through Solr admin query console(*:*), numfound is only 41 thousand
> records. Am I missing some configuration to index all records?
>
> Physical memory: 16GB
> JVM memory: 4GB
>
> Thanks,
> Srinivas
>