You are viewing a plain text version of this content. The canonical link for it is here.

Posted to solr-user@lucene.apache.org by "Benjamin, Roy" <rb...@ebay.com> on 2013/03/26 00:06:01 UTC

Any experience with adding documents batch sizes?

My application is update intensive.  The documents are pretty small, less than 1K bytes.

Just now I'm batching 4K documents with each SolrJ addDocs() call.

Wondering what I should expect with increasing this batch size?  Say 8K docs per update?

Thanks

Roy


Solr 3.6

RE: Any experience with adding documents batch sizes?

Posted by "Benjamin, Roy" <rb...@ebay.com>.

Thanks Otis!

-----Original Message-----
From: Otis Gospodnetic [mailto:otis.gospodnetic@gmail.com] 
Sent: Monday, March 25, 2013 7:41 PM
To: solr-user@lucene.apache.org
Subject: Re: Any experience with adding documents batch sizes?

Hi,

You'll have to test because there is no general rule that works in all environments, but from testing this a while back, you will reach the point of diminishing returns at some point.  You don't mention using StreamingUpdateSolrServer, so you may want to try that instead:
http://lucene.apache.org/solr/api-3_6_1/org/apache/solr/client/solrj/impl/StreamingUpdateSolrServer.html

Otis
--
Solr & ElasticSearch Support
http://sematext.com/

On Mon, Mar 25, 2013 at 7:06 PM, Benjamin, Roy <rb...@ebay.com> wrote:

> My application is update intensive.  The documents are pretty small, 
> less than 1K bytes.
>
> Just now I'm batching 4K documents with each SolrJ addDocs() call.
>
> Wondering what I should expect with increasing this batch size?  Say 
> 8K docs per update?
>
> Thanks
>
> Roy
>
>
> Solr 3.6
>
>
>
>

Re: Any experience with adding documents batch sizes?

Posted by Otis Gospodnetic <ot...@gmail.com>.

Hi,

You'll have to test because there is no general rule that works in all
environments, but from testing this a while back, you will reach the point
of diminishing returns at some point.  You don't mention using
StreamingUpdateSolrServer, so you may want to try that instead:
http://lucene.apache.org/solr/api-3_6_1/org/apache/solr/client/solrj/impl/StreamingUpdateSolrServer.html

Otis
--
Solr & ElasticSearch Support
http://sematext.com/

On Mon, Mar 25, 2013 at 7:06 PM, Benjamin, Roy <rb...@ebay.com> wrote:

> My application is update intensive.  The documents are pretty small, less
> than 1K bytes.
>
> Just now I'm batching 4K documents with each SolrJ addDocs() call.
>
> Wondering what I should expect with increasing this batch size?  Say 8K
> docs per update?
>
> Thanks
>
> Roy
>
>
> Solr 3.6
>
>
>
>