Disabling Commit/Auto-Commit (SolrCloud)

[Software Dev <st...@gmail.com>]

Is there a way to disable commit/hard-commit at runtime? For example, we
usually have our hard commit and soft-commit set really low but when we do
bulk indexing we would like to disable this to increase performance. If
there isn't a an easy way of doing this would simply pushing a new
solrconfig to solrcloud work?

Re: Disabling Commit/Auto-Commit (SolrCloud)

[Alexei Martchenko <al...@martchenko.com.br>]

I didn't mean to disable, just to put some high value there. I have a
script that updates my solr in batches of thousands so I set my commit to
100,000 because when it runs it updates 100,000 records in short time.

The other script updates in batches of hundreds and its not so fast, so its
internal loops issue a commit after X loops and/or when it finishes
processing.


alexei martchenko
Facebook <http://www.facebook.com/alexeiramone> |
Linkedin<http://br.linkedin.com/in/alexeimartchenko>|
Steam <http://steamcommunity.com/id/alexeiramone/> |
4sq<https://pt.foursquare.com/alexeiramone>| Skype: alexeiramone |
Github <https://github.com/alexeiramone> | (11) 9 7613.0966 |


2014-01-31 Mark Miller <ma...@gmail.com>:

> It's not a good idea to disable hard commit because the transaction can
> grow without limit in RAM.
>
> Also, try some performance tests. I've never seen it matter if it's set to
> like a minute, both for bulk and NRT.
>
> As far as soft commit, you could turn it off and control visibility when
> adding docs via commitWithin.
>
> - Mark
>
> http://about.me/markrmiller
>
> On Jan 31, 2014, at 12:45 PM, Software Dev <st...@gmail.com>
> wrote:
>
> > Is there a way to disable commit/hard-commit at runtime? For example, we
> > usually have our hard commit and soft-commit set really low but when we
> do
> > bulk indexing we would like to disable this to increase performance. If
> > there isn't a an easy way of doing this would simply pushing a new
> > solrconfig to solrcloud work?
>
>

Re: Disabling Commit/Auto-Commit (SolrCloud)

[Mark Miller <ma...@gmail.com>]

It’s not a good idea to disable hard commit because the transaction can grow without limit in RAM.

Also, try some performance tests. I’ve never seen it matter if it’s set to like a minute, both for bulk and NRT.

As far as soft commit, you could turn it off and control visibility when adding docs via commitWithin.

- Mark

http://about.me/markrmiller

On Jan 31, 2014, at 12:45 PM, Software Dev <st...@gmail.com> wrote:

> Is there a way to disable commit/hard-commit at runtime? For example, we
> usually have our hard commit and soft-commit set really low but when we do
> bulk indexing we would like to disable this to increase performance. If
> there isn't a an easy way of doing this would simply pushing a new
> solrconfig to solrcloud work?

Re: Disabling Commit/Auto-Commit (SolrCloud)

[Alexei Martchenko <al...@martchenko.com.br>]

Why don't you set both solrconfig commits to very high values and issue a
commit command in sparsed, small updates?

I've been doing this for ages and works perfecly for me.


alexei martchenko
Facebook <http://www.facebook.com/alexeiramone> |
Linkedin<http://br.linkedin.com/in/alexeimartchenko>|
Steam <http://steamcommunity.com/id/alexeiramone/> |
4sq<https://pt.foursquare.com/alexeiramone>| Skype: alexeiramone |
Github <https://github.com/alexeiramone> | (11) 9 7613.0966 |


2014-01-31 Software Dev <st...@gmail.com>:

> Is there a way to disable commit/hard-commit at runtime? For example, we
> usually have our hard commit and soft-commit set really low but when we do
> bulk indexing we would like to disable this to increase performance. If
> there isn't a an easy way of doing this would simply pushing a new
> solrconfig to solrcloud work?
>