commit persistence guarantee

[Alvaro Cabrerizo <to...@gmail.com>]

Hi,

Is there any guarantee that every document is persisted on disk during a
"commit avalanche" that produces the: "ERROR org.apache.solr.core.SolrCore
 – org.apache.solr.common.SolrException: Error opening new searcher. *exceeded
limit of maxWarmingSearchers*=1, try again later".

I've made some tests using jmeter to generate the situation and I
*allways*get all the documents *well
stored*, although having ~4% of requests with a 503 response, complaining
with the previous message in the log.

Regards.

notes:  I know about NearRealTime and the possibility of modifying the
commit strategy in order to be more polite with Solr ;)

Re: commit persistence guarantee

[Erick Erickson <er...@gmail.com>]

This is almost always that you're committing too often, either  soft
commit or hard commit with openSearcher=true. Shouldn't have any
effect on the consistency of your index though.

It _is_ making your Solr work harder than you want it to, so consider
increasing the commit intervals substantially. If you're indexing from
SolrJ, it's _not_ a good idea to commit except, perhaps, at the very
end of the run. Let your solrconfig settings commit for you. Super
especially if you're indexing form multiple SolrJ programs.

Best,
Erick

On Wed, May 7, 2014 at 3:02 AM, Alvaro Cabrerizo <to...@gmail.com> wrote:
> Hi,
>
> Is there any guarantee that every document is persisted on disk during a
> "commit avalanche" that produces the: "ERROR org.apache.solr.core.SolrCore
>  – org.apache.solr.common.SolrException: Error opening new searcher. *exceeded
> limit of maxWarmingSearchers*=1, try again later".
>
> I've made some tests using jmeter to generate the situation and I
> *allways*get all the documents *well
> stored*, although having ~4% of requests with a 503 response, complaining
> with the previous message in the log.
>
> Regards.
>
> notes:  I know about NearRealTime and the possibility of modifying the
> commit strategy in order to be more polite with Solr ;)