performance effect a thread doing an update has on other search threads

[derekallwardt <de...@allwardt.com>]

We have an application (backed by Solr 5.x) that does a lot of updates
interleaved with queries. For the sake of better understanding the
performance effect that the ratio of updates to queries has on query
performance, we tested the following two scenarios.

scenario 1:
10 threads doing updates to docs w/ softCommit=true and waitSearcher=true
20 threads doing queries

scenario 2:
20 threads doing updates to docs w/ softCommit=true and waitSearcher=true
10 threads doing queries

- assume that in both scenarios system load, GC, etc... is not contributing
to any performance degradation
- both scenarios pretty much ensure that the Solr caches will always be
invalidated, so the queries in either scenario don't get the benefit of the
cache 

scenario 2 has much worse query performance. Why?



--
View this message in context: http://lucene.472066.n3.nabble.com/performance-effect-a-thread-doing-an-update-has-on-other-search-threads-tp4253411.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: performance effect a thread doing an update has on other search threads

[Erick Erickson <er...@gmail.com>]

Because the second form is opening searchers, invalidating caches and
doing warmup queries twice as often would be my guess.

But this is an invalid test in my opinion. I'm reading this that
you're issuing the soft commits from the clients. This is definitely
an anti-pattern, I _strongly_ recommend that you set your commit (both
soft and hard) in solrconfig.xml and do NOT do this from the clients.

Here's a long blog on the topic:
http://searchhub.org/2013/08/23/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/

Best,
Erick

On Tue, Jan 26, 2016 at 10:04 AM, derekallwardt <de...@allwardt.com> wrote:
> We have an application (backed by Solr 5.x) that does a lot of updates
> interleaved with queries. For the sake of better understanding the
> performance effect that the ratio of updates to queries has on query
> performance, we tested the following two scenarios.
>
> scenario 1:
> 10 threads doing updates to docs w/ softCommit=true and waitSearcher=true
> 20 threads doing queries
>
> scenario 2:
> 20 threads doing updates to docs w/ softCommit=true and waitSearcher=true
> 10 threads doing queries
>
> - assume that in both scenarios system load, GC, etc... is not contributing
> to any performance degradation
> - both scenarios pretty much ensure that the Solr caches will always be
> invalidated, so the queries in either scenario don't get the benefit of the
> cache
>
> scenario 2 has much worse query performance. Why?
>
>
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/performance-effect-a-thread-doing-an-update-has-on-other-search-threads-tp4253411.html
> Sent from the Solr - User mailing list archive at Nabble.com.