You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by tbarkley29 <tb...@gmail.com> on 2020/02/20 22:31:33 UTC
Re: Solr 7.6 frequent OOM with Java 9, G1 and large heap sizes -
any tests with Java 13 and the new ZGC?
We are currently running performance tests with Solr 8.2/OpenJDK11/ZGC. We've
ran multiple successful 12 hour tests and are currently running 24 hour
tests. There are three nodes which are 4 cores and 28GB memory, JVM is 16GB.
We are getting max ~780 Page Per Second with max of ~8,000 users/min. CPU
utilization stays around 80% and memory utilization stays around 18%. We
were trying various configurations with G1GC which were unsuccessful after
about 8 hours. We also tried with CMS which failed within an hour or so.
Queries used in test were taken from Splunk from production traffic. These
performance tests are still ongoing and I'd be open to providing JVM metrics
if interested.
--
Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: Solr 7.6 frequent OOM with Java 9, G1 and large heap sizes - any
tests with Java 13 and the new ZGC?
Posted by Paras Lehana <pa...@indiamart.com>.
Hi,
We are running another 24 hour test with 8GB JVM and so far it is also
> running flawlessly.
If this is the case, as Erick mentioned, the failures were probably due to
long GC pauses. During couple of my stress testings, I had found that
decreasing JVM helps sometimes (it makes GC more frequent and less
intensive in a way). Try with different heap sizes and also consider tuning
GC.
Also, do let us about the performance of ZGC against G1GC - I'm curious.
I'm using Java 11.
On Sun, 23 Feb 2020 at 01:28, tbarkley29 <tb...@gmail.com> wrote:
> Yes 18% of total physical RAM. The failures in G1GC and CMS setup did seem
> to
> be from pause the world.
>
> We are using Solr Docker image which is using G1GC by default and we tuned
> with G1GC. Even with tuning the performance test failed after about 8
> hours.
> With ZGC we had consistent 12 and 24 hour performance test which ran
> flawlessly.
>
> We are running another 24 hour test with 8GB JVM and so far it is also
> running flawlessly. I will post an update when completed.
>
> Garbage collection is not my area of expertise but so far I am following
> the
> data and out of the box ZGC is performing drastically better.
>
>
>
> --
> Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html
>
--
--
Regards,
*Paras Lehana* [65871]
Development Engineer, *Auto-Suggest*,
IndiaMART InterMESH Ltd,
11th Floor, Tower 2, Assotech Business Cresterra,
Plot No. 22, Sector 135, Noida, Uttar Pradesh, India 201305
Mob.: +91-9560911996
Work: 0120-4056700 | Extn:
*11096*
--
*
*
<https://www.facebook.com/IndiaMART/videos/578196442936091/>
Re: Solr 7.6 frequent OOM with Java 9, G1 and large heap sizes -
any tests with Java 13 and the new ZGC?
Posted by tbarkley29 <tb...@gmail.com>.
Yes 18% of total physical RAM. The failures in G1GC and CMS setup did seem to
be from pause the world.
We are using Solr Docker image which is using G1GC by default and we tuned
with G1GC. Even with tuning the performance test failed after about 8 hours.
With ZGC we had consistent 12 and 24 hour performance test which ran
flawlessly.
We are running another 24 hour test with 8GB JVM and so far it is also
running flawlessly. I will post an update when completed.
Garbage collection is not my area of expertise but so far I am following the
data and out of the box ZGC is performing drastically better.
--
Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: Solr 7.6 frequent OOM with Java 9, G1 and large heap sizes - any
tests with Java 13 and the new ZGC?
Posted by Erick Erickson <er...@gmail.com>.
People are certainly interested. You’re running on the bleeding edge of
technology, you’re very brave ;).
I’m not quite sure how to interpret “memory utilization stays around 18%”.
18% of total physical RAM or heap? I’m assuming the former..
I’m curious, how did CMS and G1GC fail? It’s perfectly understandable if
the failures were due to stop-the-world GC pauses; they can lead to timeouts
which can cause replicas to be put into recovery, or Zookeeper to think
the node died etc… In extreme cases this means that the entire cluster goes
down.
Best,
Erick
> On Feb 20, 2020, at 5:31 PM, tbarkley29 <tb...@gmail.com> wrote:
>
> We are currently running performance tests with Solr 8.2/OpenJDK11/ZGC. We've
> ran multiple successful 12 hour tests and are currently running 24 hour
> tests. There are three nodes which are 4 cores and 28GB memory, JVM is 16GB.
> We are getting max ~780 Page Per Second with max of ~8,000 users/min. CPU
> utilization stays around 80% and memory utilization stays around 18%. We
> were trying various configurations with G1GC which were unsuccessful after
> about 8 hours. We also tried with CMS which failed within an hour or so.
> Queries used in test were taken from Splunk from production traffic. These
> performance tests are still ongoing and I'd be open to providing JVM metrics
> if interested.
>
>
>
> --
> Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html