You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Erick Erickson <er...@gmail.com> on 2019/03/25 16:04:58 UTC
We have a cause for high CPU usage on Solr 7.7
From the user’s list (thanks to Adam Guthrie and Lukas Weiss)
Short form: Apparently there’s a java 8 bug that’s the root cause, not fixed until Java 9: https://bugs.openjdk.java.net/browse/JDK-8129861
I have _not_ verified this, but we’ve had sporadic reports (Shawn?) of same. CommitTracker is the only place in the code that calls newScheduledThreadPool with 0.
From what I can tell, this isn’t fixed in Java until 9.
If the fix is just changing that call to use 1, it’s trivial but would require a respin of 7.7 to get out in the field.
Should I raise a JIRA? ‘cause we should be seeing it in Solr 8 and master too. Although we should be seeing this in Solr 8 too, although it should disappear in Master when we require Java 11.
************E-mail below.
Hi,
Apologies, I can’t figure out how to reply to the Solr mailing list.
I just ran across the same high CPU usage issue. I believe it’’s caused by
this commit which was introduced in Solr 7.7.0
https://github.com/apache/lucene-solr/commit/eb652b84edf441d8369f5188cdd5e3ae2b151434#diff-e54b251d166135a1afb7938cfe152bb5
There is a bug in JDK versions <=8 where using 0 threads in the
ScheduledThreadPool causes high CPU usage:
https://bugs.openjdk.java.net/browse/JDK-8129861
Oddly, the latest version
of solr/core/src/java/org/apache/solr/update/CommitTracker.java on
master still uses 0 executors as the default. Presumably most everyone is
using JDK 9 or greater which has the bug fixed, so they don’t experience
the bug.
Feel free to relay this back to the mailing list.
Thanks,
Adam Guthrie
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: We have a cause for high CPU usage on Solr 7.7
Posted by Ishan Chattopadhyaya <ic...@gmail.com>.
This is now tracked in SOLR-13349
<https://issues.apache.org/jira/browse/SOLR-13349>. I don't think 6.6 is
affected by this.
On Tue, Mar 26, 2019 at 3:50 PM Adrien Grand <jp...@gmail.com> wrote:
> +1 to raise a JIRA. There is an easy workaround on the user side by
> using a recent JVM, but at the same time the change to the code looks
> safe. Does it affect 6.6? I see there is an ongoing thread about
> getting 6.6.6 out.
>
> On Mon, Mar 25, 2019 at 5:05 PM Erick Erickson <er...@gmail.com>
> wrote:
> >
> > From the user’s list (thanks to Adam Guthrie and Lukas Weiss)
> >
> > Short form: Apparently there’s a java 8 bug that’s the root cause, not
> fixed until Java 9: https://bugs.openjdk.java.net/browse/JDK-8129861
> >
> > I have _not_ verified this, but we’ve had sporadic reports (Shawn?) of
> same. CommitTracker is the only place in the code that calls
> newScheduledThreadPool with 0.
> >
> > From what I can tell, this isn’t fixed in Java until 9.
> >
> > If the fix is just changing that call to use 1, it’s trivial but would
> require a respin of 7.7 to get out in the field.
> >
> > Should I raise a JIRA? ‘cause we should be seeing it in Solr 8 and
> master too. Although we should be seeing this in Solr 8 too, although it
> should disappear in Master when we require Java 11.
> >
> >
> >
> > ************E-mail below.
> > Hi,
> > Apologies, I can’t figure out how to reply to the Solr mailing list.
> > I just ran across the same high CPU usage issue. I believe it’’s caused
> by
> > this commit which was introduced in Solr 7.7.0
> >
> https://github.com/apache/lucene-solr/commit/eb652b84edf441d8369f5188cdd5e3ae2b151434#diff-e54b251d166135a1afb7938cfe152bb5
> > There is a bug in JDK versions <=8 where using 0 threads in the
> > ScheduledThreadPool causes high CPU usage:
> > https://bugs.openjdk.java.net/browse/JDK-8129861
> > Oddly, the latest version
> > of solr/core/src/java/org/apache/solr/update/CommitTracker.java on
> > master still uses 0 executors as the default. Presumably most everyone is
> > using JDK 9 or greater which has the bug fixed, so they don’t experience
> > the bug.
> > Feel free to relay this back to the mailing list.
> > Thanks,
> > Adam Guthrie
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: dev-help@lucene.apache.org
> >
>
>
> --
> Adrien
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>
Re: We have a cause for high CPU usage on Solr 7.7
Posted by Adrien Grand <jp...@gmail.com>.
+1 to raise a JIRA. There is an easy workaround on the user side by
using a recent JVM, but at the same time the change to the code looks
safe. Does it affect 6.6? I see there is an ongoing thread about
getting 6.6.6 out.
On Mon, Mar 25, 2019 at 5:05 PM Erick Erickson <er...@gmail.com> wrote:
>
> From the user’s list (thanks to Adam Guthrie and Lukas Weiss)
>
> Short form: Apparently there’s a java 8 bug that’s the root cause, not fixed until Java 9: https://bugs.openjdk.java.net/browse/JDK-8129861
>
> I have _not_ verified this, but we’ve had sporadic reports (Shawn?) of same. CommitTracker is the only place in the code that calls newScheduledThreadPool with 0.
>
> From what I can tell, this isn’t fixed in Java until 9.
>
> If the fix is just changing that call to use 1, it’s trivial but would require a respin of 7.7 to get out in the field.
>
> Should I raise a JIRA? ‘cause we should be seeing it in Solr 8 and master too. Although we should be seeing this in Solr 8 too, although it should disappear in Master when we require Java 11.
>
>
>
> ************E-mail below.
> Hi,
> Apologies, I can’t figure out how to reply to the Solr mailing list.
> I just ran across the same high CPU usage issue. I believe it’’s caused by
> this commit which was introduced in Solr 7.7.0
> https://github.com/apache/lucene-solr/commit/eb652b84edf441d8369f5188cdd5e3ae2b151434#diff-e54b251d166135a1afb7938cfe152bb5
> There is a bug in JDK versions <=8 where using 0 threads in the
> ScheduledThreadPool causes high CPU usage:
> https://bugs.openjdk.java.net/browse/JDK-8129861
> Oddly, the latest version
> of solr/core/src/java/org/apache/solr/update/CommitTracker.java on
> master still uses 0 executors as the default. Presumably most everyone is
> using JDK 9 or greater which has the bug fixed, so they don’t experience
> the bug.
> Feel free to relay this back to the mailing list.
> Thanks,
> Adam Guthrie
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
--
Adrien
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: We have a cause for high CPU usage on Solr 7.7
Posted by Bram Van Dam <br...@intix.eu>.
On 25/03/2019 17:04, Erick Erickson wrote:
> Short form: Apparently there’s a java 8 bug that’s the root cause, not fixed until Java 9: https://bugs.openjdk.java.net/browse/JDK-8129861
Thanks for this, Erick & Adam. This explains a few things I've been
seeing (in my own code as well as sporadically in Solr 7.7.0). Looks
like a couple of other users on the -user list reported similar issues.
It looks like there are already a couple of bugfixes scheduled for 7.7.2
anyway, so I guess someone was already planning a new release?
Anything I can do to help out?
- Bram