You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@maven.apache.org by Dawid Weiss <da...@gmail.com> on 2013/01/02 09:33:20 UTC
Surefire and low memory conditions
Hi. Just wanted to ping you because I remember we talked about
hardnuts for test runners (like surefire). I've been recently trying
to think of a way to shutdown the forked JVM cleanly on permgen/ OOM
conditions (and signal it back as such to the controlling process).
Seems to be quite hard because various JVMs appear to go crazy
themselves when running under low memory.
Anyway, out of curiousity I checked with surefire and it seems to hang
for me (Windows, 8 core cpu). The code is here.
https://github.com/dweiss/surefire-hardtests
The single-threaded version says the forked VM shut down without
saying good bye which seems like a fine resolution to me. This is
hardly a "bug" in surefire -- like I mentioned, running anything with
exhausted permgen or heap is very unpredictable in Java. Still, I'm
curious if you see any workarounds or solutions for the issue.
Dawid
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: Surefire and low memory conditions
Posted by Dawid Weiss <da...@gmail.com>.
Thanks Martin. My question was in fact not user-based but
developer-based. I'm developing an alternative JUnit runner for
Lucene/Solr builds and we hit lots of exceptional situations. Permgen
errors are kind of hard to deal with because after you hit it there
are very few recovery options (short of preloading all runner classes
in advance).
Dawid
On Wed, Jan 2, 2013 at 2:58 PM, Martin Gainty <mg...@hotmail.com> wrote:
>
> David
>
> I was plagued by the same problem yesterday until I shut off forkMode in maven-surefire-plugin e.g.
> <build><plugins> <plugin> <artifactId>maven-surefire-plugin</artifactId>
> <configuration>
> <forkMode>never</forkMode></configuration> I also upped the MaxHeap and PermGen params in _JAVA_OPTIONS environment variable e.g.
> _JAVA_OPTIONS: -Xmx3192m -XX:MaxPermSize=3192m
> Viel Gluck,
> Martin
> ______________________________________________
> Verzicht und Vertraulichkeitanmerkung
>
> Diese Nachricht ist vertraulich. Sollten Sie nicht der vorgesehene Empfaenger sein, so bitten wir hoeflich um eine Mitteilung. Jede unbefugte Weiterleitung oder Fertigung einer Kopie ist unzulaessig. Diese Nachricht dient lediglich dem Austausch von Informationen und entfaltet keine rechtliche Bindungswirkung. Aufgrund der leichten Manipulierbarkeit von E-Mails koennen wir keine Haftung fuer den Inhalt uebernehmen.
> > From: dawid.weiss@gmail.com
>> Date: Wed, 2 Jan 2013 09:33:20 +0100
>> Subject: Surefire and low memory conditions
>> To: dev@maven.apache.org
>>
>> Hi. Just wanted to ping you because I remember we talked about
>> hardnuts for test runners (like surefire). I've been recently trying
>> to think of a way to shutdown the forked JVM cleanly on permgen/ OOM
>> conditions (and signal it back as such to the controlling process).
>> Seems to be quite hard because various JVMs appear to go crazy
>> themselves when running under low memory.
>>
>> Anyway, out of curiousity I checked with surefire and it seems to hang
>> for me (Windows, 8 core cpu). The code is here.
>> https://github.com/dweiss/surefire-hardtests
>>
>> The single-threaded version says the forked VM shut down without
>> saying good bye which seems like a fine resolution to me. This is
>> hardly a "bug" in surefire -- like I mentioned, running anything with
>> exhausted permgen or heap is very unpredictable in Java. Still, I'm
>> curious if you see any workarounds or solutions for the issue.
>>
>> Dawid
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
>> For additional commands, e-mail: dev-help@maven.apache.org
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: Surefire and low memory conditions
Posted by Andreas Gudian <an...@gmail.com>.
Martin,
you can also use the argLine property in the Surefire configuration to
pass appropriate -X* settings to the forked processes. That way you can use
any fork mode you might want and not have to pollute the general java opts
and, more importantly, the memory realm of a multi-module build (if that is
your use case).
Brgds,
Andreas
Am Mittwoch, 2. Januar 2013 schrieb Martin Gainty :
>
> David
>
> I was plagued by the same problem yesterday until I shut off forkMode in
> maven-surefire-plugin e.g.
> <build><plugins> <plugin> <artifactId>maven-surefire-plugin</artifactId>
> <configuration>
> <forkMode>never</forkMode></configuration> I also upped the MaxHeap and
> PermGen params in _JAVA_OPTIONS environment variable e.g.
> _JAVA_OPTIONS: -Xmx3192m -XX:MaxPermSize=3192m
> Viel Gluck,
> Martin
> ______________________________________________
> Verzicht und Vertraulichkeitanmerkung
>
> Diese Nachricht ist vertraulich. Sollten Sie nicht der vorgesehene
> Empfaenger sein, so bitten wir hoeflich um eine Mitteilung. Jede unbefugte
> Weiterleitung oder Fertigung einer Kopie ist unzulaessig. Diese Nachricht
> dient lediglich dem Austausch von Informationen und entfaltet keine
> rechtliche Bindungswirkung. Aufgrund der leichten Manipulierbarkeit von
> E-Mails koennen wir keine Haftung fuer den Inhalt uebernehmen.
> > From: dawid.weiss@gmail.com <javascript:;>
> > Date: Wed, 2 Jan 2013 09:33:20 +0100
> > Subject: Surefire and low memory conditions
> > To: dev@maven.apache.org <javascript:;>
> >
> > Hi. Just wanted to ping you because I remember we talked about
> > hardnuts for test runners (like surefire). I've been recently trying
> > to think of a way to shutdown the forked JVM cleanly on permgen/ OOM
> > conditions (and signal it back as such to the controlling process).
> > Seems to be quite hard because various JVMs appear to go crazy
> > themselves when running under low memory.
> >
> > Anyway, out of curiousity I checked with surefire and it seems to hang
> > for me (Windows, 8 core cpu). The code is here.
> > https://github.com/dweiss/surefire-hardtests
> >
> > The single-threaded version says the forked VM shut down without
> > saying good bye which seems like a fine resolution to me. This is
> > hardly a "bug" in surefire -- like I mentioned, running anything with
> > exhausted permgen or heap is very unpredictable in Java. Still, I'm
> > curious if you see any workarounds or solutions for the issue.
> >
> > Dawid
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org <javascript:;>
> > For additional commands, e-mail: dev-help@maven.apache.org<javascript:;>
> >
>
RE: Surefire and low memory conditions
Posted by Martin Gainty <mg...@hotmail.com>.
David
I was plagued by the same problem yesterday until I shut off forkMode in maven-surefire-plugin e.g.
<build><plugins> <plugin> <artifactId>maven-surefire-plugin</artifactId>
<configuration>
<forkMode>never</forkMode></configuration> I also upped the MaxHeap and PermGen params in _JAVA_OPTIONS environment variable e.g.
_JAVA_OPTIONS: -Xmx3192m -XX:MaxPermSize=3192m
Viel Gluck,
Martin
______________________________________________
Verzicht und Vertraulichkeitanmerkung
Diese Nachricht ist vertraulich. Sollten Sie nicht der vorgesehene Empfaenger sein, so bitten wir hoeflich um eine Mitteilung. Jede unbefugte Weiterleitung oder Fertigung einer Kopie ist unzulaessig. Diese Nachricht dient lediglich dem Austausch von Informationen und entfaltet keine rechtliche Bindungswirkung. Aufgrund der leichten Manipulierbarkeit von E-Mails koennen wir keine Haftung fuer den Inhalt uebernehmen.
> From: dawid.weiss@gmail.com
> Date: Wed, 2 Jan 2013 09:33:20 +0100
> Subject: Surefire and low memory conditions
> To: dev@maven.apache.org
>
> Hi. Just wanted to ping you because I remember we talked about
> hardnuts for test runners (like surefire). I've been recently trying
> to think of a way to shutdown the forked JVM cleanly on permgen/ OOM
> conditions (and signal it back as such to the controlling process).
> Seems to be quite hard because various JVMs appear to go crazy
> themselves when running under low memory.
>
> Anyway, out of curiousity I checked with surefire and it seems to hang
> for me (Windows, 8 core cpu). The code is here.
> https://github.com/dweiss/surefire-hardtests
>
> The single-threaded version says the forked VM shut down without
> saying good bye which seems like a fine resolution to me. This is
> hardly a "bug" in surefire -- like I mentioned, running anything with
> exhausted permgen or heap is very unpredictable in Java. Still, I'm
> curious if you see any workarounds or solutions for the issue.
>
> Dawid
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
Re: Surefire and low memory conditions
Posted by Dawid Weiss <da...@gmail.com>.
> It would be quite simple to have a thread in the forked process
> simply pulsing a heartbeat back to the plugin ?
>
> And if 3 heartbeats are missed, we simply kill it ? (Pardon the pun ;)
Yeah, I sort of have that already but not in a background thread form
(it's a longer story). Anyway, a background thread with heartbeats is
still going to work even under OOM conditions as long as you don't
load new classes or allocate new resources. This background thread
would need to actively check for those low-memory conditions either
using crude methods (like trying to allocate an array of bytes or load
a dummy class) or some higher-order logic (inspect memory pool
statistics returned from management beans). The latter, while more
appealing, has the disadvantage that it may try to allocate more
memory (or load unseen classes) in which case it'll fail unexpectedly
too and these failures are very hard to anticipate.
Also, I was kind of concerned about having an active background thread
that may interfere with tests somehow (God knows what people but in
their tests). I guess you're right though -- there hardly seems to be
any other way.
Dawid
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: Surefire and low memory conditions
Posted by Kristian Rosenvold <kr...@gmail.com>.
It would be quite simple to have a thread in the forked process
simply pulsing a heartbeat back to the plugin ?
And if 3 heartbeats are missed, we simply kill it ? (Pardon the pun ;)
Kristian
2013/1/2 Dawid Weiss <da...@gmail.com>:
> Hi. Just wanted to ping you because I remember we talked about
> hardnuts for test runners (like surefire). I've been recently trying
> to think of a way to shutdown the forked JVM cleanly on permgen/ OOM
> conditions (and signal it back as such to the controlling process).
> Seems to be quite hard because various JVMs appear to go crazy
> themselves when running under low memory.
>
> Anyway, out of curiousity I checked with surefire and it seems to hang
> for me (Windows, 8 core cpu). The code is here.
> https://github.com/dweiss/surefire-hardtests
>
> The single-threaded version says the forked VM shut down without
> saying good bye which seems like a fine resolution to me. This is
> hardly a "bug" in surefire -- like I mentioned, running anything with
> exhausted permgen or heap is very unpredictable in Java. Still, I'm
> curious if you see any workarounds or solutions for the issue.
>
> Dawid
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org