super-large sysout logs from tests?

[Dawid Weiss <da...@gmail.com>]

Yonik, I remember you had a problem with OOMs when the master was
buffering super large outputs from slave JVMs. I haven't worked on
this issue, but I had it somewhere in the back of my head. Is this
something that is important/ of priority? I don't see any tests
dumping a lot of output at the moment.

Dawid

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org

Re: super-large sysout logs from tests?

[Dawid Weiss <da...@cs.put.poznan.pl>]

> Yep, and I think Mark has had the same issue.
> IIRC, sometimes it was very difficult to tell - you would see an "JVM
> exited abnormally" message or something.

This shouldn't be the case any more with junit4; the slave JVMs are
always tracked until they really die and if they crash the output from
the crash will be emitted to the console/ output files (well, that's
what happens when I crash them via JNI in my tests). Feel free to try
on that fork on github -- if you let me know which test is causing
this and if there's any way I can reproduce it, I'll try to play
around.

>> something that is important/ of priority? I don't see any tests
>> dumping a lot of output at the moment.
>
> That's because by default the level is set to SEVERE now.
> The issue happens when we set that to INFO to help debug what's happening.

All the buffering is still in memory currently but I don't think it'll
be a big issue to convert it to spill to disk. One thing I noticed
(that was actually a cause of those gson errors) was that left-over
threads from zookeeper keep emitting sysouts long after their test
case is over; this does propagate to other test cases, unfortunately.
In the longer run I would like to implement a mechanism to prevent
this from happening, but at the moment it may look a bit odd in the
logs. The best temporary solution for this would be to try to join or
wait for any background threads, if at all possible.

Dawid

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org

Re: super-large sysout logs from tests?

[Yonik Seeley <yo...@lucidimagination.com>]

On Sun, Feb 19, 2012 at 4:44 PM, Dawid Weiss <da...@gmail.com> wrote:
> Yonik, I remember you had a problem with OOMs when the master was
> buffering super large outputs from slave JVMs.

Yep, and I think Mark has had the same issue.
IIRC, sometimes it was very difficult to tell - you would see an "JVM
exited abnormally" message or something.

> I haven't worked on
> this issue, but I had it somewhere in the back of my head. Is this
> something that is important/ of priority? I don't see any tests
> dumping a lot of output at the moment.

That's because by default the level is set to SEVERE now.
The issue happens when we set that to INFO to help debug what's happening.

-Yonik
lucidimagination.com

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org