You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Sami Siren <ss...@gmail.com> on 2012/06/29 09:53:32 UTC
Thread "leaking" in junit tests
Hi,
In 3.x when a Solr test case leaves threads running there's a warning
printed like this:
WARNING: test class left thread running:
Thread[MultiThreadedHttpConnectionManager cleanup,5,main]
I do not see such warnings in 4.x or trunk, how do I enable such warnings there?
--
Sami Siren
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Thread "leaking" in junit tests
Posted by Dawid Weiss <da...@cs.put.poznan.pl>.
> I absolutely agree, even though IMO warning is better than silence.
I plan to work on it soon, we've released Carrot2 and Lingo3G and
there's a chunk of free time for me to work on randomized testing.
> Anyway I got what I wanted: a failure in my test ;)
So maybe I'll hint you a bit more -- look at the log messages before
the test fails with a leaked thread. The runner will try to "probe"
the thread's stack at random intervals to give you an idea where it's
been before it was killed. The probing cuts off identical stack
portion so you can pretty much tell what a thread was doing.
/**
* The number of "probes" of the offending thread's stack trace
before an attempt
* is made to kill it. Snapshots are taken at random intervals
between 10 and 100
* milliseconds each and dumped to system logger to facilitate
debugging of the offending thread.
*
* <p>Setting this value to 0 means no samples will be taken.
*/
int stackSamples() default 10;
Dawid
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Thread "leaking" in junit tests
Posted by Sami Siren <ss...@gmail.com>.
On Fri, Jun 29, 2012 at 12:01 PM, Dawid Weiss
<da...@cs.put.poznan.pl> wrote:
> My pleasure. Dealing with leaked threads is a tough area and it needs
> to be addressed at some point sooner or later. The warnings are not
> enough because nobody looks at them (for example in jenkins runs).
> Tests (suites) simply shouldn't leak threads, period (yes, it is my
> idee fixe to have this done ;).
I absolutely agree, even though IMO warning is better than silence.
Anyway I got what I wanted: a failure in my test ;)
--
Sami Siren
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Thread "leaking" in junit tests
Posted by Dawid Weiss <da...@cs.put.poznan.pl>.
My pleasure. Dealing with leaked threads is a tough area and it needs
to be addressed at some point sooner or later. The warnings are not
enough because nobody looks at them (for example in jenkins runs).
Tests (suites) simply shouldn't leak threads, period (yes, it is my
idee fixe to have this done ;).
Dawid
On Fri, Jun 29, 2012 at 10:56 AM, Sami Siren <ss...@gmail.com> wrote:
> Thanks for the detailed explanation Dawid!
>
> --
> Sami Siren
>
>
> On Fri, Jun 29, 2012 at 11:14 AM, Dawid Weiss
> <da...@cs.put.poznan.pl> wrote:
>>> WARNING: test class left thread running:
>>> Thread[MultiThreadedHttpConnectionManager cleanup,5,main]
>>>
>>> I do not see such warnings in 4.x or trunk, how do I enable such warnings there?
>>
>> Open LuceneTestCase and change this attribute to true:
>>
>> @ThreadLeaks(failTestIfLeaking = false)
>>
>> this will effectively kill those leaked threads and fail the test that
>> leaked them. Unfortunately the scenarios here are complex because
>> threads are started within a test scope and belong to suite scope (are
>> reused by all threads in a suite). Previously, the warnings were
>> detected for the suite scope only which would correspond to:
>>
>> @ThreadLeaks(failTestIfLeaking = true, leakedThreadsBelongToSuite =
>> true, linger = 1000)
>>
>> meaning:
>>
>> - fail a suite if threads leaked from it,
>> - if a thread leak occurs within a test, promote it to the suite scope,
>> - linger 1000 milliseconds before deciding a thread has leaked (this
>> is required for thread pools and such because they never wait for
>> their children threads to actually complete).
>>
>> Many Solr and a few Lucene tests currently leak threads so the flag is
>> off. There is an issue to improve reporting of this (and handling of
>> this) here:
>>
>> https://issues.apache.org/jira/browse/LUCENE-3985
>>
>> I just didn't get to it yet. Or rather: I did, but it requires changes
>> to the runner -- see here:
>> https://github.com/carrotsearch/randomizedtesting/issues/98
>>
>> Dawid
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Thread "leaking" in junit tests
Posted by Sami Siren <ss...@gmail.com>.
Thanks for the detailed explanation Dawid!
--
Sami Siren
On Fri, Jun 29, 2012 at 11:14 AM, Dawid Weiss
<da...@cs.put.poznan.pl> wrote:
>> WARNING: test class left thread running:
>> Thread[MultiThreadedHttpConnectionManager cleanup,5,main]
>>
>> I do not see such warnings in 4.x or trunk, how do I enable such warnings there?
>
> Open LuceneTestCase and change this attribute to true:
>
> @ThreadLeaks(failTestIfLeaking = false)
>
> this will effectively kill those leaked threads and fail the test that
> leaked them. Unfortunately the scenarios here are complex because
> threads are started within a test scope and belong to suite scope (are
> reused by all threads in a suite). Previously, the warnings were
> detected for the suite scope only which would correspond to:
>
> @ThreadLeaks(failTestIfLeaking = true, leakedThreadsBelongToSuite =
> true, linger = 1000)
>
> meaning:
>
> - fail a suite if threads leaked from it,
> - if a thread leak occurs within a test, promote it to the suite scope,
> - linger 1000 milliseconds before deciding a thread has leaked (this
> is required for thread pools and such because they never wait for
> their children threads to actually complete).
>
> Many Solr and a few Lucene tests currently leak threads so the flag is
> off. There is an issue to improve reporting of this (and handling of
> this) here:
>
> https://issues.apache.org/jira/browse/LUCENE-3985
>
> I just didn't get to it yet. Or rather: I did, but it requires changes
> to the runner -- see here:
> https://github.com/carrotsearch/randomizedtesting/issues/98
>
> Dawid
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Thread "leaking" in junit tests
Posted by Dawid Weiss <da...@cs.put.poznan.pl>.
> WARNING: test class left thread running:
> Thread[MultiThreadedHttpConnectionManager cleanup,5,main]
>
> I do not see such warnings in 4.x or trunk, how do I enable such warnings there?
Open LuceneTestCase and change this attribute to true:
@ThreadLeaks(failTestIfLeaking = false)
this will effectively kill those leaked threads and fail the test that
leaked them. Unfortunately the scenarios here are complex because
threads are started within a test scope and belong to suite scope (are
reused by all threads in a suite). Previously, the warnings were
detected for the suite scope only which would correspond to:
@ThreadLeaks(failTestIfLeaking = true, leakedThreadsBelongToSuite =
true, linger = 1000)
meaning:
- fail a suite if threads leaked from it,
- if a thread leak occurs within a test, promote it to the suite scope,
- linger 1000 milliseconds before deciding a thread has leaked (this
is required for thread pools and such because they never wait for
their children threads to actually complete).
Many Solr and a few Lucene tests currently leak threads so the flag is
off. There is an issue to improve reporting of this (and handling of
this) here:
https://issues.apache.org/jira/browse/LUCENE-3985
I just didn't get to it yet. Or rather: I did, but it requires changes
to the runner -- see here:
https://github.com/carrotsearch/randomizedtesting/issues/98
Dawid
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org