Many threads started by Ehcache when running core-integ

[Stefan Seelmann <se...@apache.org>]

Hi guys,

I had (and still have) problems running the core-integ tests on a
Windows 7 VM. A first analysis using jvisualvm showed that for each
test two new threads were started from Ehcache and never stopped. The
cause is that for each new DirectoryService instance a new
CacheManager is created in CacheService.

To solve that problem I tried to to call CacheManager.shutdown() in
CacheService.destroy(). However then the DNFactory didn't work any
more because it uses a static reference to the cache and that could be
a shutted down instance.

So the next step was to transform the static DNFactory. I renamed
DNFactory to DefaultDNFactory, extracted an DNFactory interface and
removed some unused methods. The DNFactory is now an instance of the
directory service and managed by the directory service. A consequence
is that some test classes can't use the DNFactory any more because it
needs to be initialized by the DirectoryService.

In the end we still create a new CacheManager (which starts its two
threads) for each test but the threads are stopped after each test.

Kind Regards,
Stefan

Re: Many threads started by Ehcache when running core-integ

[Emmanuel Lecharny <el...@gmail.com>]

On 11/24/10 11:51 AM, Kiran Ayyagari wrote:
> On Wed, Nov 24, 2010 at 12:37 PM, Stefan Seelmann<se...@apache.org>  wrote:
>>> it would have been better if I have had checked it again, this is a
>>> mistake on my part
>>> Thanks for the insight and for fixing it Seelmann
>> It wasn't my intention to blame anyone, sorry if it appears so, my
>> bad. But it is much easier to find problems to point to them, then
>> building the perfect system :-)
>>
> Nope, no way I considered it as blaming, it needs to be fixed anyway
> and I remember placing
> a FIXME tag in the DNFactory code and I didn't look at the code afterwards.
> And yes +1 for building a perfect system and IMHO it is possible
> through open communication and
> thought process like this.
>
> I highly appreciate it Stefan :)
>
Guys, the server code is more than 370 000 SLOCs those days. There is 
*no way* we can not be blamed for what we added to the server in some 
area or another : the more code we have, the more mistakes we do. This 
is a simple arithmetic fact.

We do mistakes, some other fixes them, and this is how we are a good 
community :)

-- 
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com

Re: Many threads started by Ehcache when running core-integ

[Kiran Ayyagari <ka...@apache.org>]

On Wed, Nov 24, 2010 at 12:37 PM, Stefan Seelmann <se...@apache.org> wrote:
>> it would have been better if I have had checked it again, this is a
>> mistake on my part
>> Thanks for the insight and for fixing it Seelmann
>
> It wasn't my intention to blame anyone, sorry if it appears so, my
> bad. But it is much easier to find problems to point to them, then
> building the perfect system :-)
>
Nope, no way I considered it as blaming, it needs to be fixed anyway
and I remember placing
a FIXME tag in the DNFactory code and I didn't look at the code afterwards.
And yes +1 for building a perfect system and IMHO it is possible
through open communication and
thought process like this.

I highly appreciate it Stefan :)

-- 
Kiran Ayyagari

Re: Many threads started by Ehcache when running core-integ

[Stefan Seelmann <se...@apache.org>]

> it would have been better if I have had checked it again, this is a
> mistake on my part
> Thanks for the insight and for fixing it Seelmann

It wasn't my intention to blame anyone, sorry if it appears so, my
bad. But it is much easier to find problems to point to them, then
building the perfect system :-)

Kind Regards,
Stefan

Re: Many threads started by Ehcache when running core-integ

[Kiran Ayyagari <ka...@apache.org>]

On Wed, Nov 24, 2010 at 10:10 AM, Emmanuel Lecharny <el...@gmail.com> wrote:
> On 11/24/10 8:58 AM, Stefan Seelmann wrote:
>>
>> Hi guys,
>>
>> I had (and still have) problems running the core-integ tests on a
>> Windows 7 VM. A first analysis using jvisualvm showed that for each
>> test two new threads were started from Ehcache and never stopped. The
>> cause is that for each new DirectoryService instance a new
>> CacheManager is created in CacheService.
>
> Don't we shutdown the CacheManager when the DS is stopped ?
>>
>> To solve that problem I tried to to call CacheManager.shutdown() in
>> CacheService.destroy(). However then the DNFactory didn't work any
>> more because it uses a static reference to the cache and that could be
>> a shutted down instance.
>
> I guess you answered my question :) And this is bad design if the DNFactory
> uses a static cache reference.
>>
yeap, indeed, I have blindly made this static to be able to set it
from other tests
(wrongly presuming that we don't initialize this way from any other place)
>> So the next step was to transform the static DNFactory. I renamed
>> DNFactory to DefaultDNFactory, extracted an DNFactory interface and
>> removed some unused methods. The DNFactory is now an instance of the
>> directory service and managed by the directory service. A consequence
>> is that some test classes can't use the DNFactory any more because it
>> needs to be initialized by the DirectoryService.
>
> That's a good move; And regarding the DNFactory, we just have to define a
> mock DS to create the needed cacheManager.
>>
>> In the end we still create a new CacheManager (which starts its two
>> threads) for each test but the threads are stopped after each test.
>
> Great ! Thanks for having dealt with this issue.
>
> Designing an efficient cache system is not easy... I don't blame anyone for
> he current design, it's just that it's hard.
>
it would have been better if I have had checked it again, this is a
mistake on my part
Thanks for the insight and for fixing it Seelmann

-- 
Kiran Ayyagari

Re: Many threads started by Ehcache when running core-integ

[Emmanuel Lecharny <el...@gmail.com>]

On 11/24/10 8:58 AM, Stefan Seelmann wrote:
> Hi guys,
>
> I had (and still have) problems running the core-integ tests on a
> Windows 7 VM. A first analysis using jvisualvm showed that for each
> test two new threads were started from Ehcache and never stopped. The
> cause is that for each new DirectoryService instance a new
> CacheManager is created in CacheService.
Don't we shutdown the CacheManager when the DS is stopped ?
> To solve that problem I tried to to call CacheManager.shutdown() in
> CacheService.destroy(). However then the DNFactory didn't work any
> more because it uses a static reference to the cache and that could be
> a shutted down instance.
I guess you answered my question :) And this is bad design if the 
DNFactory uses a static cache reference.
> So the next step was to transform the static DNFactory. I renamed
> DNFactory to DefaultDNFactory, extracted an DNFactory interface and
> removed some unused methods. The DNFactory is now an instance of the
> directory service and managed by the directory service. A consequence
> is that some test classes can't use the DNFactory any more because it
> needs to be initialized by the DirectoryService.
That's a good move; And regarding the DNFactory, we just have to define 
a mock DS to create the needed cacheManager.
> In the end we still create a new CacheManager (which starts its two
> threads) for each test but the threads are stopped after each test.
Great ! Thanks for having dealt with this issue.

Designing an efficient cache system is not easy... I don't blame anyone 
for he current design, it's just that it's hard.


-- 
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com