SimpleServiceManager

[Andy Gumbrecht <an...@orprovision.com>]

Can a committer please check and apply this patch to trunk.

https://issues.apache.org/jira/browse/OPENEJB-1249

Thanks,
Andy.

Re: [english 100%] Re: SimpleServiceManager

[David Blevins <da...@visi.com>]

On Apr 20, 2010, at 3:23 PM, Andy wrote:

> On 20.04.2010 22:57, David Blevins wrote:
>>
>> On Apr 20, 2010, at 1:20 AM, Andy Gumbrecht wrote:
>>
>>> Can a committer please check and apply this patch to trunk.
>>>
>>> https://issues.apache.org/jira/browse/OPENEJB-1249
>>>
>>
>> Hey Andy!
>>
>> We have that same code for shutting down the RAs in the  
>> Assembler.destroy() method (which is called from  
>> OpenEJB.destroy()).  I tried a similar patch but ran into the  
>> ActiveMQ Resource Adapter hang I posted about.  Maybe you might  
>> have better luck.  I've attached it to the JIRA.
>>
>>  https://issues.apache.org/jira/secure/attachment/12442347/Server.java.patch
>>
>> If that works for you I'll go ahead and commit it.
>>
>> -David
>>
>>
> Hi David,
>
> I have been using the posted 'stop' patch in a pre-production, but  
> very heavily tested, environment. I have had no obvious issues with  
> a remote shutdown of the ActiveMQ RA, but will now have a deeper  
> look to make sure I am not hiding anything.
>
> I have been playing around with the current ActiveMQ RA start/stop  
> code, so maybe I have a fix even though I can't reproduce a hang.  
> I'll post a patch for review when it's up to scratch.
>
> However, this story may help in the hunt for the hang. The Quartz RA  
> had proved to be the real show stopper for me - though this is not  
> an issue I can really confirm yet, as I cannot find the cause. I  
> really thought it was an ActiveMQ hang, and struggled for several  
> days on dumps that 'always' pointed to ActiveMQ code. Eventually I  
> found that if I cut out ActiveMQ then I'd get a startup, but then  
> deploying apps using my custom deployer bean (based on Deployer) was  
> hanging during the deploy at various locations, but this time nearly  
> always in Derby code during persistence unit creation. I was pulling  
> my hair out until for some reason I pulled out the Quartz RA - and  
> everything started working.
>
> I never once saw anything to do with Quartz in thread dumps that  
> could/should be anything to do with a hang, but I can reproduce 'my'  
> ActiveMQ/Derby/Deployer hang every time just by dropping Quartz into  
> the brew. I am now fairly sure this is a RA class loading issue, but  
> it currently beats me as to where and how to pin this down - it is  
> one of the strangest hangs I have ever had. Nothing locked or  
> blocked, but most definitely waiting on 'something' at some point  
> after the Quartz RA start, and always near or on a Class.forName  
> call in either ActiveMQ or Derby.

Very interesting.  And very glad to hear you've been looking into it.   
It's a tough one.

Maybe there's something we need to fix on the Quartz RA side.  Tricky  
thing is in EJB 3.1 we'll be using Quartz heavily to support the new  
@Schedule annotation, so if there are issues, they've just started.

Have you noticed any of this on Java 6?

> I have been tinkering with the the TempClassLoader, but to no avail  
> - although as a nice side effect I found that reusing the 'peek'  
> ByteArrayStream (the method is synchronized anyway) much reduced  
> memory peaks during class loading, I'll post that for review soon.

Nice!

> I'll annoy you some more later.

Looking forward to it :)

> Have a nice day.

You too.

-David

Re: [english 100%] Re: SimpleServiceManager

[Andy <an...@orprovision.com>]

On 20.04.2010 22:57, David Blevins wrote:
>
> On Apr 20, 2010, at 1:20 AM, Andy Gumbrecht wrote:
>
>> Can a committer please check and apply this patch to trunk.
>>
>> https://issues.apache.org/jira/browse/OPENEJB-1249
>>
>
> Hey Andy!
>
> We have that same code for shutting down the RAs in the 
> Assembler.destroy() method (which is called from OpenEJB.destroy()).  
> I tried a similar patch but ran into the ActiveMQ Resource Adapter 
> hang I posted about.  Maybe you might have better luck.  I've attached 
> it to the JIRA.
>
>   
> https://issues.apache.org/jira/secure/attachment/12442347/Server.java.patch 
>
>
> If that works for you I'll go ahead and commit it.
>
> -David
>
>
Hi David,

I have been using the posted 'stop' patch in a pre-production, but very 
heavily tested, environment. I have had no obvious issues with a remote 
shutdown of the ActiveMQ RA, but will now have a deeper look to make 
sure I am not hiding anything.

I have been playing around with the current ActiveMQ RA start/stop code, 
so maybe I have a fix even though I can't reproduce a hang. I'll post a 
patch for review when it's up to scratch.

However, this story may help in the hunt for the hang. The Quartz RA had 
proved to be the real show stopper for me - though this is not an issue 
I can really confirm yet, as I cannot find the cause. I really thought 
it was an ActiveMQ hang, and struggled for several days on dumps that 
'always' pointed to ActiveMQ code. Eventually I found that if I cut out 
ActiveMQ then I'd get a startup, but then deploying apps using my custom 
deployer bean (based on Deployer) was hanging during the deploy at 
various locations, but this time nearly always in Derby code during 
persistence unit creation. I was pulling my hair out until for some 
reason I pulled out the Quartz RA - and everything started working.

I never once saw anything to do with Quartz in thread dumps that 
could/should be anything to do with a hang, but I can reproduce 'my' 
ActiveMQ/Derby/Deployer hang every time just by dropping Quartz into the 
brew. I am now fairly sure this is a RA class loading issue, but it 
currently beats me as to where and how to pin this down - it is one of 
the strangest hangs I have ever had. Nothing locked or blocked, but most 
definitely waiting on 'something' at some point after the Quartz RA 
start, and always near or on a Class.forName call in either ActiveMQ or 
Derby.

I have been tinkering with the the TempClassLoader, but to no avail - 
although as a nice side effect I found that reusing the 'peek' 
ByteArrayStream (the method is synchronized anyway) much reduced memory 
peaks during class loading, I'll post that for review soon.

I'll annoy you some more later. Have a nice day.

Andy.

____________
Virus checked by G Data AntiVirus
Version: AVA 19.11014 dated 20.04.2010

Re: SimpleServiceManager

[David Blevins <da...@visi.com>]

On Apr 20, 2010, at 1:20 AM, Andy Gumbrecht wrote:

> Can a committer please check and apply this patch to trunk.
>
> https://issues.apache.org/jira/browse/OPENEJB-1249
>

Hey Andy!

We have that same code for shutting down the RAs in the  
Assembler.destroy() method (which is called from OpenEJB.destroy()).   
I tried a similar patch but ran into the ActiveMQ Resource Adapter  
hang I posted about.  Maybe you might have better luck.  I've attached  
it to the JIRA.

   https://issues.apache.org/jira/secure/attachment/12442347/Server.java.patch

If that works for you I'll go ahead and commit it.

-David