You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by lzr <js...@163.com> on 2013/07/16 13:23:26 UTC
ActiveMQ client app hold unexpectedly!!!
Dear all,
I found a one of the ActiveMQ thread got stuck unexpectedly for several days. Customer was very angry to the ActiveMQ for the bad behavior. Did anybody experienced this issue? It would be much appreciated if any clue!!!
My application is normal Java JMS application like the following:
new ActiveMQConnectionFactory.
create Connection;
conn.start.
create session.
create destination, producer and message
send messages
commit;
conn.close.
The follows is the stack when I looked into it with debug tools:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:169)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1787)
java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:356)
org.apache.activemq.transport.FutureResponse.getResult(FutureResponse.java:40)
org.apache.activemq.transport.ResponseCorrelator.request(ResponseCorrelator.java:80)
org.apache.activemq.ActiveMQConnection.syncSendPacket(ActiveMQConnection.java:1233)
org.apache.activemq.ActiveMQConnection.ensureConnectionInfoSent(ActiveMQConnection.java:1339)
org.apache.activemq.ActiveMQConnection.start(ActiveMQConnection.java:488)
nc.bs.mqv2.mmgp.mq.UFMQProcessor.getConnection(UFMQProcessor.java:487)
nc.bs.mqv2.mmgp.mq.UFMQProcessor.getConnection(UFMQProcessor.java:462)
nc.bs.mqv2.mmgp.mq.UFMQProcessor.sendSerializedObjMsg(UFMQProcessor.java:128)
nc.bs.mqv2.meam.mq.sendbasedatas.AbstractMeamBDSyncService.sendBaseData(AbstractMeamBDSyncService.java:64)
nc.bs.mqv2.mes.bd.mq.basedoc.AbstractBDServerSendService.dosendDataProcess(AbstractBDServerSendService.java:213)
nc.bs.mqv2.mes.bd.mq.basedoc.AbstractBDServerSendService.sendDatas(AbstractBDServerSendService.java:62)
nc.impl.mqv2.mes.bd.mq.SyncSendServiceImpl.ncserverSendDatas(SyncSendServiceImpl.java:58)
nc.gejb.meam.cmt.MeamEjbBean.ncserverSendDatas(MeamEjbBean.java:8741)
nc.gejb.meam.cmt.EJSLocalStatelessnc_gejb_meam_cmt_Meam_b44269e7.ncserverSendDatas(Unknown Source)
sun.reflect.GeneratedMethodAccessor447.invoke(Unknown Source)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.lang.reflect.Method.invoke(Method.java:615)
nc.bs.framework.ejb.AbstractEJBServiceHandler.invoke(AbstractEJBServiceHandler.java:56)
nc.bs.framework.ejb.AbstractEJBServiceHandler.invoke(AbstractEJBServiceHandler.java:34)
$Proxy58.ncserverSendDatas(Unknown Source)
nc.bs.mqv2.meam.mq.task.DoSendBaseDataTaskUtil.runTask(DoSendBaseDataTaskUtil.java:63)
nc.bs.mqv2.meam.mq.task.NCServerBaseDatasSendTask.executeTask(NCServerBaseDatasSendTask.java:26)
nc.bs.pub.taskcenter.BgBizsPluginProcess.callBusinessPlugin(BgBizsPluginProcess.java:43)
nc.bs.pub.pa.read.TaskBodyBase.callBizsPlugin(TaskBodyBase.java:188)
nc.bs.pub.pa.read.TaskBodyBase.callBusinessPlugin(TaskBodyBase.java:174)
nc.bs.pub.pa.read.TaskBodyBase.createMetaMsg(TaskBodyBase.java:95)
nc.bs.pub.pa.read.TaskBodyBase.createAndSendMessage(TaskBodyBase.java:41)
nc.bs.pub.taskcenter.BackgroundTaskBody.execute(BackgroundTaskBody.java:47)
nc.impl.uap.scheduler.ExecutorServiceImpl.executeTask_RequiresNew(ExecutorServiceImpl.java:27)
nc.itf.uap.mw.MwBizEJBEjbBean.executeTask_RequiresNew(MwBizEJBEjbBean.java:1682)
nc.itf.uap.mw.EJSLocalStatelessnc_itf_uap_mw_MwBizEJB_f98e92a8.executeTask_RequiresNew(Unknown Source)
sun.reflect.GeneratedMethodAccessor1036.invoke(Unknown Source)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.lang.reflect.Method.invoke(Method.java:615)
nc.bs.framework.ejb.AbstractEJBServiceHandler.invoke(AbstractEJBServiceHandler.java:56)
nc.bs.framework.ejb.AbstractEJBServiceHandler.invoke(AbstractEJBServiceHandler.java:34)
$Proxy39.executeTask_RequiresNew(Unknown Source)
nc.bs.uap.scheduler.impl.TaskExecutor.executeTask(TaskExecutor.java:110)
nc.bs.uap.scheduler.impl.TaskExecutor.run(TaskExecutor.java:75)
nc.bs.framework.execute.Executor$1.run(Executor.java:80)
nc.bs.framework.execute.commonj.impl.WorkAdaptor.run(WorkAdaptor.java:29)
com.ibm.ws.asynchbeans.J2EEContext.run(J2EEContext.java:1114)
com.ibm.ws.asynchbeans.WorkWithExecutionContextImpl.go(WorkWithExecutionContextImpl.java:195)
com.ibm.ws.asynchbeans.CJWorkItemImpl.run(CJWorkItemImpl.java:187)
com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1469)
Best regards,
Zhuran Li
Re:Re:Re: Re: Re: ActiveMQ client app hold unexpectedly!!!
Posted by SuoNayi <su...@163.com>.
If you use failover transport, there is an timeout option that can be enabled to address the issue.
see http://activemq.apache.org/failover-transport-reference.html
At 2013-07-16 20:32:54,lzr <js...@163.com> wrote:
>I found the issue just this time. It worked fine for about two month with the ActiveMQ M/S cluster.
>I've thought of the network glitch; What I feel strange most is there is no any error/exception on both the ActiveMQ client side and server side.
>I went through the related codes and found there is a FutureResponse.getResult method with parameter 'timeout'. Maybe this method is more suitable for getting the result. Not very sure, hehe.
>
>
>
>
>
>
>
>
>At 2013-07-16 20:23:01,"Christian Posta" <ch...@gmail.com> wrote:
>>Is this something that happens all the time? Maybe your network had a
>>glitch and didn't deliver the response back to the client.
>>
>>
>>On Tue, Jul 16, 2013 at 8:18 AM, lzr <js...@163.com> wrote:
>>
>>> I just check the configuration file, actually I've disabled the flow
>>> control:
>>>
>>>
>>> <destinationPolicy>
>>> <policyMap>
>>> <policyEntries>
>>> <policyEntry topic=">" producerFlowControl="false"
>>> memoryLimit="100mb">
>>> <pendingSubscriberPolicy>
>>> <vmCursor />
>>> </pendingSubscriberPolicy>
>>> </policyEntry>
>>> <policyEntry queue=">" producerFlowControl="false"
>>> memoryLimit="100mb">
>>> <!-- Use VM cursor for better latency
>>> For more information, see:
>>>
>>>
>>> http://activemq.apache.org/message-cursors.html
>>>
>>>
>>> <pendingQueuePolicy>
>>> <vmQueueCursor/>
>>> </pendingQueuePolicy>
>>> -->
>>> </policyEntry>
>>> </policyEntries>
>>> </policyMap>
>>> </destinationPolicy>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> At 2013-07-16 20:15:11,"Christian Posta" <ch...@gmail.com>
>>> wrote:
>>> >Yes, on the first time PFC kicked in you would see the log messages. Not
>>> on
>>> >subsequent times.
>>> >
>>> >
>>> >On Tue, Jul 16, 2013 at 8:01 AM, lzr <js...@163.com> wrote:
>>> >
>>> >> Based on my experiences if the issue caused by the producer flow control
>>> >> configuration, the logs will have some warning messages, right? In fact
>>> I
>>> >> didn't found any warning or errors in the activemq.log.
>>> >> Anyway, I'll check the configuration files right now.
>>> >>
>>> >>
>>> >> thanks lot,
>>> >> Zhuran.
>>> >>
>>> >>
>>> >> At 2013-07-16 19:36:44,"Christian Posta" <ch...@gmail.com>
>>> >> wrote:
>>> >>
>>> >> >This means the client didn't' receive a response from the broker.
>>> >> >Did the broker producer flow control this client? ie, was the
>>> destination
>>> >> >to which the client sending full?
>>> >> >
>>> >> >
>>> >> >On Tue, Jul 16, 2013 at 7:23 AM, lzr <js...@163.com> wrote:
>>> >> >
>>> >> >> Dear all,
>>> >> >>
>>> >> >>
>>> >> >> I found a one of the ActiveMQ thread got stuck unexpectedly for
>>> several
>>> >> >> days. Customer was very angry to the ActiveMQ for the bad behavior.
>>> Did
>>> >> >> anybody experienced this issue? It would be much appreciated if any
>>> >> clue!!!
>>> >> >> My application is normal Java JMS application like the following:
>>> >> >> new ActiveMQConnectionFactory.
>>> >> >> create Connection;
>>> >> >> conn.start.
>>> >> >> create session.
>>> >> >> create destination, producer and message
>>> >> >> send messages
>>> >> >> commit;
>>> >> >> conn.close.
>>> >> >>
>>> >> >>
>>> >> >> The follows is the stack when I looked into it with debug tools:
>>> >> >>
>>> >> >> sun.misc.Unsafe.park(Native Method)
>>> >> >>
>>> >> >> java.util.concurrent.locks.LockSupport.park(LockSupport.java:169)
>>> >> >>
>>> >> >>
>>> >> >>
>>> >>
>>> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1787)
>>> >> >>
>>> >> >>
>>> >>
>>> java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:356)
>>> >> >>
>>> >> >>
>>> >> >>
>>> >>
>>> org.apache.activemq.transport.FutureResponse.getResult(FutureResponse.java:40)
>>> >> >>
>>> >> >>
>>> >> >>
>>> >>
>>> org.apache.activemq.transport.ResponseCorrelator.request(ResponseCorrelator.java:80)
>>> >> >>
>>> >> >>
>>> >> >>
>>> >>
>>> org.apache.activemq.ActiveMQConnection.syncSendPacket(ActiveMQConnection.java:1233)
>>> >> >>
>>> >> >>
>>> >> >>
>>> >>
>>> org.apache.activemq.ActiveMQConnection.ensureConnectionInfoSent(ActiveMQConnection.java:1339)
>>> >> >>
>>> >> >>
>>> >>
>>> org.apache.activemq.ActiveMQConnection.start(ActiveMQConnection.java:488)
>>> >> >>
>>> >> >>
>>> nc.bs.mqv2.mmgp.mq.UFMQProcessor.getConnection(UFMQProcessor.java:487)
>>> >> >>
>>> >> >>
>>> nc.bs.mqv2.mmgp.mq.UFMQProcessor.getConnection(UFMQProcessor.java:462)
>>> >> >>
>>> >> >>
>>> >> >>
>>> >>
>>> nc.bs.mqv2.mmgp.mq.UFMQProcessor.sendSerializedObjMsg(UFMQProcessor.java:128)
>>> >> >>
>>> >> >>
>>> >> >>
>>> >>
>>> nc.bs.mqv2.meam.mq.sendbasedatas.AbstractMeamBDSyncService.sendBaseData(AbstractMeamBDSyncService.java:64)
>>> >> >>
>>> >> >>
>>> >> >>
>>> >>
>>> nc.bs.mqv2.mes.bd.mq.basedoc.AbstractBDServerSendService.dosendDataProcess(AbstractBDServerSendService.java:213)
>>> >> >>
>>> >> >>
>>> >> >>
>>> >>
>>> nc.bs.mqv2.mes.bd.mq.basedoc.AbstractBDServerSendService.sendDatas(AbstractBDServerSendService.java:62)
>>> >> >>
>>> >> >>
>>> >> >>
>>> >>
>>> nc.impl.mqv2.mes.bd.mq.SyncSendServiceImpl.ncserverSendDatas(SyncSendServiceImpl.java:58)
>>> >> >>
>>> >> >> nc.gejb.meam.cmt.MeamEjbBean.ncserverSendDatas(MeamEjbBean.java:8741)
>>> >> >>
>>> >> >>
>>> >>
>>> nc.gejb.meam.cmt.EJSLocalStatelessnc_gejb_meam_cmt_Meam_b44269e7.ncserverSendDatas(Unknown
>>> >> >> Source)
>>> >> >>
>>> >> >> sun.reflect.GeneratedMethodAccessor447.invoke(Unknown Source)
>>> >> >>
>>> >> >>
>>> >> >>
>>> >>
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>> >> >>
>>> >> >> java.lang.reflect.Method.invoke(Method.java:615)
>>> >> >>
>>> >> >>
>>> >> >>
>>> >>
>>> nc.bs.framework.ejb.AbstractEJBServiceHandler.invoke(AbstractEJBServiceHandler.java:56)
>>> >> >>
>>> >> >>
>>> >> >>
>>> >>
>>> nc.bs.framework.ejb.AbstractEJBServiceHandler.invoke(AbstractEJBServiceHandler.java:34)
>>> >> >>
>>> >> >> $Proxy58.ncserverSendDatas(Unknown Source)
>>> >> >>
>>> >> >>
>>> >> >>
>>> >>
>>> nc.bs.mqv2.meam.mq.task.DoSendBaseDataTaskUtil.runTask(DoSendBaseDataTaskUtil.java:63)
>>> >> >>
>>> >> >>
>>> >> >>
>>> >>
>>> nc.bs.mqv2.meam.mq.task.NCServerBaseDatasSendTask.executeTask(NCServerBaseDatasSendTask.java:26)
>>> >> >>
>>> >> >>
>>> >> >>
>>> >>
>>> nc.bs.pub.taskcenter.BgBizsPluginProcess.callBusinessPlugin(BgBizsPluginProcess.java:43)
>>> >> >>
>>> >> >> nc.bs.pub.pa.read.TaskBodyBase.callBizsPlugin(TaskBodyBase.java:188)
>>> >> >>
>>> >> >>
>>> nc.bs.pub.pa.read.TaskBodyBase.callBusinessPlugin(TaskBodyBase.java:174)
>>> >> >>
>>> >> >> nc.bs.pub.pa.read.TaskBodyBase.createMetaMsg(TaskBodyBase.java:95)
>>> >> >>
>>> >> >>
>>> >>
>>> nc.bs.pub.pa.read.TaskBodyBase.createAndSendMessage(TaskBodyBase.java:41)
>>> >> >>
>>> >> >>
>>> >>
>>> nc.bs.pub.taskcenter.BackgroundTaskBody.execute(BackgroundTaskBody.java:47)
>>> >> >>
>>> >> >>
>>> >> >>
>>> >>
>>> nc.impl.uap.scheduler.ExecutorServiceImpl.executeTask_RequiresNew(ExecutorServiceImpl.java:27)
>>> >> >>
>>> >> >>
>>> >> >>
>>> >>
>>> nc.itf.uap.mw.MwBizEJBEjbBean.executeTask_RequiresNew(MwBizEJBEjbBean.java:1682)
>>> >> >>
>>> >> >>
>>> >>
>>> nc.itf.uap.mw.EJSLocalStatelessnc_itf_uap_mw_MwBizEJB_f98e92a8.executeTask_RequiresNew(Unknown
>>> >> >> Source)
>>> >> >>
>>> >> >> sun.reflect.GeneratedMethodAccessor1036.invoke(Unknown Source)
>>> >> >>
>>> >> >>
>>> >> >>
>>> >>
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>> >> >>
>>> >> >> java.lang.reflect.Method.invoke(Method.java:615)
>>> >> >>
>>> >> >>
>>> >> >>
>>> >>
>>> nc.bs.framework.ejb.AbstractEJBServiceHandler.invoke(AbstractEJBServiceHandler.java:56)
>>> >> >>
>>> >> >>
>>> >> >>
>>> >>
>>> nc.bs.framework.ejb.AbstractEJBServiceHandler.invoke(AbstractEJBServiceHandler.java:34)
>>> >> >>
>>> >> >> $Proxy39.executeTask_RequiresNew(Unknown Source)
>>> >> >>
>>> >> >>
>>> nc.bs.uap.scheduler.impl.TaskExecutor.executeTask(TaskExecutor.java:110)
>>> >> >>
>>> >> >> nc.bs.uap.scheduler.impl.TaskExecutor.run(TaskExecutor.java:75)
>>> >> >>
>>> >> >> nc.bs.framework.execute.Executor$1.run(Executor.java:80)
>>> >> >>
>>> >> >>
>>> >>
>>> nc.bs.framework.execute.commonj.impl.WorkAdaptor.run(WorkAdaptor.java:29)
>>> >> >>
>>> >> >> com.ibm.ws.asynchbeans.J2EEContext.run(J2EEContext.java:1114)
>>> >> >>
>>> >> >>
>>> >> >>
>>> >>
>>> com.ibm.ws.asynchbeans.WorkWithExecutionContextImpl.go(WorkWithExecutionContextImpl.java:195)
>>> >> >>
>>> >> >> com.ibm.ws.asynchbeans.CJWorkItemImpl.run(CJWorkItemImpl.java:187)
>>> >> >>
>>> >> >> com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1469)
>>> >> >>
>>> >> >>
>>> >> >>
>>> >> >>
>>> >> >> Best regards,
>>> >> >>
>>> >> >> Zhuran Li
>>> >> >
>>> >> >
>>> >> >
>>> >> >
>>> >> >--
>>> >> >*Christian Posta*
>>> >> >http://www.christianposta.com/blog
>>> >> >twitter: @christianposta
>>> >>
>>> >
>>> >
>>> >
>>> >--
>>> >*Christian Posta*
>>> >http://www.christianposta.com/blog
>>> >twitter: @christianposta
>>>
>>
>>
>>
>>--
>>*Christian Posta*
>>http://www.christianposta.com/blog
>>twitter: @christianposta
Re:Re: Re: Re: ActiveMQ client app hold unexpectedly!!!
Posted by lzr <js...@163.com>.
I found the issue just this time. It worked fine for about two month with the ActiveMQ M/S cluster.
I've thought of the network glitch; What I feel strange most is there is no any error/exception on both the ActiveMQ client side and server side.
I went through the related codes and found there is a FutureResponse.getResult method with parameter 'timeout'. Maybe this method is more suitable for getting the result. Not very sure, hehe.
At 2013-07-16 20:23:01,"Christian Posta" <ch...@gmail.com> wrote:
>Is this something that happens all the time? Maybe your network had a
>glitch and didn't deliver the response back to the client.
>
>
>On Tue, Jul 16, 2013 at 8:18 AM, lzr <js...@163.com> wrote:
>
>> I just check the configuration file, actually I've disabled the flow
>> control:
>>
>>
>> <destinationPolicy>
>> <policyMap>
>> <policyEntries>
>> <policyEntry topic=">" producerFlowControl="false"
>> memoryLimit="100mb">
>> <pendingSubscriberPolicy>
>> <vmCursor />
>> </pendingSubscriberPolicy>
>> </policyEntry>
>> <policyEntry queue=">" producerFlowControl="false"
>> memoryLimit="100mb">
>> <!-- Use VM cursor for better latency
>> For more information, see:
>>
>>
>> http://activemq.apache.org/message-cursors.html
>>
>>
>> <pendingQueuePolicy>
>> <vmQueueCursor/>
>> </pendingQueuePolicy>
>> -->
>> </policyEntry>
>> </policyEntries>
>> </policyMap>
>> </destinationPolicy>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> At 2013-07-16 20:15:11,"Christian Posta" <ch...@gmail.com>
>> wrote:
>> >Yes, on the first time PFC kicked in you would see the log messages. Not
>> on
>> >subsequent times.
>> >
>> >
>> >On Tue, Jul 16, 2013 at 8:01 AM, lzr <js...@163.com> wrote:
>> >
>> >> Based on my experiences if the issue caused by the producer flow control
>> >> configuration, the logs will have some warning messages, right? In fact
>> I
>> >> didn't found any warning or errors in the activemq.log.
>> >> Anyway, I'll check the configuration files right now.
>> >>
>> >>
>> >> thanks lot,
>> >> Zhuran.
>> >>
>> >>
>> >> At 2013-07-16 19:36:44,"Christian Posta" <ch...@gmail.com>
>> >> wrote:
>> >>
>> >> >This means the client didn't' receive a response from the broker.
>> >> >Did the broker producer flow control this client? ie, was the
>> destination
>> >> >to which the client sending full?
>> >> >
>> >> >
>> >> >On Tue, Jul 16, 2013 at 7:23 AM, lzr <js...@163.com> wrote:
>> >> >
>> >> >> Dear all,
>> >> >>
>> >> >>
>> >> >> I found a one of the ActiveMQ thread got stuck unexpectedly for
>> several
>> >> >> days. Customer was very angry to the ActiveMQ for the bad behavior.
>> Did
>> >> >> anybody experienced this issue? It would be much appreciated if any
>> >> clue!!!
>> >> >> My application is normal Java JMS application like the following:
>> >> >> new ActiveMQConnectionFactory.
>> >> >> create Connection;
>> >> >> conn.start.
>> >> >> create session.
>> >> >> create destination, producer and message
>> >> >> send messages
>> >> >> commit;
>> >> >> conn.close.
>> >> >>
>> >> >>
>> >> >> The follows is the stack when I looked into it with debug tools:
>> >> >>
>> >> >> sun.misc.Unsafe.park(Native Method)
>> >> >>
>> >> >> java.util.concurrent.locks.LockSupport.park(LockSupport.java:169)
>> >> >>
>> >> >>
>> >> >>
>> >>
>> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1787)
>> >> >>
>> >> >>
>> >>
>> java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:356)
>> >> >>
>> >> >>
>> >> >>
>> >>
>> org.apache.activemq.transport.FutureResponse.getResult(FutureResponse.java:40)
>> >> >>
>> >> >>
>> >> >>
>> >>
>> org.apache.activemq.transport.ResponseCorrelator.request(ResponseCorrelator.java:80)
>> >> >>
>> >> >>
>> >> >>
>> >>
>> org.apache.activemq.ActiveMQConnection.syncSendPacket(ActiveMQConnection.java:1233)
>> >> >>
>> >> >>
>> >> >>
>> >>
>> org.apache.activemq.ActiveMQConnection.ensureConnectionInfoSent(ActiveMQConnection.java:1339)
>> >> >>
>> >> >>
>> >>
>> org.apache.activemq.ActiveMQConnection.start(ActiveMQConnection.java:488)
>> >> >>
>> >> >>
>> nc.bs.mqv2.mmgp.mq.UFMQProcessor.getConnection(UFMQProcessor.java:487)
>> >> >>
>> >> >>
>> nc.bs.mqv2.mmgp.mq.UFMQProcessor.getConnection(UFMQProcessor.java:462)
>> >> >>
>> >> >>
>> >> >>
>> >>
>> nc.bs.mqv2.mmgp.mq.UFMQProcessor.sendSerializedObjMsg(UFMQProcessor.java:128)
>> >> >>
>> >> >>
>> >> >>
>> >>
>> nc.bs.mqv2.meam.mq.sendbasedatas.AbstractMeamBDSyncService.sendBaseData(AbstractMeamBDSyncService.java:64)
>> >> >>
>> >> >>
>> >> >>
>> >>
>> nc.bs.mqv2.mes.bd.mq.basedoc.AbstractBDServerSendService.dosendDataProcess(AbstractBDServerSendService.java:213)
>> >> >>
>> >> >>
>> >> >>
>> >>
>> nc.bs.mqv2.mes.bd.mq.basedoc.AbstractBDServerSendService.sendDatas(AbstractBDServerSendService.java:62)
>> >> >>
>> >> >>
>> >> >>
>> >>
>> nc.impl.mqv2.mes.bd.mq.SyncSendServiceImpl.ncserverSendDatas(SyncSendServiceImpl.java:58)
>> >> >>
>> >> >> nc.gejb.meam.cmt.MeamEjbBean.ncserverSendDatas(MeamEjbBean.java:8741)
>> >> >>
>> >> >>
>> >>
>> nc.gejb.meam.cmt.EJSLocalStatelessnc_gejb_meam_cmt_Meam_b44269e7.ncserverSendDatas(Unknown
>> >> >> Source)
>> >> >>
>> >> >> sun.reflect.GeneratedMethodAccessor447.invoke(Unknown Source)
>> >> >>
>> >> >>
>> >> >>
>> >>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> >> >>
>> >> >> java.lang.reflect.Method.invoke(Method.java:615)
>> >> >>
>> >> >>
>> >> >>
>> >>
>> nc.bs.framework.ejb.AbstractEJBServiceHandler.invoke(AbstractEJBServiceHandler.java:56)
>> >> >>
>> >> >>
>> >> >>
>> >>
>> nc.bs.framework.ejb.AbstractEJBServiceHandler.invoke(AbstractEJBServiceHandler.java:34)
>> >> >>
>> >> >> $Proxy58.ncserverSendDatas(Unknown Source)
>> >> >>
>> >> >>
>> >> >>
>> >>
>> nc.bs.mqv2.meam.mq.task.DoSendBaseDataTaskUtil.runTask(DoSendBaseDataTaskUtil.java:63)
>> >> >>
>> >> >>
>> >> >>
>> >>
>> nc.bs.mqv2.meam.mq.task.NCServerBaseDatasSendTask.executeTask(NCServerBaseDatasSendTask.java:26)
>> >> >>
>> >> >>
>> >> >>
>> >>
>> nc.bs.pub.taskcenter.BgBizsPluginProcess.callBusinessPlugin(BgBizsPluginProcess.java:43)
>> >> >>
>> >> >> nc.bs.pub.pa.read.TaskBodyBase.callBizsPlugin(TaskBodyBase.java:188)
>> >> >>
>> >> >>
>> nc.bs.pub.pa.read.TaskBodyBase.callBusinessPlugin(TaskBodyBase.java:174)
>> >> >>
>> >> >> nc.bs.pub.pa.read.TaskBodyBase.createMetaMsg(TaskBodyBase.java:95)
>> >> >>
>> >> >>
>> >>
>> nc.bs.pub.pa.read.TaskBodyBase.createAndSendMessage(TaskBodyBase.java:41)
>> >> >>
>> >> >>
>> >>
>> nc.bs.pub.taskcenter.BackgroundTaskBody.execute(BackgroundTaskBody.java:47)
>> >> >>
>> >> >>
>> >> >>
>> >>
>> nc.impl.uap.scheduler.ExecutorServiceImpl.executeTask_RequiresNew(ExecutorServiceImpl.java:27)
>> >> >>
>> >> >>
>> >> >>
>> >>
>> nc.itf.uap.mw.MwBizEJBEjbBean.executeTask_RequiresNew(MwBizEJBEjbBean.java:1682)
>> >> >>
>> >> >>
>> >>
>> nc.itf.uap.mw.EJSLocalStatelessnc_itf_uap_mw_MwBizEJB_f98e92a8.executeTask_RequiresNew(Unknown
>> >> >> Source)
>> >> >>
>> >> >> sun.reflect.GeneratedMethodAccessor1036.invoke(Unknown Source)
>> >> >>
>> >> >>
>> >> >>
>> >>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> >> >>
>> >> >> java.lang.reflect.Method.invoke(Method.java:615)
>> >> >>
>> >> >>
>> >> >>
>> >>
>> nc.bs.framework.ejb.AbstractEJBServiceHandler.invoke(AbstractEJBServiceHandler.java:56)
>> >> >>
>> >> >>
>> >> >>
>> >>
>> nc.bs.framework.ejb.AbstractEJBServiceHandler.invoke(AbstractEJBServiceHandler.java:34)
>> >> >>
>> >> >> $Proxy39.executeTask_RequiresNew(Unknown Source)
>> >> >>
>> >> >>
>> nc.bs.uap.scheduler.impl.TaskExecutor.executeTask(TaskExecutor.java:110)
>> >> >>
>> >> >> nc.bs.uap.scheduler.impl.TaskExecutor.run(TaskExecutor.java:75)
>> >> >>
>> >> >> nc.bs.framework.execute.Executor$1.run(Executor.java:80)
>> >> >>
>> >> >>
>> >>
>> nc.bs.framework.execute.commonj.impl.WorkAdaptor.run(WorkAdaptor.java:29)
>> >> >>
>> >> >> com.ibm.ws.asynchbeans.J2EEContext.run(J2EEContext.java:1114)
>> >> >>
>> >> >>
>> >> >>
>> >>
>> com.ibm.ws.asynchbeans.WorkWithExecutionContextImpl.go(WorkWithExecutionContextImpl.java:195)
>> >> >>
>> >> >> com.ibm.ws.asynchbeans.CJWorkItemImpl.run(CJWorkItemImpl.java:187)
>> >> >>
>> >> >> com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1469)
>> >> >>
>> >> >>
>> >> >>
>> >> >>
>> >> >> Best regards,
>> >> >>
>> >> >> Zhuran Li
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >--
>> >> >*Christian Posta*
>> >> >http://www.christianposta.com/blog
>> >> >twitter: @christianposta
>> >>
>> >
>> >
>> >
>> >--
>> >*Christian Posta*
>> >http://www.christianposta.com/blog
>> >twitter: @christianposta
>>
>
>
>
>--
>*Christian Posta*
>http://www.christianposta.com/blog
>twitter: @christianposta
Re: Re: Re: ActiveMQ client app hold unexpectedly!!!
Posted by Christian Posta <ch...@gmail.com>.
Is this something that happens all the time? Maybe your network had a
glitch and didn't deliver the response back to the client.
On Tue, Jul 16, 2013 at 8:18 AM, lzr <js...@163.com> wrote:
> I just check the configuration file, actually I've disabled the flow
> control:
>
>
> <destinationPolicy>
> <policyMap>
> <policyEntries>
> <policyEntry topic=">" producerFlowControl="false"
> memoryLimit="100mb">
> <pendingSubscriberPolicy>
> <vmCursor />
> </pendingSubscriberPolicy>
> </policyEntry>
> <policyEntry queue=">" producerFlowControl="false"
> memoryLimit="100mb">
> <!-- Use VM cursor for better latency
> For more information, see:
>
>
> http://activemq.apache.org/message-cursors.html
>
>
> <pendingQueuePolicy>
> <vmQueueCursor/>
> </pendingQueuePolicy>
> -->
> </policyEntry>
> </policyEntries>
> </policyMap>
> </destinationPolicy>
>
>
>
>
>
>
>
>
>
>
> At 2013-07-16 20:15:11,"Christian Posta" <ch...@gmail.com>
> wrote:
> >Yes, on the first time PFC kicked in you would see the log messages. Not
> on
> >subsequent times.
> >
> >
> >On Tue, Jul 16, 2013 at 8:01 AM, lzr <js...@163.com> wrote:
> >
> >> Based on my experiences if the issue caused by the producer flow control
> >> configuration, the logs will have some warning messages, right? In fact
> I
> >> didn't found any warning or errors in the activemq.log.
> >> Anyway, I'll check the configuration files right now.
> >>
> >>
> >> thanks lot,
> >> Zhuran.
> >>
> >>
> >> At 2013-07-16 19:36:44,"Christian Posta" <ch...@gmail.com>
> >> wrote:
> >>
> >> >This means the client didn't' receive a response from the broker.
> >> >Did the broker producer flow control this client? ie, was the
> destination
> >> >to which the client sending full?
> >> >
> >> >
> >> >On Tue, Jul 16, 2013 at 7:23 AM, lzr <js...@163.com> wrote:
> >> >
> >> >> Dear all,
> >> >>
> >> >>
> >> >> I found a one of the ActiveMQ thread got stuck unexpectedly for
> several
> >> >> days. Customer was very angry to the ActiveMQ for the bad behavior.
> Did
> >> >> anybody experienced this issue? It would be much appreciated if any
> >> clue!!!
> >> >> My application is normal Java JMS application like the following:
> >> >> new ActiveMQConnectionFactory.
> >> >> create Connection;
> >> >> conn.start.
> >> >> create session.
> >> >> create destination, producer and message
> >> >> send messages
> >> >> commit;
> >> >> conn.close.
> >> >>
> >> >>
> >> >> The follows is the stack when I looked into it with debug tools:
> >> >>
> >> >> sun.misc.Unsafe.park(Native Method)
> >> >>
> >> >> java.util.concurrent.locks.LockSupport.park(LockSupport.java:169)
> >> >>
> >> >>
> >> >>
> >>
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1787)
> >> >>
> >> >>
> >>
> java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:356)
> >> >>
> >> >>
> >> >>
> >>
> org.apache.activemq.transport.FutureResponse.getResult(FutureResponse.java:40)
> >> >>
> >> >>
> >> >>
> >>
> org.apache.activemq.transport.ResponseCorrelator.request(ResponseCorrelator.java:80)
> >> >>
> >> >>
> >> >>
> >>
> org.apache.activemq.ActiveMQConnection.syncSendPacket(ActiveMQConnection.java:1233)
> >> >>
> >> >>
> >> >>
> >>
> org.apache.activemq.ActiveMQConnection.ensureConnectionInfoSent(ActiveMQConnection.java:1339)
> >> >>
> >> >>
> >>
> org.apache.activemq.ActiveMQConnection.start(ActiveMQConnection.java:488)
> >> >>
> >> >>
> nc.bs.mqv2.mmgp.mq.UFMQProcessor.getConnection(UFMQProcessor.java:487)
> >> >>
> >> >>
> nc.bs.mqv2.mmgp.mq.UFMQProcessor.getConnection(UFMQProcessor.java:462)
> >> >>
> >> >>
> >> >>
> >>
> nc.bs.mqv2.mmgp.mq.UFMQProcessor.sendSerializedObjMsg(UFMQProcessor.java:128)
> >> >>
> >> >>
> >> >>
> >>
> nc.bs.mqv2.meam.mq.sendbasedatas.AbstractMeamBDSyncService.sendBaseData(AbstractMeamBDSyncService.java:64)
> >> >>
> >> >>
> >> >>
> >>
> nc.bs.mqv2.mes.bd.mq.basedoc.AbstractBDServerSendService.dosendDataProcess(AbstractBDServerSendService.java:213)
> >> >>
> >> >>
> >> >>
> >>
> nc.bs.mqv2.mes.bd.mq.basedoc.AbstractBDServerSendService.sendDatas(AbstractBDServerSendService.java:62)
> >> >>
> >> >>
> >> >>
> >>
> nc.impl.mqv2.mes.bd.mq.SyncSendServiceImpl.ncserverSendDatas(SyncSendServiceImpl.java:58)
> >> >>
> >> >> nc.gejb.meam.cmt.MeamEjbBean.ncserverSendDatas(MeamEjbBean.java:8741)
> >> >>
> >> >>
> >>
> nc.gejb.meam.cmt.EJSLocalStatelessnc_gejb_meam_cmt_Meam_b44269e7.ncserverSendDatas(Unknown
> >> >> Source)
> >> >>
> >> >> sun.reflect.GeneratedMethodAccessor447.invoke(Unknown Source)
> >> >>
> >> >>
> >> >>
> >>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >> >>
> >> >> java.lang.reflect.Method.invoke(Method.java:615)
> >> >>
> >> >>
> >> >>
> >>
> nc.bs.framework.ejb.AbstractEJBServiceHandler.invoke(AbstractEJBServiceHandler.java:56)
> >> >>
> >> >>
> >> >>
> >>
> nc.bs.framework.ejb.AbstractEJBServiceHandler.invoke(AbstractEJBServiceHandler.java:34)
> >> >>
> >> >> $Proxy58.ncserverSendDatas(Unknown Source)
> >> >>
> >> >>
> >> >>
> >>
> nc.bs.mqv2.meam.mq.task.DoSendBaseDataTaskUtil.runTask(DoSendBaseDataTaskUtil.java:63)
> >> >>
> >> >>
> >> >>
> >>
> nc.bs.mqv2.meam.mq.task.NCServerBaseDatasSendTask.executeTask(NCServerBaseDatasSendTask.java:26)
> >> >>
> >> >>
> >> >>
> >>
> nc.bs.pub.taskcenter.BgBizsPluginProcess.callBusinessPlugin(BgBizsPluginProcess.java:43)
> >> >>
> >> >> nc.bs.pub.pa.read.TaskBodyBase.callBizsPlugin(TaskBodyBase.java:188)
> >> >>
> >> >>
> nc.bs.pub.pa.read.TaskBodyBase.callBusinessPlugin(TaskBodyBase.java:174)
> >> >>
> >> >> nc.bs.pub.pa.read.TaskBodyBase.createMetaMsg(TaskBodyBase.java:95)
> >> >>
> >> >>
> >>
> nc.bs.pub.pa.read.TaskBodyBase.createAndSendMessage(TaskBodyBase.java:41)
> >> >>
> >> >>
> >>
> nc.bs.pub.taskcenter.BackgroundTaskBody.execute(BackgroundTaskBody.java:47)
> >> >>
> >> >>
> >> >>
> >>
> nc.impl.uap.scheduler.ExecutorServiceImpl.executeTask_RequiresNew(ExecutorServiceImpl.java:27)
> >> >>
> >> >>
> >> >>
> >>
> nc.itf.uap.mw.MwBizEJBEjbBean.executeTask_RequiresNew(MwBizEJBEjbBean.java:1682)
> >> >>
> >> >>
> >>
> nc.itf.uap.mw.EJSLocalStatelessnc_itf_uap_mw_MwBizEJB_f98e92a8.executeTask_RequiresNew(Unknown
> >> >> Source)
> >> >>
> >> >> sun.reflect.GeneratedMethodAccessor1036.invoke(Unknown Source)
> >> >>
> >> >>
> >> >>
> >>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >> >>
> >> >> java.lang.reflect.Method.invoke(Method.java:615)
> >> >>
> >> >>
> >> >>
> >>
> nc.bs.framework.ejb.AbstractEJBServiceHandler.invoke(AbstractEJBServiceHandler.java:56)
> >> >>
> >> >>
> >> >>
> >>
> nc.bs.framework.ejb.AbstractEJBServiceHandler.invoke(AbstractEJBServiceHandler.java:34)
> >> >>
> >> >> $Proxy39.executeTask_RequiresNew(Unknown Source)
> >> >>
> >> >>
> nc.bs.uap.scheduler.impl.TaskExecutor.executeTask(TaskExecutor.java:110)
> >> >>
> >> >> nc.bs.uap.scheduler.impl.TaskExecutor.run(TaskExecutor.java:75)
> >> >>
> >> >> nc.bs.framework.execute.Executor$1.run(Executor.java:80)
> >> >>
> >> >>
> >>
> nc.bs.framework.execute.commonj.impl.WorkAdaptor.run(WorkAdaptor.java:29)
> >> >>
> >> >> com.ibm.ws.asynchbeans.J2EEContext.run(J2EEContext.java:1114)
> >> >>
> >> >>
> >> >>
> >>
> com.ibm.ws.asynchbeans.WorkWithExecutionContextImpl.go(WorkWithExecutionContextImpl.java:195)
> >> >>
> >> >> com.ibm.ws.asynchbeans.CJWorkItemImpl.run(CJWorkItemImpl.java:187)
> >> >>
> >> >> com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1469)
> >> >>
> >> >>
> >> >>
> >> >>
> >> >> Best regards,
> >> >>
> >> >> Zhuran Li
> >> >
> >> >
> >> >
> >> >
> >> >--
> >> >*Christian Posta*
> >> >http://www.christianposta.com/blog
> >> >twitter: @christianposta
> >>
> >
> >
> >
> >--
> >*Christian Posta*
> >http://www.christianposta.com/blog
> >twitter: @christianposta
>
--
*Christian Posta*
http://www.christianposta.com/blog
twitter: @christianposta
Re:Re: Re: ActiveMQ client app hold unexpectedly!!!
Posted by lzr <js...@163.com>.
I just check the configuration file, actually I've disabled the flow control:
<destinationPolicy>
<policyMap>
<policyEntries>
<policyEntry topic=">" producerFlowControl="false" memoryLimit="100mb">
<pendingSubscriberPolicy>
<vmCursor />
</pendingSubscriberPolicy>
</policyEntry>
<policyEntry queue=">" producerFlowControl="false" memoryLimit="100mb">
<!-- Use VM cursor for better latency
For more information, see:
http://activemq.apache.org/message-cursors.html
<pendingQueuePolicy>
<vmQueueCursor/>
</pendingQueuePolicy>
-->
</policyEntry>
</policyEntries>
</policyMap>
</destinationPolicy>
At 2013-07-16 20:15:11,"Christian Posta" <ch...@gmail.com> wrote:
>Yes, on the first time PFC kicked in you would see the log messages. Not on
>subsequent times.
>
>
>On Tue, Jul 16, 2013 at 8:01 AM, lzr <js...@163.com> wrote:
>
>> Based on my experiences if the issue caused by the producer flow control
>> configuration, the logs will have some warning messages, right? In fact I
>> didn't found any warning or errors in the activemq.log.
>> Anyway, I'll check the configuration files right now.
>>
>>
>> thanks lot,
>> Zhuran.
>>
>>
>> At 2013-07-16 19:36:44,"Christian Posta" <ch...@gmail.com>
>> wrote:
>>
>> >This means the client didn't' receive a response from the broker.
>> >Did the broker producer flow control this client? ie, was the destination
>> >to which the client sending full?
>> >
>> >
>> >On Tue, Jul 16, 2013 at 7:23 AM, lzr <js...@163.com> wrote:
>> >
>> >> Dear all,
>> >>
>> >>
>> >> I found a one of the ActiveMQ thread got stuck unexpectedly for several
>> >> days. Customer was very angry to the ActiveMQ for the bad behavior. Did
>> >> anybody experienced this issue? It would be much appreciated if any
>> clue!!!
>> >> My application is normal Java JMS application like the following:
>> >> new ActiveMQConnectionFactory.
>> >> create Connection;
>> >> conn.start.
>> >> create session.
>> >> create destination, producer and message
>> >> send messages
>> >> commit;
>> >> conn.close.
>> >>
>> >>
>> >> The follows is the stack when I looked into it with debug tools:
>> >>
>> >> sun.misc.Unsafe.park(Native Method)
>> >>
>> >> java.util.concurrent.locks.LockSupport.park(LockSupport.java:169)
>> >>
>> >>
>> >>
>> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1787)
>> >>
>> >>
>> java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:356)
>> >>
>> >>
>> >>
>> org.apache.activemq.transport.FutureResponse.getResult(FutureResponse.java:40)
>> >>
>> >>
>> >>
>> org.apache.activemq.transport.ResponseCorrelator.request(ResponseCorrelator.java:80)
>> >>
>> >>
>> >>
>> org.apache.activemq.ActiveMQConnection.syncSendPacket(ActiveMQConnection.java:1233)
>> >>
>> >>
>> >>
>> org.apache.activemq.ActiveMQConnection.ensureConnectionInfoSent(ActiveMQConnection.java:1339)
>> >>
>> >>
>> org.apache.activemq.ActiveMQConnection.start(ActiveMQConnection.java:488)
>> >>
>> >> nc.bs.mqv2.mmgp.mq.UFMQProcessor.getConnection(UFMQProcessor.java:487)
>> >>
>> >> nc.bs.mqv2.mmgp.mq.UFMQProcessor.getConnection(UFMQProcessor.java:462)
>> >>
>> >>
>> >>
>> nc.bs.mqv2.mmgp.mq.UFMQProcessor.sendSerializedObjMsg(UFMQProcessor.java:128)
>> >>
>> >>
>> >>
>> nc.bs.mqv2.meam.mq.sendbasedatas.AbstractMeamBDSyncService.sendBaseData(AbstractMeamBDSyncService.java:64)
>> >>
>> >>
>> >>
>> nc.bs.mqv2.mes.bd.mq.basedoc.AbstractBDServerSendService.dosendDataProcess(AbstractBDServerSendService.java:213)
>> >>
>> >>
>> >>
>> nc.bs.mqv2.mes.bd.mq.basedoc.AbstractBDServerSendService.sendDatas(AbstractBDServerSendService.java:62)
>> >>
>> >>
>> >>
>> nc.impl.mqv2.mes.bd.mq.SyncSendServiceImpl.ncserverSendDatas(SyncSendServiceImpl.java:58)
>> >>
>> >> nc.gejb.meam.cmt.MeamEjbBean.ncserverSendDatas(MeamEjbBean.java:8741)
>> >>
>> >>
>> nc.gejb.meam.cmt.EJSLocalStatelessnc_gejb_meam_cmt_Meam_b44269e7.ncserverSendDatas(Unknown
>> >> Source)
>> >>
>> >> sun.reflect.GeneratedMethodAccessor447.invoke(Unknown Source)
>> >>
>> >>
>> >>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> >>
>> >> java.lang.reflect.Method.invoke(Method.java:615)
>> >>
>> >>
>> >>
>> nc.bs.framework.ejb.AbstractEJBServiceHandler.invoke(AbstractEJBServiceHandler.java:56)
>> >>
>> >>
>> >>
>> nc.bs.framework.ejb.AbstractEJBServiceHandler.invoke(AbstractEJBServiceHandler.java:34)
>> >>
>> >> $Proxy58.ncserverSendDatas(Unknown Source)
>> >>
>> >>
>> >>
>> nc.bs.mqv2.meam.mq.task.DoSendBaseDataTaskUtil.runTask(DoSendBaseDataTaskUtil.java:63)
>> >>
>> >>
>> >>
>> nc.bs.mqv2.meam.mq.task.NCServerBaseDatasSendTask.executeTask(NCServerBaseDatasSendTask.java:26)
>> >>
>> >>
>> >>
>> nc.bs.pub.taskcenter.BgBizsPluginProcess.callBusinessPlugin(BgBizsPluginProcess.java:43)
>> >>
>> >> nc.bs.pub.pa.read.TaskBodyBase.callBizsPlugin(TaskBodyBase.java:188)
>> >>
>> >> nc.bs.pub.pa.read.TaskBodyBase.callBusinessPlugin(TaskBodyBase.java:174)
>> >>
>> >> nc.bs.pub.pa.read.TaskBodyBase.createMetaMsg(TaskBodyBase.java:95)
>> >>
>> >>
>> nc.bs.pub.pa.read.TaskBodyBase.createAndSendMessage(TaskBodyBase.java:41)
>> >>
>> >>
>> nc.bs.pub.taskcenter.BackgroundTaskBody.execute(BackgroundTaskBody.java:47)
>> >>
>> >>
>> >>
>> nc.impl.uap.scheduler.ExecutorServiceImpl.executeTask_RequiresNew(ExecutorServiceImpl.java:27)
>> >>
>> >>
>> >>
>> nc.itf.uap.mw.MwBizEJBEjbBean.executeTask_RequiresNew(MwBizEJBEjbBean.java:1682)
>> >>
>> >>
>> nc.itf.uap.mw.EJSLocalStatelessnc_itf_uap_mw_MwBizEJB_f98e92a8.executeTask_RequiresNew(Unknown
>> >> Source)
>> >>
>> >> sun.reflect.GeneratedMethodAccessor1036.invoke(Unknown Source)
>> >>
>> >>
>> >>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> >>
>> >> java.lang.reflect.Method.invoke(Method.java:615)
>> >>
>> >>
>> >>
>> nc.bs.framework.ejb.AbstractEJBServiceHandler.invoke(AbstractEJBServiceHandler.java:56)
>> >>
>> >>
>> >>
>> nc.bs.framework.ejb.AbstractEJBServiceHandler.invoke(AbstractEJBServiceHandler.java:34)
>> >>
>> >> $Proxy39.executeTask_RequiresNew(Unknown Source)
>> >>
>> >> nc.bs.uap.scheduler.impl.TaskExecutor.executeTask(TaskExecutor.java:110)
>> >>
>> >> nc.bs.uap.scheduler.impl.TaskExecutor.run(TaskExecutor.java:75)
>> >>
>> >> nc.bs.framework.execute.Executor$1.run(Executor.java:80)
>> >>
>> >>
>> nc.bs.framework.execute.commonj.impl.WorkAdaptor.run(WorkAdaptor.java:29)
>> >>
>> >> com.ibm.ws.asynchbeans.J2EEContext.run(J2EEContext.java:1114)
>> >>
>> >>
>> >>
>> com.ibm.ws.asynchbeans.WorkWithExecutionContextImpl.go(WorkWithExecutionContextImpl.java:195)
>> >>
>> >> com.ibm.ws.asynchbeans.CJWorkItemImpl.run(CJWorkItemImpl.java:187)
>> >>
>> >> com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1469)
>> >>
>> >>
>> >>
>> >>
>> >> Best regards,
>> >>
>> >> Zhuran Li
>> >
>> >
>> >
>> >
>> >--
>> >*Christian Posta*
>> >http://www.christianposta.com/blog
>> >twitter: @christianposta
>>
>
>
>
>--
>*Christian Posta*
>http://www.christianposta.com/blog
>twitter: @christianposta
Re: Re: ActiveMQ client app hold unexpectedly!!!
Posted by Christian Posta <ch...@gmail.com>.
Yes, on the first time PFC kicked in you would see the log messages. Not on
subsequent times.
On Tue, Jul 16, 2013 at 8:01 AM, lzr <js...@163.com> wrote:
> Based on my experiences if the issue caused by the producer flow control
> configuration, the logs will have some warning messages, right? In fact I
> didn't found any warning or errors in the activemq.log.
> Anyway, I'll check the configuration files right now.
>
>
> thanks lot,
> Zhuran.
>
>
> At 2013-07-16 19:36:44,"Christian Posta" <ch...@gmail.com>
> wrote:
>
> >This means the client didn't' receive a response from the broker.
> >Did the broker producer flow control this client? ie, was the destination
> >to which the client sending full?
> >
> >
> >On Tue, Jul 16, 2013 at 7:23 AM, lzr <js...@163.com> wrote:
> >
> >> Dear all,
> >>
> >>
> >> I found a one of the ActiveMQ thread got stuck unexpectedly for several
> >> days. Customer was very angry to the ActiveMQ for the bad behavior. Did
> >> anybody experienced this issue? It would be much appreciated if any
> clue!!!
> >> My application is normal Java JMS application like the following:
> >> new ActiveMQConnectionFactory.
> >> create Connection;
> >> conn.start.
> >> create session.
> >> create destination, producer and message
> >> send messages
> >> commit;
> >> conn.close.
> >>
> >>
> >> The follows is the stack when I looked into it with debug tools:
> >>
> >> sun.misc.Unsafe.park(Native Method)
> >>
> >> java.util.concurrent.locks.LockSupport.park(LockSupport.java:169)
> >>
> >>
> >>
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1787)
> >>
> >>
> java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:356)
> >>
> >>
> >>
> org.apache.activemq.transport.FutureResponse.getResult(FutureResponse.java:40)
> >>
> >>
> >>
> org.apache.activemq.transport.ResponseCorrelator.request(ResponseCorrelator.java:80)
> >>
> >>
> >>
> org.apache.activemq.ActiveMQConnection.syncSendPacket(ActiveMQConnection.java:1233)
> >>
> >>
> >>
> org.apache.activemq.ActiveMQConnection.ensureConnectionInfoSent(ActiveMQConnection.java:1339)
> >>
> >>
> org.apache.activemq.ActiveMQConnection.start(ActiveMQConnection.java:488)
> >>
> >> nc.bs.mqv2.mmgp.mq.UFMQProcessor.getConnection(UFMQProcessor.java:487)
> >>
> >> nc.bs.mqv2.mmgp.mq.UFMQProcessor.getConnection(UFMQProcessor.java:462)
> >>
> >>
> >>
> nc.bs.mqv2.mmgp.mq.UFMQProcessor.sendSerializedObjMsg(UFMQProcessor.java:128)
> >>
> >>
> >>
> nc.bs.mqv2.meam.mq.sendbasedatas.AbstractMeamBDSyncService.sendBaseData(AbstractMeamBDSyncService.java:64)
> >>
> >>
> >>
> nc.bs.mqv2.mes.bd.mq.basedoc.AbstractBDServerSendService.dosendDataProcess(AbstractBDServerSendService.java:213)
> >>
> >>
> >>
> nc.bs.mqv2.mes.bd.mq.basedoc.AbstractBDServerSendService.sendDatas(AbstractBDServerSendService.java:62)
> >>
> >>
> >>
> nc.impl.mqv2.mes.bd.mq.SyncSendServiceImpl.ncserverSendDatas(SyncSendServiceImpl.java:58)
> >>
> >> nc.gejb.meam.cmt.MeamEjbBean.ncserverSendDatas(MeamEjbBean.java:8741)
> >>
> >>
> nc.gejb.meam.cmt.EJSLocalStatelessnc_gejb_meam_cmt_Meam_b44269e7.ncserverSendDatas(Unknown
> >> Source)
> >>
> >> sun.reflect.GeneratedMethodAccessor447.invoke(Unknown Source)
> >>
> >>
> >>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >>
> >> java.lang.reflect.Method.invoke(Method.java:615)
> >>
> >>
> >>
> nc.bs.framework.ejb.AbstractEJBServiceHandler.invoke(AbstractEJBServiceHandler.java:56)
> >>
> >>
> >>
> nc.bs.framework.ejb.AbstractEJBServiceHandler.invoke(AbstractEJBServiceHandler.java:34)
> >>
> >> $Proxy58.ncserverSendDatas(Unknown Source)
> >>
> >>
> >>
> nc.bs.mqv2.meam.mq.task.DoSendBaseDataTaskUtil.runTask(DoSendBaseDataTaskUtil.java:63)
> >>
> >>
> >>
> nc.bs.mqv2.meam.mq.task.NCServerBaseDatasSendTask.executeTask(NCServerBaseDatasSendTask.java:26)
> >>
> >>
> >>
> nc.bs.pub.taskcenter.BgBizsPluginProcess.callBusinessPlugin(BgBizsPluginProcess.java:43)
> >>
> >> nc.bs.pub.pa.read.TaskBodyBase.callBizsPlugin(TaskBodyBase.java:188)
> >>
> >> nc.bs.pub.pa.read.TaskBodyBase.callBusinessPlugin(TaskBodyBase.java:174)
> >>
> >> nc.bs.pub.pa.read.TaskBodyBase.createMetaMsg(TaskBodyBase.java:95)
> >>
> >>
> nc.bs.pub.pa.read.TaskBodyBase.createAndSendMessage(TaskBodyBase.java:41)
> >>
> >>
> nc.bs.pub.taskcenter.BackgroundTaskBody.execute(BackgroundTaskBody.java:47)
> >>
> >>
> >>
> nc.impl.uap.scheduler.ExecutorServiceImpl.executeTask_RequiresNew(ExecutorServiceImpl.java:27)
> >>
> >>
> >>
> nc.itf.uap.mw.MwBizEJBEjbBean.executeTask_RequiresNew(MwBizEJBEjbBean.java:1682)
> >>
> >>
> nc.itf.uap.mw.EJSLocalStatelessnc_itf_uap_mw_MwBizEJB_f98e92a8.executeTask_RequiresNew(Unknown
> >> Source)
> >>
> >> sun.reflect.GeneratedMethodAccessor1036.invoke(Unknown Source)
> >>
> >>
> >>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >>
> >> java.lang.reflect.Method.invoke(Method.java:615)
> >>
> >>
> >>
> nc.bs.framework.ejb.AbstractEJBServiceHandler.invoke(AbstractEJBServiceHandler.java:56)
> >>
> >>
> >>
> nc.bs.framework.ejb.AbstractEJBServiceHandler.invoke(AbstractEJBServiceHandler.java:34)
> >>
> >> $Proxy39.executeTask_RequiresNew(Unknown Source)
> >>
> >> nc.bs.uap.scheduler.impl.TaskExecutor.executeTask(TaskExecutor.java:110)
> >>
> >> nc.bs.uap.scheduler.impl.TaskExecutor.run(TaskExecutor.java:75)
> >>
> >> nc.bs.framework.execute.Executor$1.run(Executor.java:80)
> >>
> >>
> nc.bs.framework.execute.commonj.impl.WorkAdaptor.run(WorkAdaptor.java:29)
> >>
> >> com.ibm.ws.asynchbeans.J2EEContext.run(J2EEContext.java:1114)
> >>
> >>
> >>
> com.ibm.ws.asynchbeans.WorkWithExecutionContextImpl.go(WorkWithExecutionContextImpl.java:195)
> >>
> >> com.ibm.ws.asynchbeans.CJWorkItemImpl.run(CJWorkItemImpl.java:187)
> >>
> >> com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1469)
> >>
> >>
> >>
> >>
> >> Best regards,
> >>
> >> Zhuran Li
> >
> >
> >
> >
> >--
> >*Christian Posta*
> >http://www.christianposta.com/blog
> >twitter: @christianposta
>
--
*Christian Posta*
http://www.christianposta.com/blog
twitter: @christianposta
Re:Re: ActiveMQ client app hold unexpectedly!!!
Posted by lzr <js...@163.com>.
Based on my experiences if the issue caused by the producer flow control configuration, the logs will have some warning messages, right? In fact I didn't found any warning or errors in the activemq.log.
Anyway, I'll check the configuration files right now.
thanks lot,
Zhuran.
At 2013-07-16 19:36:44,"Christian Posta" <ch...@gmail.com> wrote:
>This means the client didn't' receive a response from the broker.
>Did the broker producer flow control this client? ie, was the destination
>to which the client sending full?
>
>
>On Tue, Jul 16, 2013 at 7:23 AM, lzr <js...@163.com> wrote:
>
>> Dear all,
>>
>>
>> I found a one of the ActiveMQ thread got stuck unexpectedly for several
>> days. Customer was very angry to the ActiveMQ for the bad behavior. Did
>> anybody experienced this issue? It would be much appreciated if any clue!!!
>> My application is normal Java JMS application like the following:
>> new ActiveMQConnectionFactory.
>> create Connection;
>> conn.start.
>> create session.
>> create destination, producer and message
>> send messages
>> commit;
>> conn.close.
>>
>>
>> The follows is the stack when I looked into it with debug tools:
>>
>> sun.misc.Unsafe.park(Native Method)
>>
>> java.util.concurrent.locks.LockSupport.park(LockSupport.java:169)
>>
>>
>> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1787)
>>
>> java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:356)
>>
>>
>> org.apache.activemq.transport.FutureResponse.getResult(FutureResponse.java:40)
>>
>>
>> org.apache.activemq.transport.ResponseCorrelator.request(ResponseCorrelator.java:80)
>>
>>
>> org.apache.activemq.ActiveMQConnection.syncSendPacket(ActiveMQConnection.java:1233)
>>
>>
>> org.apache.activemq.ActiveMQConnection.ensureConnectionInfoSent(ActiveMQConnection.java:1339)
>>
>> org.apache.activemq.ActiveMQConnection.start(ActiveMQConnection.java:488)
>>
>> nc.bs.mqv2.mmgp.mq.UFMQProcessor.getConnection(UFMQProcessor.java:487)
>>
>> nc.bs.mqv2.mmgp.mq.UFMQProcessor.getConnection(UFMQProcessor.java:462)
>>
>>
>> nc.bs.mqv2.mmgp.mq.UFMQProcessor.sendSerializedObjMsg(UFMQProcessor.java:128)
>>
>>
>> nc.bs.mqv2.meam.mq.sendbasedatas.AbstractMeamBDSyncService.sendBaseData(AbstractMeamBDSyncService.java:64)
>>
>>
>> nc.bs.mqv2.mes.bd.mq.basedoc.AbstractBDServerSendService.dosendDataProcess(AbstractBDServerSendService.java:213)
>>
>>
>> nc.bs.mqv2.mes.bd.mq.basedoc.AbstractBDServerSendService.sendDatas(AbstractBDServerSendService.java:62)
>>
>>
>> nc.impl.mqv2.mes.bd.mq.SyncSendServiceImpl.ncserverSendDatas(SyncSendServiceImpl.java:58)
>>
>> nc.gejb.meam.cmt.MeamEjbBean.ncserverSendDatas(MeamEjbBean.java:8741)
>>
>> nc.gejb.meam.cmt.EJSLocalStatelessnc_gejb_meam_cmt_Meam_b44269e7.ncserverSendDatas(Unknown
>> Source)
>>
>> sun.reflect.GeneratedMethodAccessor447.invoke(Unknown Source)
>>
>>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>
>> java.lang.reflect.Method.invoke(Method.java:615)
>>
>>
>> nc.bs.framework.ejb.AbstractEJBServiceHandler.invoke(AbstractEJBServiceHandler.java:56)
>>
>>
>> nc.bs.framework.ejb.AbstractEJBServiceHandler.invoke(AbstractEJBServiceHandler.java:34)
>>
>> $Proxy58.ncserverSendDatas(Unknown Source)
>>
>>
>> nc.bs.mqv2.meam.mq.task.DoSendBaseDataTaskUtil.runTask(DoSendBaseDataTaskUtil.java:63)
>>
>>
>> nc.bs.mqv2.meam.mq.task.NCServerBaseDatasSendTask.executeTask(NCServerBaseDatasSendTask.java:26)
>>
>>
>> nc.bs.pub.taskcenter.BgBizsPluginProcess.callBusinessPlugin(BgBizsPluginProcess.java:43)
>>
>> nc.bs.pub.pa.read.TaskBodyBase.callBizsPlugin(TaskBodyBase.java:188)
>>
>> nc.bs.pub.pa.read.TaskBodyBase.callBusinessPlugin(TaskBodyBase.java:174)
>>
>> nc.bs.pub.pa.read.TaskBodyBase.createMetaMsg(TaskBodyBase.java:95)
>>
>> nc.bs.pub.pa.read.TaskBodyBase.createAndSendMessage(TaskBodyBase.java:41)
>>
>> nc.bs.pub.taskcenter.BackgroundTaskBody.execute(BackgroundTaskBody.java:47)
>>
>>
>> nc.impl.uap.scheduler.ExecutorServiceImpl.executeTask_RequiresNew(ExecutorServiceImpl.java:27)
>>
>>
>> nc.itf.uap.mw.MwBizEJBEjbBean.executeTask_RequiresNew(MwBizEJBEjbBean.java:1682)
>>
>> nc.itf.uap.mw.EJSLocalStatelessnc_itf_uap_mw_MwBizEJB_f98e92a8.executeTask_RequiresNew(Unknown
>> Source)
>>
>> sun.reflect.GeneratedMethodAccessor1036.invoke(Unknown Source)
>>
>>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>
>> java.lang.reflect.Method.invoke(Method.java:615)
>>
>>
>> nc.bs.framework.ejb.AbstractEJBServiceHandler.invoke(AbstractEJBServiceHandler.java:56)
>>
>>
>> nc.bs.framework.ejb.AbstractEJBServiceHandler.invoke(AbstractEJBServiceHandler.java:34)
>>
>> $Proxy39.executeTask_RequiresNew(Unknown Source)
>>
>> nc.bs.uap.scheduler.impl.TaskExecutor.executeTask(TaskExecutor.java:110)
>>
>> nc.bs.uap.scheduler.impl.TaskExecutor.run(TaskExecutor.java:75)
>>
>> nc.bs.framework.execute.Executor$1.run(Executor.java:80)
>>
>> nc.bs.framework.execute.commonj.impl.WorkAdaptor.run(WorkAdaptor.java:29)
>>
>> com.ibm.ws.asynchbeans.J2EEContext.run(J2EEContext.java:1114)
>>
>>
>> com.ibm.ws.asynchbeans.WorkWithExecutionContextImpl.go(WorkWithExecutionContextImpl.java:195)
>>
>> com.ibm.ws.asynchbeans.CJWorkItemImpl.run(CJWorkItemImpl.java:187)
>>
>> com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1469)
>>
>>
>>
>>
>> Best regards,
>>
>> Zhuran Li
>
>
>
>
>--
>*Christian Posta*
>http://www.christianposta.com/blog
>twitter: @christianposta
Re: ActiveMQ client app hold unexpectedly!!!
Posted by Christian Posta <ch...@gmail.com>.
This means the client didn't' receive a response from the broker.
Did the broker producer flow control this client? ie, was the destination
to which the client sending full?
On Tue, Jul 16, 2013 at 7:23 AM, lzr <js...@163.com> wrote:
> Dear all,
>
>
> I found a one of the ActiveMQ thread got stuck unexpectedly for several
> days. Customer was very angry to the ActiveMQ for the bad behavior. Did
> anybody experienced this issue? It would be much appreciated if any clue!!!
> My application is normal Java JMS application like the following:
> new ActiveMQConnectionFactory.
> create Connection;
> conn.start.
> create session.
> create destination, producer and message
> send messages
> commit;
> conn.close.
>
>
> The follows is the stack when I looked into it with debug tools:
>
> sun.misc.Unsafe.park(Native Method)
>
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:169)
>
>
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1787)
>
> java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:356)
>
>
> org.apache.activemq.transport.FutureResponse.getResult(FutureResponse.java:40)
>
>
> org.apache.activemq.transport.ResponseCorrelator.request(ResponseCorrelator.java:80)
>
>
> org.apache.activemq.ActiveMQConnection.syncSendPacket(ActiveMQConnection.java:1233)
>
>
> org.apache.activemq.ActiveMQConnection.ensureConnectionInfoSent(ActiveMQConnection.java:1339)
>
> org.apache.activemq.ActiveMQConnection.start(ActiveMQConnection.java:488)
>
> nc.bs.mqv2.mmgp.mq.UFMQProcessor.getConnection(UFMQProcessor.java:487)
>
> nc.bs.mqv2.mmgp.mq.UFMQProcessor.getConnection(UFMQProcessor.java:462)
>
>
> nc.bs.mqv2.mmgp.mq.UFMQProcessor.sendSerializedObjMsg(UFMQProcessor.java:128)
>
>
> nc.bs.mqv2.meam.mq.sendbasedatas.AbstractMeamBDSyncService.sendBaseData(AbstractMeamBDSyncService.java:64)
>
>
> nc.bs.mqv2.mes.bd.mq.basedoc.AbstractBDServerSendService.dosendDataProcess(AbstractBDServerSendService.java:213)
>
>
> nc.bs.mqv2.mes.bd.mq.basedoc.AbstractBDServerSendService.sendDatas(AbstractBDServerSendService.java:62)
>
>
> nc.impl.mqv2.mes.bd.mq.SyncSendServiceImpl.ncserverSendDatas(SyncSendServiceImpl.java:58)
>
> nc.gejb.meam.cmt.MeamEjbBean.ncserverSendDatas(MeamEjbBean.java:8741)
>
> nc.gejb.meam.cmt.EJSLocalStatelessnc_gejb_meam_cmt_Meam_b44269e7.ncserverSendDatas(Unknown
> Source)
>
> sun.reflect.GeneratedMethodAccessor447.invoke(Unknown Source)
>
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>
> java.lang.reflect.Method.invoke(Method.java:615)
>
>
> nc.bs.framework.ejb.AbstractEJBServiceHandler.invoke(AbstractEJBServiceHandler.java:56)
>
>
> nc.bs.framework.ejb.AbstractEJBServiceHandler.invoke(AbstractEJBServiceHandler.java:34)
>
> $Proxy58.ncserverSendDatas(Unknown Source)
>
>
> nc.bs.mqv2.meam.mq.task.DoSendBaseDataTaskUtil.runTask(DoSendBaseDataTaskUtil.java:63)
>
>
> nc.bs.mqv2.meam.mq.task.NCServerBaseDatasSendTask.executeTask(NCServerBaseDatasSendTask.java:26)
>
>
> nc.bs.pub.taskcenter.BgBizsPluginProcess.callBusinessPlugin(BgBizsPluginProcess.java:43)
>
> nc.bs.pub.pa.read.TaskBodyBase.callBizsPlugin(TaskBodyBase.java:188)
>
> nc.bs.pub.pa.read.TaskBodyBase.callBusinessPlugin(TaskBodyBase.java:174)
>
> nc.bs.pub.pa.read.TaskBodyBase.createMetaMsg(TaskBodyBase.java:95)
>
> nc.bs.pub.pa.read.TaskBodyBase.createAndSendMessage(TaskBodyBase.java:41)
>
> nc.bs.pub.taskcenter.BackgroundTaskBody.execute(BackgroundTaskBody.java:47)
>
>
> nc.impl.uap.scheduler.ExecutorServiceImpl.executeTask_RequiresNew(ExecutorServiceImpl.java:27)
>
>
> nc.itf.uap.mw.MwBizEJBEjbBean.executeTask_RequiresNew(MwBizEJBEjbBean.java:1682)
>
> nc.itf.uap.mw.EJSLocalStatelessnc_itf_uap_mw_MwBizEJB_f98e92a8.executeTask_RequiresNew(Unknown
> Source)
>
> sun.reflect.GeneratedMethodAccessor1036.invoke(Unknown Source)
>
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>
> java.lang.reflect.Method.invoke(Method.java:615)
>
>
> nc.bs.framework.ejb.AbstractEJBServiceHandler.invoke(AbstractEJBServiceHandler.java:56)
>
>
> nc.bs.framework.ejb.AbstractEJBServiceHandler.invoke(AbstractEJBServiceHandler.java:34)
>
> $Proxy39.executeTask_RequiresNew(Unknown Source)
>
> nc.bs.uap.scheduler.impl.TaskExecutor.executeTask(TaskExecutor.java:110)
>
> nc.bs.uap.scheduler.impl.TaskExecutor.run(TaskExecutor.java:75)
>
> nc.bs.framework.execute.Executor$1.run(Executor.java:80)
>
> nc.bs.framework.execute.commonj.impl.WorkAdaptor.run(WorkAdaptor.java:29)
>
> com.ibm.ws.asynchbeans.J2EEContext.run(J2EEContext.java:1114)
>
>
> com.ibm.ws.asynchbeans.WorkWithExecutionContextImpl.go(WorkWithExecutionContextImpl.java:195)
>
> com.ibm.ws.asynchbeans.CJWorkItemImpl.run(CJWorkItemImpl.java:187)
>
> com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1469)
>
>
>
>
> Best regards,
>
> Zhuran Li
--
*Christian Posta*
http://www.christianposta.com/blog
twitter: @christianposta