You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@karaf.apache.org by Jörg Jansen <Jo...@inform-software.com> on 2021/09/22 18:29:48 UTC

RE: Shutdown issue ActiveMq

Hi JB,

any update?

Thanks,
Joerg

-----Original Message-----
From: JB Onofré <jb...@nanthrax.net> 
Sent: Montag, 16. August 2021 08:27
To: user@karaf.apache.org
Subject: Re: Shutdown issue ActiveMq

Hi

I will take a look with the resources you sent on your last email. 
No need a jira for now, we will create one if the issue is confirmed. 

Regards
JB

> Le 16 août 2021 à 08:25, Jörg Jansen <Jo...@inform-software.com> a écrit :
> 
> Good morning,
> 
> I would like to bring up this point once again and kindly ask if anybody has an idea, of a possible cause of this issue?
> Should I raise a JIRA ticket?
> 
> Kind regards,
> Joerg
> 
> -----Original Message-----
> From: Jörg Jansen
> Sent: Montag, 5. Juli 2021 10:47
> To: 'user@karaf.apache.org' <us...@karaf.apache.org>
> Subject: RE: Shutdown issue ActiveMq
> 
> Hi JB,
> 
> I just pushed an example to github, which shows the configuration and 
> a reproduction guide: https://github.com/jojansen/jms-shutdown.git
> Maybe you have time to look into it? 
> 
> Regards,
> Joerg
> 
> -----Original Message-----
> From: Jörg Jansen
> Sent: Freitag, 2. Juli 2021 15:02
> To: user@karaf.apache.org
> Subject: RE: Shutdown issue ActiveMq
> 
> Sure,
> 
> the feature looks like: 
> <feature name="gs-lisa-activemq" version="${project.version}" description="Provide JMS connection factory and ActiveMqBroker">
>    <feature>jms</feature>
>    <feature version="[4,5)">spring</feature>
>    <feature version="${activemq.version}">activemq-broker-noweb</feature>
>    <feature version="${activemq.version}">activemq-client</feature>
>    <feature version="${ops4j.pax.jms.version}">pax-jms-pool-pooledjms</feature>
>    <feature version="${ops4j.pax.jms.version}">pax-jms-activemq</feature>
>    <feature version="${camel.version}">camel-jms</feature>
> 
>    <config name="org.ops4j.connectionfactory-lisa-amq">
>      name = lisa-amq
>      osgi.jndi.service.name = jms/lisa-amq
>      password = ${activemq.jms.password}
>      connectionFactoryType = ConnectionFactory
>      type = activemq
>      url = ${activemq.url}
>      user = ${activemq.jms.user}
> 
>      # hints for pax-jms-config to use selected org.ops4j.pax.jms.service.PooledConnectionFactoryFactory
>      pool = pooledjms
>      xa = false
> 
>      # pooled-jms specific configuration of org.messaginghub.pooled.jms.JmsPoolConnectionFactory
>      pool.idleTimeout = 10
>      pool.maxConnections = 10
>      pool.blockIfSessionPoolIsFull = true
>      pax.jms.managed = true
>    </config>
>  </feature>
> 
> Thanks and regards
> Joerg
> 
> -----Original Message-----
> From: Jean-Baptiste Onofre <jb...@nanthrax.net>
> Sent: Freitag, 2. Juli 2021 14:53
> To: user@karaf.apache.org
> Subject: Re: Shutdown issue ActiveMq
> 
> It should be fine if your feature installed pax-jms config has prerequisite.
> 
> Can you share snippet of your feature referencing activemq-broker ?
> 
> Regards
> JB
> 
>> Le 2 juil. 2021 à 14:21, Jörg Jansen <Jo...@inform-software.com> a écrit :
>> 
>> It's installed as a prerequisite feature (which is referenced by my customized boot feature). 
>> 
>> Regards,
>> Joerg
>> 
>> -----Original Message-----
>> From: Jean-Baptiste Onofre <jb...@nanthrax.net>
>> Sent: Freitag, 2. Juli 2021 14:07
>> To: user@karaf.apache.org
>> Subject: Re: Shutdown issue ActiveMq
>> 
>> Ah the broker is installed as a feature in Karaf ? I thought your broker was outside of Karaf (standalone).
>> 
>> Does activemq-broker feature installed as boot feature ? Or do you install after startup using feature:install ?
>> 
>> Regards
>> JB
>> 
>>>> Le 2 juil. 2021 à 11:14, Jörg Jansen <Jo...@inform-software.com> a écrit :
>>> 
>>> Hi JB,
>>> 
>>> I'm using the AUTO ack, but switching to CLIENT doesn't make any difference. 
>>> How can I change the AMQ pool?
>>> From my understanding it should be the preferred way to use pooledjms.
>>> 
>>> This behavior sounds to me, that it may be be a ordering problem in my assembly. 
>>> The broker should be shutdown after all cliens. 
>>> 
>>> Regards,
>>> Joerg
>>> 
>>> -----Original Message-----
>>> From: Jean-Baptiste Onofre <jb...@nanthrax.net>
>>> Sent: Freitag, 2. Juli 2021 09:52
>>> To: user <us...@karaf.apache.org>
>>> Subject: Re: Shutdown issue ActiveMq
>>> 
>>> Hi,
>>> 
>>> Did you try with another kind of pooler ? Like "native" ActiveMQ pool instead of wrapped pooledjms ?
>>> 
>>> What kind of ack do you use in your Camel route ? AUTO (default) or CLIENT ?
>>> 
>>> Regards
>>> JB
>>> 
>>>> Le 2 juil. 2021 à 08:05, Jörg Jansen <Jo...@inform-software.com> a écrit :
>>>> 
>>>> Hi everybody,
>>>> 
>>>> I'm facing a problem with our JMS connection, when shutting down the karaf container. 
>>>> The JMS connectionFactory is provided/configured via ops4j-jms.
>>>> When shutting down karaf, I receive the message: "Caught exception trying rollback() when putting session back into the pool, will invalidate. javax.jms.IllegalStateException: The Session is closed".
>>>> 
>>>> Does anybody have an idea how to fix it? 
>>>> Seaching for a workaround brings up to set the idleTimeout to 0, but this has no effect. 
>>>> 
>>>> The currently used versions are: 
>>>> Karaf: 4.3.2
>>>> ActiveMq: 5.16.0
>>>> Camel: 3.7.4
>>>> 
>>>> Any help would be really appreciated.
>>>> 
>>>> Additional details are available below.
>>>> 
>>>> Thanks in advance,
>>>> Joerg
>>>> 
>>>> 
>>>> *******************************************************************
>>>> *
>>>> Stacktrace
>>>> ********************************************************************
>>>> 2021-07-02T07:46:30,260 | WARN  | Camel (gs-os-connector) thread #22 - JmsConsumer[AMQ.GSLISA.OS.BUFFERED-SPX] | JmsPoolSession                   | 213 - org.messaginghub.pooled.jms - 1.2.1 | Caught exception trying rollback() when putting session back into the pool, will invalidate. javax.jms.IllegalStateException: The Session is closed
>>>> javax.jms.IllegalStateException: The Session is closed
>>>>    at org.apache.activemq.ActiveMQSession.checkClosed(ActiveMQSession.java:772) ~[?:?]
>>>>    at org.apache.activemq.ActiveMQSession.rollback(ActiveMQSession.java:597) ~[?:?]
>>>>    at org.messaginghub.pooled.jms.JmsPoolSession.close(JmsPoolSession.java:112) [!/:?]
>>>>    at org.springframework.jms.support.JmsUtils.closeSession(JmsUtils.java:109) [!/:?]
>>>>    at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.clearResources(DefaultMessageListenerContainer.java:1289) [!/:?]
>>>>    at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:1135) [!/:?]
>>>>    at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) [?:?]
>>>>    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) [?:?]
>>>>    at java.lang.Thread.run(Unknown Source) [?:?]
>>>> 
>>>> 
>>>> *******************************************************************
>>>> *
>>>> Feature config
>>>> *******************************************************************
>>>> * <feature name="gs-lisa-activemq" version="${project.version}"
>>>> description="Provide JMS connection factory and ActiveMqBroker" > 
>>>> <feature>jms</feature>  <feature version="[4,5)">spring</feature> 
>>>> <feature 
>>>> version="${activemq.version}">activemq-broker-noweb</feature>
>>>> <feature version="${activemq.version}">activemq-client</feature>
>>>> <feature
>>>> version="${ops4j.pax.jms.version}">pax-jms-pool-pooledjms</feature>
>>>> <feature
>>>> version="${ops4j.pax.jms.version}">pax-jms-activemq</feature>
>>>> <feature version="${camel.version}">camel-jms</feature>
>>>> 
>>>> <config name="org.ops4j.connectionfactory-lisa-amq">
>>>> name = lisa-amq
>>>> osgi.jndi.service.name = jms/lisa-amq  password = 
>>>> ${activemq.jms.password}  connectionFactoryType = ConnectionFactory 
>>>> type = activemq  url = ${activemq.url}  user = ${activemq.jms.user}
>>>> 
>>>> # hints for pax-jms-config to use selected 
>>>> org.ops4j.pax.jms.service.PooledConnectionFactoryFactory
>>>> pool = pooledjms
>>>> xa = false
>>>> 
>>>> # pooled-jms specific configuration of 
>>>> org.messaginghub.pooled.jms.JmsPoolConnectionFactory
>>>> pool.idleTimeout = 10
>>>> pool.maxConnections = 10
>>>> pool.blockIfSessionPoolIsFull = true  pax.jms.managed = true 
>>>> </config> </feature>
>>>> 
>>>> 
>>> 
>> 
> 


Re: Shutdown issue ActiveMq

Posted by Łukasz Dywicki <lu...@code-house.org>.
Hey Jorg,
I am not too familiar with pax-jms, I do recall some of spring-jms 
troubles in the past (especially camel).
The big question is if your components actually do have an option for 
graceful shutdown. What you observe might be a very basic.

Looking at the sequence of actions you can see two paths of execution:
1. Container shutdown announced
2. Pool shutdown
3. Connection shutdown

1. Container shutdown announced
2. Component shutdown
3. Last will message
4. Connection close

If your only link between pool and component is connection itself you 
will observe a lot of such troubles. In order to do such you need to 
bind state of pool and client so they work together.
While shutdown of client should not cause shutdown of pool the reversed 
scenario is true. Also blueprint references might not be "up" at the 
time camel context is still awake.

A very basic workaround for that is building up a set of listeners ie. 
listening if connection factory is up in your client and vice versa.
Reverse listening (pool for clients) will help you to make graceful 
shutdown.

Hope that above puts some light on topic.

Best,
Łukasz
--
Code-House
http://code-house.org


On 22.09.2021 20:29, Jörg Jansen wrote:
> Hi JB,
> 
> any update?
> 
> Thanks,
> Joerg
> 
> -----Original Message-----
> From: JB Onofré <jb...@nanthrax.net>
> Sent: Montag, 16. August 2021 08:27
> To: user@karaf.apache.org
> Subject: Re: Shutdown issue ActiveMq
> 
> Hi
> 
> I will take a look with the resources you sent on your last email.
> No need a jira for now, we will create one if the issue is confirmed.
> 
> Regards
> JB
> 
>> Le 16 août 2021 à 08:25, Jörg Jansen <Jo...@inform-software.com> a écrit :
>>
>> Good morning,
>>
>> I would like to bring up this point once again and kindly ask if anybody has an idea, of a possible cause of this issue?
>> Should I raise a JIRA ticket?
>>
>> Kind regards,
>> Joerg
>>
>> -----Original Message-----
>> From: Jörg Jansen
>> Sent: Montag, 5. Juli 2021 10:47
>> To: 'user@karaf.apache.org' <us...@karaf.apache.org>
>> Subject: RE: Shutdown issue ActiveMq
>>
>> Hi JB,
>>
>> I just pushed an example to github, which shows the configuration and
>> a reproduction guide: https://github.com/jojansen/jms-shutdown.git
>> Maybe you have time to look into it?
>>
>> Regards,
>> Joerg
>>
>> -----Original Message-----
>> From: Jörg Jansen
>> Sent: Freitag, 2. Juli 2021 15:02
>> To: user@karaf.apache.org
>> Subject: RE: Shutdown issue ActiveMq
>>
>> Sure,
>>
>> the feature looks like:
>> <feature name="gs-lisa-activemq" version="${project.version}" description="Provide JMS connection factory and ActiveMqBroker">
>>     <feature>jms</feature>
>>     <feature version="[4,5)">spring</feature>
>>     <feature version="${activemq.version}">activemq-broker-noweb</feature>
>>     <feature version="${activemq.version}">activemq-client</feature>
>>     <feature version="${ops4j.pax.jms.version}">pax-jms-pool-pooledjms</feature>
>>     <feature version="${ops4j.pax.jms.version}">pax-jms-activemq</feature>
>>     <feature version="${camel.version}">camel-jms</feature>
>>
>>     <config name="org.ops4j.connectionfactory-lisa-amq">
>>       name = lisa-amq
>>       osgi.jndi.service.name = jms/lisa-amq
>>       password = ${activemq.jms.password}
>>       connectionFactoryType = ConnectionFactory
>>       type = activemq
>>       url = ${activemq.url}
>>       user = ${activemq.jms.user}
>>
>>       # hints for pax-jms-config to use selected org.ops4j.pax.jms.service.PooledConnectionFactoryFactory
>>       pool = pooledjms
>>       xa = false
>>
>>       # pooled-jms specific configuration of org.messaginghub.pooled.jms.JmsPoolConnectionFactory
>>       pool.idleTimeout = 10
>>       pool.maxConnections = 10
>>       pool.blockIfSessionPoolIsFull = true
>>       pax.jms.managed = true
>>     </config>
>>   </feature>
>>
>> Thanks and regards
>> Joerg
>>
>> -----Original Message-----
>> From: Jean-Baptiste Onofre <jb...@nanthrax.net>
>> Sent: Freitag, 2. Juli 2021 14:53
>> To: user@karaf.apache.org
>> Subject: Re: Shutdown issue ActiveMq
>>
>> It should be fine if your feature installed pax-jms config has prerequisite.
>>
>> Can you share snippet of your feature referencing activemq-broker ?
>>
>> Regards
>> JB
>>
>>> Le 2 juil. 2021 à 14:21, Jörg Jansen <Jo...@inform-software.com> a écrit :
>>>
>>> It's installed as a prerequisite feature (which is referenced by my customized boot feature).
>>>
>>> Regards,
>>> Joerg
>>>
>>> -----Original Message-----
>>> From: Jean-Baptiste Onofre <jb...@nanthrax.net>
>>> Sent: Freitag, 2. Juli 2021 14:07
>>> To: user@karaf.apache.org
>>> Subject: Re: Shutdown issue ActiveMq
>>>
>>> Ah the broker is installed as a feature in Karaf ? I thought your broker was outside of Karaf (standalone).
>>>
>>> Does activemq-broker feature installed as boot feature ? Or do you install after startup using feature:install ?
>>>
>>> Regards
>>> JB
>>>
>>>>> Le 2 juil. 2021 à 11:14, Jörg Jansen <Jo...@inform-software.com> a écrit :
>>>>
>>>> Hi JB,
>>>>
>>>> I'm using the AUTO ack, but switching to CLIENT doesn't make any difference.
>>>> How can I change the AMQ pool?
>>>>  From my understanding it should be the preferred way to use pooledjms.
>>>>
>>>> This behavior sounds to me, that it may be be a ordering problem in my assembly.
>>>> The broker should be shutdown after all cliens.
>>>>
>>>> Regards,
>>>> Joerg
>>>>
>>>> -----Original Message-----
>>>> From: Jean-Baptiste Onofre <jb...@nanthrax.net>
>>>> Sent: Freitag, 2. Juli 2021 09:52
>>>> To: user <us...@karaf.apache.org>
>>>> Subject: Re: Shutdown issue ActiveMq
>>>>
>>>> Hi,
>>>>
>>>> Did you try with another kind of pooler ? Like "native" ActiveMQ pool instead of wrapped pooledjms ?
>>>>
>>>> What kind of ack do you use in your Camel route ? AUTO (default) or CLIENT ?
>>>>
>>>> Regards
>>>> JB
>>>>
>>>>> Le 2 juil. 2021 à 08:05, Jörg Jansen <Jo...@inform-software.com> a écrit :
>>>>>
>>>>> Hi everybody,
>>>>>
>>>>> I'm facing a problem with our JMS connection, when shutting down the karaf container.
>>>>> The JMS connectionFactory is provided/configured via ops4j-jms.
>>>>> When shutting down karaf, I receive the message: "Caught exception trying rollback() when putting session back into the pool, will invalidate. javax.jms.IllegalStateException: The Session is closed".
>>>>>
>>>>> Does anybody have an idea how to fix it?
>>>>> Seaching for a workaround brings up to set the idleTimeout to 0, but this has no effect.
>>>>>
>>>>> The currently used versions are:
>>>>> Karaf: 4.3.2
>>>>> ActiveMq: 5.16.0
>>>>> Camel: 3.7.4
>>>>>
>>>>> Any help would be really appreciated.
>>>>>
>>>>> Additional details are available below.
>>>>>
>>>>> Thanks in advance,
>>>>> Joerg
>>>>>
>>>>>
>>>>> *******************************************************************
>>>>> *
>>>>> Stacktrace
>>>>> ********************************************************************
>>>>> 2021-07-02T07:46:30,260 | WARN  | Camel (gs-os-connector) thread #22 - JmsConsumer[AMQ.GSLISA.OS.BUFFERED-SPX] | JmsPoolSession                   | 213 - org.messaginghub.pooled.jms - 1.2.1 | Caught exception trying rollback() when putting session back into the pool, will invalidate. javax.jms.IllegalStateException: The Session is closed
>>>>> javax.jms.IllegalStateException: The Session is closed
>>>>>     at org.apache.activemq.ActiveMQSession.checkClosed(ActiveMQSession.java:772) ~[?:?]
>>>>>     at org.apache.activemq.ActiveMQSession.rollback(ActiveMQSession.java:597) ~[?:?]
>>>>>     at org.messaginghub.pooled.jms.JmsPoolSession.close(JmsPoolSession.java:112) [!/:?]
>>>>>     at org.springframework.jms.support.JmsUtils.closeSession(JmsUtils.java:109) [!/:?]
>>>>>     at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.clearResources(DefaultMessageListenerContainer.java:1289) [!/:?]
>>>>>     at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:1135) [!/:?]
>>>>>     at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) [?:?]
>>>>>     at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) [?:?]
>>>>>     at java.lang.Thread.run(Unknown Source) [?:?]
>>>>>
>>>>>
>>>>> *******************************************************************
>>>>> *
>>>>> Feature config
>>>>> *******************************************************************
>>>>> * <feature name="gs-lisa-activemq" version="${project.version}"
>>>>> description="Provide JMS connection factory and ActiveMqBroker" >
>>>>> <feature>jms</feature>  <feature version="[4,5)">spring</feature>
>>>>> <feature
>>>>> version="${activemq.version}">activemq-broker-noweb</feature>
>>>>> <feature version="${activemq.version}">activemq-client</feature>
>>>>> <feature
>>>>> version="${ops4j.pax.jms.version}">pax-jms-pool-pooledjms</feature>
>>>>> <feature
>>>>> version="${ops4j.pax.jms.version}">pax-jms-activemq</feature>
>>>>> <feature version="${camel.version}">camel-jms</feature>
>>>>>
>>>>> <config name="org.ops4j.connectionfactory-lisa-amq">
>>>>> name = lisa-amq
>>>>> osgi.jndi.service.name = jms/lisa-amq  password =
>>>>> ${activemq.jms.password}  connectionFactoryType = ConnectionFactory
>>>>> type = activemq  url = ${activemq.url}  user = ${activemq.jms.user}
>>>>>
>>>>> # hints for pax-jms-config to use selected
>>>>> org.ops4j.pax.jms.service.PooledConnectionFactoryFactory
>>>>> pool = pooledjms
>>>>> xa = false
>>>>>
>>>>> # pooled-jms specific configuration of
>>>>> org.messaginghub.pooled.jms.JmsPoolConnectionFactory
>>>>> pool.idleTimeout = 10
>>>>> pool.maxConnections = 10
>>>>> pool.blockIfSessionPoolIsFull = true  pax.jms.managed = true
>>>>> </config> </feature>
>>>>>
>>>>>
>>>>
>>>
>>
>