You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by "Mahadikar, Megaraj" <me...@sap.com> on 2019/03/01 05:00:26 UTC

JMS Exception : Cannot remove session from connection that had not been registered

Hi,

We are using ActiveMq in a master/slave configuration. When a failover of the broker is triggered, some of the clients experience a connection drop and would never recover back. On the client we see the below exceptions.

An problem has been detected with the JMS Connection.javax.jms.JMSException: Cannot remove session from connection that had not been registered: <connection-id>
                at org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:49)
                at org.apache.activemq.ActiveMQConnection.onAsyncException(ActiveMQConnection.java:1967)
                at org.apache.activemq.ActiveMQConnection$3$1.run(ActiveMQConnection.java:1887)
                at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)
                at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:622)
                at java.lang.Thread.run(Thread.java:807)
Caused by: java.lang.IllegalStateException: Cannot remove session from connection that had not been registered: ID: <connection-id>
                at org.apache.activemq.broker.TransportConnection.processRemoveSession(TransportConnection.java:683)
                at org.apache.activemq.command.RemoveInfo.visit(RemoveInfo.java:74)
                at org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:329)
                at org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:184)
                at org.apache.activemq.transport.MutexTransport.onCommand(MutexTransport.java:50)
                at org.apache.activemq.transport.TransportFilter.onCommand(TransportFilter.java:69)
                at org.apache.activemq.transport.WireFormatNegotiator.onCommand(WireFormatNegotiator.java:113)
                at org.apache.activemq.transport.AbstractInactivityMonitor.onCommand(AbstractInactivityMonitor.java:288)
                at org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:83)
                at org.apache.activemq.transport.nio.NIOSSLTransport.doConsume(NIOSSLTransport.java:449)
                at org.apache.activemq.transport.nio.NIOSSLTransport.processCommand(NIOSSLTransport.java:288)
                at org.apache.activemq.transport.nio.NIOSSLTransport.serviceRead(NIOSSLTransport.java:203)
                at org.apache.activemq.transport.nio.NIOSSLTransport$1.onSelect(NIOSSLTransport.java:158)
                at org.apache.activemq.transport.nio.SelectorSelection.onSelect(SelectorSelection.java:94)
                at org.apache.activemq.transport.nio.SelectorWorker$1.run(SelectorWorker.java:119)
                ... 3 common frames omitted

Has anyone faced a similar issue and what might be the cause for such an error.

Regards,
Megaraj

Re: JMS Exception : Cannot remove session from connection that had not been registered

Posted by "Songa, Harshavardhan Reddy" <ha...@sap.com>.
Hi Tim,

We are using an activeMq version 5.8.0. Both the clients and brokers are in the same versions. We are using NIOSSL protocol.
We have around 6000 clients, out of which around 50 clients were affected. We observed this only once and unfortunately we couldn’t reproduce this in any unit tests. Also we don’t observe this issue when the connections are relatively lower in number.

The client immediately failed to connect to the slave once a failover was triggered. There was no other exceptions prior to this.

Thanks & regards,
Harshavardhan S


From: Tim Bain <tb...@alumni.duke.edu>
Date: Saturday, 2 March 2019 at 11:01 PM
To: ActiveMQ Users <us...@activemq.apache.org>
Cc: "Songa, Harshavardhan Reddy" <ha...@sap.com>, "Jose, Jaison" <ja...@sap.com>
Subject: Re: JMS Exception : Cannot remove session from connection that had not been registered

Megaraj,

This sounds like another occurrence of https://issues.apache.org/jira/browse/AMQ-5788, which was closed because neither its reporter nor another commenter provided further information to help narrow down the problem. If you're able to provide enough information to narrow in on the root cause, we could get that issue re-opened for a fix.

First some basics. What version of the ActiveMQ client JAR are you using, and what version of the broker (if the versions are different)? Also, it looks from the stack trace like you're using the ssl protocol; can you confirm that?

Next, can you tell us how reproducible this is? You said "some of the clients" which sounds like maybe it's not 100% of the time, but is this 1 in 100, 1 in 2, etc.? The best case would be if it was 100% reproducible and you could reproduce it consistently in a unit test, but it sounds like we might not get that. But even if it's relatively consistently reproducible (1 in 5, for example), that might be enough for someone to go on.

Can you reproduce this (sometimes, at least) with only a single connected consumer, or does it require multiple consumers to manifest? If so, how many are needed? If someone other than you is going to attempt to debug this, that person will need to be able to reproduce the problem with a minimal configuration, so I hope that you'll be able to help cut out any configuration aspects that are more complicated (e.g. multiple consumers, if it's still reproducible with just one) that will make it more complicated for someone else to reproduce.

Finally, what can you tell us about the behavior immediately before the failure of those clients that failed relative to those that didn't? Were there previous errors or warnings in the logs for those consumers that might be relative? Are the ones that fail always doing something (e.g. consumers only fail when they connected within 1 second of the failure) that the others aren't? Is there anything else that might help us narrow in on where to look for the root cause?

Thanks,
Tim


On Fri, Mar 1, 2019 at 7:23 AM Mahadikar, Megaraj <me...@sap.com>> wrote:
Hi,

We are using ActiveMq in a master/slave configuration. When a failover of the broker is triggered, some of the clients experience a connection drop and would never recover back. On the client we see the below exceptions.

An problem has been detected with the JMS Connection.javax.jms.JMSException: Cannot remove session from connection that had not been registered: <connection-id>
                at org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:49)
                at org.apache.activemq.ActiveMQConnection.onAsyncException(ActiveMQConnection.java:1967)
                at org.apache.activemq.ActiveMQConnection$3$1.run(ActiveMQConnection.java:1887)
                at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)
                at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:622)
                at java.lang.Thread.run(Thread.java:807)
Caused by: java.lang.IllegalStateException: Cannot remove session from connection that had not been registered: ID: <connection-id>
                at org.apache.activemq.broker.TransportConnection.processRemoveSession(TransportConnection.java:683)
                at org.apache.activemq.command.RemoveInfo.visit(RemoveInfo.java:74)
                at org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:329)
                at org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:184)
                at org.apache.activemq.transport.MutexTransport.onCommand(MutexTransport.java:50)
                at org.apache.activemq.transport.TransportFilter.onCommand(TransportFilter.java:69)
                at org.apache.activemq.transport.WireFormatNegotiator.onCommand(WireFormatNegotiator.java:113)
                at org.apache.activemq.transport.AbstractInactivityMonitor.onCommand(AbstractInactivityMonitor.java:288)
                at org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:83)
                at org.apache.activemq.transport.nio.NIOSSLTransport.doConsume(NIOSSLTransport.java:449)
                at org.apache.activemq.transport.nio.NIOSSLTransport.processCommand(NIOSSLTransport.java:288)
                at org.apache.activemq.transport.nio.NIOSSLTransport.serviceRead(NIOSSLTransport.java:203)
                at org.apache.activemq.transport.nio.NIOSSLTransport$1.onSelect(NIOSSLTransport.java:158)
                at org.apache.activemq.transport.nio.SelectorSelection.onSelect(SelectorSelection.java:94)
                at org.apache.activemq.transport.nio.SelectorWorker$1.run(SelectorWorker.java:119)
                ... 3 common frames omitted

Has anyone faced a similar issue and what might be the cause for such an error.

Regards,
Megaraj

Re: JMS Exception : Cannot remove session from connection that had not been registered

Posted by Tim Bain <tb...@alumni.duke.edu>.
Megaraj,

This sounds like another occurrence of
https://issues.apache.org/jira/browse/AMQ-5788, which was closed because
neither its reporter nor another commenter provided further information to
help narrow down the problem. If you're able to provide enough information
to narrow in on the root cause, we could get that issue re-opened for a fix.

First some basics. What version of the ActiveMQ client JAR are you using,
and what version of the broker (if the versions are different)? Also, it
looks from the stack trace like you're using the ssl protocol; can you
confirm that?

Next, can you tell us how reproducible this is? You said "some of the
clients" which sounds like maybe it's not 100% of the time, but is this 1
in 100, 1 in 2, etc.? The best case would be if it was 100% reproducible
and you could reproduce it consistently in a unit test, but it sounds like
we might not get that. But even if it's relatively consistently
reproducible (1 in 5, for example), that might be enough for someone to go
on.

Can you reproduce this (sometimes, at least) with only a single connected
consumer, or does it require multiple consumers to manifest? If so, how
many are needed? If someone other than you is going to attempt to debug
this, that person will need to be able to reproduce the problem with a
minimal configuration, so I hope that you'll be able to help cut out any
configuration aspects that are more complicated (e.g. multiple consumers,
if it's still reproducible with just one) that will make it more
complicated for someone else to reproduce.

Finally, what can you tell us about the behavior immediately before the
failure of those clients that failed relative to those that didn't? Were
there previous errors or warnings in the logs for those consumers that
might be relative? Are the ones that fail always doing something (e.g.
consumers only fail when they connected within 1 second of the failure)
that the others aren't? Is there anything else that might help us narrow in
on where to look for the root cause?

Thanks,
Tim


On Fri, Mar 1, 2019 at 7:23 AM Mahadikar, Megaraj <me...@sap.com>
wrote:

> Hi,
>
> We are using ActiveMq in a master/slave configuration. When a failover of
> the broker is triggered, some of the clients experience a connection drop
> and would never recover back. On the client we see the below exceptions.
>
> An problem has been detected with the JMS
> Connection.javax.jms.JMSException: Cannot remove session from connection
> that had not been registered: <connection-id>
>                 at
> org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:49)
>                 at
> org.apache.activemq.ActiveMQConnection.onAsyncException(ActiveMQConnection.java:1967)
>                 at
> org.apache.activemq.ActiveMQConnection$3$1.run(ActiveMQConnection.java:1887)
>                 at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)
>                 at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:622)
>                 at java.lang.Thread.run(Thread.java:807)
> Caused by: java.lang.IllegalStateException: Cannot remove session from
> connection that had not been registered: ID: <connection-id>
>                 at
> org.apache.activemq.broker.TransportConnection.processRemoveSession(TransportConnection.java:683)
>                 at
> org.apache.activemq.command.RemoveInfo.visit(RemoveInfo.java:74)
>                 at
> org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:329)
>                 at
> org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:184)
>                 at
> org.apache.activemq.transport.MutexTransport.onCommand(MutexTransport.java:50)
>                 at
> org.apache.activemq.transport.TransportFilter.onCommand(TransportFilter.java:69)
>                 at
> org.apache.activemq.transport.WireFormatNegotiator.onCommand(WireFormatNegotiator.java:113)
>                 at
> org.apache.activemq.transport.AbstractInactivityMonitor.onCommand(AbstractInactivityMonitor.java:288)
>                 at
> org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:83)
>                 at
> org.apache.activemq.transport.nio.NIOSSLTransport.doConsume(NIOSSLTransport.java:449)
>                 at
> org.apache.activemq.transport.nio.NIOSSLTransport.processCommand(NIOSSLTransport.java:288)
>                 at
> org.apache.activemq.transport.nio.NIOSSLTransport.serviceRead(NIOSSLTransport.java:203)
>                 at
> org.apache.activemq.transport.nio.NIOSSLTransport$1.onSelect(NIOSSLTransport.java:158)
>                 at
> org.apache.activemq.transport.nio.SelectorSelection.onSelect(SelectorSelection.java:94)
>                 at
> org.apache.activemq.transport.nio.SelectorWorker$1.run(SelectorWorker.java:119)
>                 ... 3 common frames omitted
>
> Has anyone faced a similar issue and what might be the cause for such an
> error.
>
> Regards,
> Megaraj
>
>