[jira] [Created] (AMQ-9051) Ghost connections in the broker holding messages

["Lionel Cons (Jira)" <ji...@apache.org>]

Lionel Cons created AMQ-9051:
--------------------------------

             Summary: Ghost connections in the broker holding messages
                 Key: AMQ-9051
                 URL: https://issues.apache.org/jira/browse/AMQ-9051
             Project: ActiveMQ
          Issue Type: Bug
    Affects Versions: 5.16.5, 5.16.4
         Environment: CentOS 7 and Java 8.
            Reporter: Lionel Cons


ActiveMQ can have "ghost connections". These are connections effectively closed (as confirmed by the operating system) but still appearing in ActiveMQ.

Here is an example.

Subscription object as reported by JMX:
{code:java}
   "org.apache.activemq:brokerName=agileinf_broker.acme.org,clientId=ID_broker.acme.org-21914-1645625556236-3_3056814,consumerId=ID_broker.acme.org-21914-1645625556236-3_3056814_-1_1,destinationName=queue.name,destinationType=Queue,endpoint=Consumer,type=Broker" : {
      "Active" : true,
      "ClientId" : "ID:broker.acme.org-21914-1645625556236-3:3056814",
      "Connection" : {
         "objectName" : "org.apache.activemq:brokerName=agileinf_broker.acme.org,connectionName=ID_broker.acme.org-21914-1645625556236-3_3056814,connectionViewType=clientId,connector=clientConnectors,connectorName=stomp,type=Broker"
      },
      "ConnectionId" : "ID:broker.acme.org-21914-1645625556236-3:3056814",
      "ConsumedCount" : 0,
      "DequeueCounter" : 496,
      "DestinationName" : "queue.name",
      "DestinationQueue" : true,
      "DestinationTemporary" : false,
      "DestinationTopic" : false,
      "DispatchAsync" : true,
      "DispatchedCounter" : 548,
      "DispatchedQueueSize" : 52,
      "Durable" : false,
      "EnqueueCounter" : 549,
      "Exclusive" : false,
      "MaximumPendingMessageLimit" : 0,
      "MessageCountAwaitingAcknowledge" : 52,
      "Network" : false,
      "NoLocal" : false,
      "PendingQueueSize" : 1,
      "PrefetchSize" : 563,
      "Priority" : 0,
      "Retroactive" : false,
      "Selector" : null,
      "SessionId" : -1,
      "SlowConsumer" : false,
      "SubscriptionId" : 1,
      "SubscriptionName" : null,
      "UserName" : null
   },
{code}
Connection object as reported by JMX:
{code:java}
   "org.apache.activemq:brokerName=agileinf_broker.acme.org,connectionName=ID_broker.acme.org-21914-1645625556236-3_3056814,connectionViewType=clientId,connector=clientConnectors,connectorName=stomp,type=Broker" : {
      "Active" : true,
      "ActiveTransactionCount" : 0,
      "Blocked" : false,
      "ClientId" : "ID:broker.acme.org-21914-1645625556236-3:3056814",
      "Connected" : true,
      "Consumers" : [
         {
            "objectName" : "org.apache.activemq:brokerName=agileinf_broker.acme.org,clientId=ID_broker.acme.org-21914-1645625556236-3_3056814,consumerId=ID_broker.acme.org-21914-1645625556236-3_3056814_-1_1,destinationName=queue.name,destinationType=Queue,endpoint=Consumer,type=Broker"
         }
      ],
      "DispatchQueueSize" : 0,
      "OldestActiveTransactionDuration" : null,
      "Producers" : [],
      "RemoteAddress" : "tcp://192.168.27.159:44730",
      "Slow" : false,
      "UserName" : null
   },
{code}
Yet system tools like {{lsof}} and {{netstat}} show no matching peer ({{{}192.168.27.159:44730{}}}). The peer itself shows no connections to the broker.

These ghost connections sometimes have pending messages that are therefore stuck on the broker.

The connections are using STOMP, with heart-beats enabled.

These ghost connections are not transient. They are not cleaned by the broker (although they should wrt heart-beats).

This happens on both 5.16.4 and 5.16.5. Other ActiveMQ versions have not been tested.

We haven't found a way to reproduce the problem. It just happens for one messaging application, after some time.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)