You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by Paul Burgess <PB...@PeerSoftware.com> on 2023/09/29 12:41:40 UTC
Network connectors issue on link failure
Hello,
I have a scenario where messages are failing to propagate over a network connector between two brokers.
My scenario has been simplified for the purpose of diagnosis. I have 2 brokers with 2 network connectors between them going one way, utilising two physical connections to achieve both performance AND resilience. Each broker has several consumers connected but there is only 1 producer. All messages occur on the same topic with selectors:
┌────────┐ ┌────────┐
│Consumer│ │Consumer│
└───────┬┘ └──┬─────┘
│ │
│ │
┌─┴──────┐ NC1 ┌───────┴┐
┌────────┐ │ ├──────────────────►│ │ ┌────────┐
│Producer├───────┤Broker1 │ │Broker2 ├──────┤Consumer│
└────────┘ │ ├──────────────────►│ │ └────────┘
└─┬──────┘ NC2 └───────┬┘
│ │
│ │
│ │
┌───────┴┐ ┌──┴─────┐
│Consumer│ │Consumer│
└────────┘ └────────┘
The error occurs when one of the network connectors fail. When this happens, consumers connected to Broker2 stop receiving messages, sometimes all messages, sometimes just a subset.
Note: If I restart a consumer connected to Broker2 that is set to receive all messages, then everything starts working again.
I ran this scenario on both 5.17.5 and 5.18.2.
Below is my activemq.xml for Broker1:
<beans
xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
http://activemq.apache.org/schema/core http://activemq.apache.org/schema/core/activemq-core.xsd">
<broker xmlns="http://activemq.apache.org/schema/core" start="false" brokerName="localhost" dataDirectory="${activemq.data}" schedulerSupport="false" persistent="true" >
<transportConnectors>
<transportConnector name="nio" uri="nio://0.0.0.0:61616"/>
</transportConnectors>
<networkConnectors>
<networkConnector name="UK-WindowsPMC1-USCentral-WindowsBroker1-BR0-000" consumerPriorityBase="-5" decreaseNetworkConsumerPriority="true" networkTTL="1" conduitSubscriptions="false" uri="static:(tcp://10.10.10.3:61616)"></networkConnector>
<networkConnector name="UK-WindowsPMC1-USCentral-WindowsBroker1-BR1-000" consumerPriorityBase="-5" decreaseNetworkConsumerPriority="true" networkTTL="1" conduitSubscriptions="false" uri="static:(tcp://10.0.0.3:61616)"></networkConnector>
</networkConnectors>
</broker>
</beans>
The configuration for Broker2 is identical, but without the network connectors.
I believe that this setup should work and that if one of the network connectors fail the other one should take over. Is this not the case? Is there something that I'm missing?
Many thanks
Paul
Re: Network connectors issue on link failure
Posted by Matt Pavlovich <ma...@gmail.com>.
Hi Paul-
Message flow and cluster troubleshooting is generally beyond what is feasible to support via mailing list.
Check out ActiveMQ support options, or try to jump into the #activemq Slack channel (I sent you an invite).
Thanks,
Matt Pavlovich
> On Sep 29, 2023, at 7:41 AM, Paul Burgess <PB...@PeerSoftware.com> wrote:
>
> Hello,
>
> I have a scenario where messages are failing to propagate over a network connector between two brokers.
>
> My scenario has been simplified for the purpose of diagnosis. I have 2 brokers with 2 network connectors between them going one way, utilising two physical connections to achieve both performance AND resilience. Each broker has several consumers connected but there is only 1 producer. All messages occur on the same topic with selectors:
>
> ┌────────┐ ┌────────┐
> │Consumer│ │Consumer│
> └───────┬┘ └──┬─────┘
> │ │
> │ │
> ┌─┴──────┐ NC1 ┌───────┴┐
> ┌────────┐ │ ├──────────────────►│ │ ┌────────┐
> │Producer├───────┤Broker1 │ │Broker2 ├──────┤Consumer│
> └────────┘ │ ├──────────────────►│ │ └────────┘
> └─┬──────┘ NC2 └───────┬┘
> │ │
> │ │
> │ │
> ┌───────┴┐ ┌──┴─────┐
> │Consumer│ │Consumer│
> └────────┘ └────────┘
>
> The error occurs when one of the network connectors fail. When this happens, consumers connected to Broker2 stop receiving messages, sometimes all messages, sometimes just a subset.
>
> Note: If I restart a consumer connected to Broker2 that is set to receive all messages, then everything starts working again.
>
> I ran this scenario on both 5.17.5 and 5.18.2.
>
> Below is my activemq.xml for Broker1:
>
> <beans
> xmlns="http://www.springframework.org/schema/beans"
> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
> xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
> http://activemq.apache.org/schema/core http://activemq.apache.org/schema/core/activemq-core.xsd">
>
> <broker xmlns="http://activemq.apache.org/schema/core" start="false" brokerName="localhost" dataDirectory="${activemq.data}" schedulerSupport="false" persistent="true" >
>
> <transportConnectors>
> <transportConnector name="nio" uri="nio://0.0.0.0:61616"/>
> </transportConnectors>
>
> <networkConnectors>
> <networkConnector name="UK-WindowsPMC1-USCentral-WindowsBroker1-BR0-000" consumerPriorityBase="-5" decreaseNetworkConsumerPriority="true" networkTTL="1" conduitSubscriptions="false" uri="static:(tcp://10.10.10.3:61616)"></networkConnector>
> <networkConnector name="UK-WindowsPMC1-USCentral-WindowsBroker1-BR1-000" consumerPriorityBase="-5" decreaseNetworkConsumerPriority="true" networkTTL="1" conduitSubscriptions="false" uri="static:(tcp://10.0.0.3:61616)"></networkConnector>
> </networkConnectors>
> </broker>
> </beans>
>
>
> The configuration for Broker2 is identical, but without the network connectors.
>
> I believe that this setup should work and that if one of the network connectors fail the other one should take over. Is this not the case? Is there something that I'm missing?
>
> Many thanks
>
> Paul
>
>
>