You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@activemq.apache.org by "Clebert Suconic (Jira)" <ji...@apache.org> on 2020/08/24 13:24:00 UTC
[jira] [Closed] (ARTEMIS-2885) MQ is not able to re-balance cluster
topology when client do failover
[ https://issues.apache.org/jira/browse/ARTEMIS-2885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Clebert Suconic closed ARTEMIS-2885.
------------------------------------
Resolution: Not A Problem
> MQ is not able to re-balance cluster topology when client do failover
> ---------------------------------------------------------------------
>
> Key: ARTEMIS-2885
> URL: https://issues.apache.org/jira/browse/ARTEMIS-2885
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Components: ActiveMQ-Artemis-Native, Broker, OpenWire
> Affects Versions: 2.12.0
> Environment: Prod, Pre-Prod
> Reporter: Aman Verma
> Assignee: Clebert Suconic
> Priority: Blocker
> Attachments: master_broker.txt, slave_broker.txt
>
>
> We have camel (client) connecting to Artemis MQ and using failover URL : **failover://(ssl://masternode:61616)?randomize=falseamp;maxReconnectAttempts=0**
> MQ is running as active passive node : Master & Slave with shared HA (NFS mounted journals)
> Issue : When master goes down then camel does not connect to Slave node if we give failover url as above (though we have updated artemis acceptor of broker with : updateClusterClients=true and rebalanceClusterClients=true)
> Same goes with clients connecting with ha=true parameter - No fail-over is happening.
> Attached the master and slave broker.xml's
> ------------------------
> Slave Broker.xml
> <?xml version='1.0'?>
> <!--
> Licensed to the Apache Software Foundation (ASF) under one
> or more contributor license agreements. See the NOTICE file
> distributed with this work for additional information
> regarding copyright ownership. The ASF licenses this file
> to you under the Apache License, Version 2.0 (the
> "License"); you may not use this file except in compliance
> with the License. You may obtain a copy of the License at
> http://www.apache.org/licenses/LICENSE-2.0
> Unless required by applicable law or agreed to in writing,
> software distributed under the License is distributed on an
> "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
> KIND, either express or implied. See the License for the
> specific language governing permissions and limitations
> under the License.
> -->
> <configuration xmlns="urn:activemq"
> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
> xmlns:xi="http://www.w3.org/2001/XInclude"
> xsi:schemaLocation="urn:activemq /schema/artemis-configuration.xsd">
> <core xmlns="urn:activemq:core" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
> xsi:schemaLocation="urn:activemq:core ">
> <name>slave</name>
> <persistence-enabled>true</persistence-enabled>
> <!-- this could be ASYNCIO, MAPPED, NIO
> ASYNCIO: Linux Libaio
> MAPPED: mmap files
> NIO: Plain Java Files
> -->
> <journal-type>NIO</journal-type>
> <paging-directory>/opt/data/paging</paging-directory>
> <bindings-directory>/opt/data/bindings</bindings-directory>
> <journal-directory>/opt/data/journal</journal-directory>
> <large-messages-directory>/opt/data/large-messages</large-messages-directory>
> <journal-datasync>true</journal-datasync>
> <journal-min-files>2</journal-min-files>
> <journal-pool-files>10</journal-pool-files>
> <journal-device-block-size>4096</journal-device-block-size>
> <journal-file-size>10M</journal-file-size>
>
> <!--
> This value was determined through a calculation.
> Your system could perform 0.4 writes per millisecond
> on the current journal configuration.
> That translates as a sync write every 2524000 nanoseconds.
> Note: If you specify 0 the system will perform writes directly to the disk.
> We recommend this to be 0 if you are using journalType=MAPPED and journal-datasync=false.
> -->
> <journal-buffer-timeout>2524000</journal-buffer-timeout>
> <!--
> When using ASYNCIO, this will determine the writing queue depth for libaio.
> -->
> <journal-max-io>1</journal-max-io>
> <!--
> You can verify the network health of a particular NIC by specifying the <network-check-NIC> element.
> <network-check-NIC>theNicName</network-check-NIC>
> -->
> <!--
> Use this to use an HTTP server to validate the network
> <network-check-URL-list>http://www.apache.org</network-check-URL-list> -->
> <!-- <network-check-period>10000</network-check-period> -->
> <!-- <network-check-timeout>1000</network-check-timeout> -->
> <!-- this is a comma separated list, no spaces, just DNS or IPs
> it should accept IPV6
> Warning: Make sure you understand your network topology as this is meant to validate if your network is valid.
> Using IPs that could eventually disappear or be partially visible may defeat the purpose.
> You can use a list of multiple IPs, and if any successful ping will make the server OK to continue running -->
> <!-- <network-check-list>10.0.0.1</network-check-list> -->
> <!-- use this to customize the ping used for ipv4 addresses -->
> <!-- <network-check-ping-command>ping -c 1 -t %d %s</network-check-ping-command> -->
> <!-- use this to customize the ping used for ipv6 addresses -->
> <!-- <network-check-ping6-command>ping6 -c 1 %2$s</network-check-ping6-command> -->
> <!-- how often we are looking for how many bytes are being used on the disk in ms -->
> <disk-scan-period>5000</disk-scan-period>
> <!-- once the disk hits this limit the system will block, or close the connection in certain protocols
> that won't support flow control. -->
> <max-disk-usage>90</max-disk-usage>
> <!-- should the broker detect dead locks and other issues -->
> <critical-analyzer>true</critical-analyzer>
> <critical-analyzer-timeout>120000</critical-analyzer-timeout>
> <critical-analyzer-check-period>60000</critical-analyzer-check-period>
> <critical-analyzer-policy>HALT</critical-analyzer-policy>
>
> <page-sync-timeout>2524000</page-sync-timeout>
> <!-- the system will enter into page mode once you hit this limit.
> This is an estimate in bytes of how much the messages are using in memory
> The system will use half of the available memory (-Xmx) by default for the global-max-size.
> You may specify a different value here if you need to customize it to your needs.
> <global-max-size>100Mb</global-max-size>
> -->
> <acceptors>
> <!-- useEpoll means: it will use Netty epoll if you are on a system (Linux) that supports it -->
> <!-- amqpCredits: The number of credits sent to AMQP producers -->
> <!-- amqpLowCredits: The server will send the # credits specified at amqpCredits at this low mark -->
> <!-- amqpDuplicateDetection: If you are not using duplicate detection, set this to false
> as duplicate detection requires applicationProperties to be parsed on the server. -->
> <!-- Note: If an acceptor needs to be compatible with HornetQ and/or Artemis 1.x clients add
> "anycastPrefix=jms.queue.;multicastPrefix=jms.topic." to the acceptor url.
> See https://issues.apache.org/jira/browse/ARTEMIS-1644 for more information. -->
> <!-- Acceptor for every supported protocol -->
> <acceptor name="artemis">tcp://slaveip:61616?tcpSendBufferSize=1048576;tcpReceiveBufferSize=1048576;protocols=CORE,AMQP,STOMP,HORNETQ,MQTT,OPENWIRE;useEpoll=true;amqpCredits=1000;amqpLowCredits=300;amqpDuplicateDetection=true;sslEnabled=true;needClientAuth=true;keyStorePath=/opt/mqkeypath;keyStorePassword=ENC(value);trustStorePath=/opt/mqtspath;trustStorePassword=ENC(value1);enabledProtocols=TLSv1.2;enabledCipherSuites=some_cipherlist;supportAdvisory=false;suppressInternalManagementObjects=false;updateClusterClients=true;rebalanceClusterClients=true</acceptor>
> </acceptors>
> <security-settings>
> <security-setting match="#">
> <permission type="createNonDurableQueue" roles="amq"/>
> <permission type="deleteNonDurableQueue" roles="amq"/>
> <permission type="createDurableQueue" roles="amq"/>
> <permission type="deleteDurableQueue" roles="amq"/>
> <permission type="createAddress" roles="amq"/>
> <permission type="deleteAddress" roles="amq"/>
> <permission type="consume" roles="amq"/>
> <permission type="browse" roles="amq"/>
> <permission type="send" roles="amq"/>
> <!-- we need this otherwise ./artemis data imp wouldn't work -->
> <permission type="manage" roles="amq"/>
> </security-setting>
> </security-settings>
> <address-settings>
> <!-- if you define auto-create on certain queues, management has to be auto-create -->
> <address-setting match="activemq.management#">
> <dead-letter-address>DLQ</dead-letter-address>
> <expiry-address>ExpiryQueue</expiry-address>
> <redelivery-delay>0</redelivery-delay>
> <!-- with -1 only the global-max-size is in use for limiting -->
> <max-size-bytes>-1</max-size-bytes>
> <message-counter-history-day-limit>10</message-counter-history-day-limit>
> <address-full-policy>PAGE</address-full-policy>
> <auto-create-queues>true</auto-create-queues>
> <auto-create-addresses>true</auto-create-addresses>
> <auto-create-jms-queues>true</auto-create-jms-queues>
> <auto-create-jms-topics>true</auto-create-jms-topics>
> </address-setting>
> <!--default for catch all-->
> <address-setting match="#">
> <dead-letter-address>DLQ</dead-letter-address>
> <expiry-address>ExpiryQueue</expiry-address>
> <redelivery-delay>0</redelivery-delay>
> <!-- with -1 only the global-max-size is in use for limiting -->
> <max-size-bytes>-1</max-size-bytes>
> <message-counter-history-day-limit>10</message-counter-history-day-limit>
> <address-full-policy>PAGE</address-full-policy>
> <auto-create-queues>true</auto-create-queues>
> <auto-create-addresses>true</auto-create-addresses>
> <auto-create-jms-queues>true</auto-create-jms-queues>
> <auto-create-jms-topics>true</auto-create-jms-topics>
> </address-setting>
> </address-settings>
> <addresses>
> <address name="DLQ">
> <anycast>
> <queue name="DLQ" />
> </anycast>
> </address>
> <address name="ExpiryQueue">
> <anycast>
> <queue name="ExpiryQueue" />
> </anycast>
> </address>
> </addresses>
> <!-- Uncomment the following if you want to use the Standard LoggingActiveMQServerPlugin pluging to log in events
> <broker-plugins>
> <broker-plugin class-name="org.apache.activemq.artemis.core.server.plugin.impl.LoggingActiveMQServerPlugin">
> <property key="LOG_ALL_EVENTS" value="true"/>
> <property key="LOG_CONNECTION_EVENTS" value="true"/>
> <property key="LOG_SESSION_EVENTS" value="true"/>
> <property key="LOG_CONSUMER_EVENTS" value="true"/>
> <property key="LOG_DELIVERING_EVENTS" value="true"/>
> <property key="LOG_SENDING_EVENTS" value="true"/>
> <property key="LOG_INTERNAL_EVENTS" value="true"/>
> </broker-plugin>
> </broker-plugins>
> -->
> <connectors>
> <connector name="master-connector">tcp://masterip:61616?sslEnabled=true;keyStorePath=/opt/mqkeypath;keyStorePassword=ENC(value)</connector>
> <connector name="slave-connector">tcp://slaveip:61616?sslEnabled=true;keyStorePath=/opt/mqkeypath;keyStorePassword=ENC(value)</connector>
> </connectors>
> <cluster-user>mycluster</cluster-user>
> <cluster-password>mycluster</cluster-password>
> <cluster-connections>
> <cluster-connection name="mycluster">
> <connector-ref>slave-connector</connector-ref>
> <static-connectors>
> <connector-ref>master-connector</connector-ref>
> </static-connectors>
> </cluster-connection>
> </cluster-connections>
> <ha-policy>
> <shared-store>
> <slave>
> <failover-on-shutdown>true</failover-on-shutdown>
> <allow-failback>true</allow-failback>
> </slave>
> </shared-store>
> </ha-policy>
> </core>
> </configuration>
> --------------------------
> Master Broker.xml
> <?xml version='1.0'?>
> <!--
> Licensed to the Apache Software Foundation (ASF) under one
> or more contributor license agreements. See the NOTICE file
> distributed with this work for additional information
> regarding copyright ownership. The ASF licenses this file
> to you under the Apache License, Version 2.0 (the
> "License"); you may not use this file except in compliance
> with the License. You may obtain a copy of the License at
> http://www.apache.org/licenses/LICENSE-2.0
> Unless required by applicable law or agreed to in writing,
> software distributed under the License is distributed on an
> "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
> KIND, either express or implied. See the License for the
> specific language governing permissions and limitations
> under the License.
> -->
> <configuration xmlns="urn:activemq"
> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
> xmlns:xi="http://www.w3.org/2001/XInclude"
> xsi:schemaLocation="urn:activemq /schema/artemis-configuration.xsd">
> <core xmlns="urn:activemq:core" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
> xsi:schemaLocation="urn:activemq:core ">
> <name>master</name>
> <persistence-enabled>true</persistence-enabled>
> <!-- this could be ASYNCIO, MAPPED, NIO
> ASYNCIO: Linux Libaio
> MAPPED: mmap files
> NIO: Plain Java Files
> -->
> <journal-type>NIO</journal-type>
> <paging-directory>/opt/data/paging</paging-directory>
> <bindings-directory>/opt/data/bindings</bindings-directory>
> <journal-directory>/opt/data/journal</journal-directory>
> <large-messages-directory>/opt/data/large-messages</large-messages-directory>
> <journal-datasync>true</journal-datasync>
> <journal-min-files>2</journal-min-files>
> <journal-pool-files>10</journal-pool-files>
> <journal-device-block-size>4096</journal-device-block-size>
> <journal-file-size>10M</journal-file-size>
>
> <!--
> This value was determined through a calculation.
> Your system could perform 0.37 writes per millisecond
> on the current journal configuration.
> That translates as a sync write every 2728000 nanoseconds.
> Note: If you specify 0 the system will perform writes directly to the disk.
> We recommend this to be 0 if you are using journalType=MAPPED and journal-datasync=false.
> -->
> <journal-buffer-timeout>2728000</journal-buffer-timeout>
> <!--
> When using ASYNCIO, this will determine the writing queue depth for libaio.
> -->
> <journal-max-io>1</journal-max-io>
> <!--
> You can verify the network health of a particular NIC by specifying the <network-check-NIC> element.
> <network-check-NIC>theNicName</network-check-NIC>
> -->
> <!--
> Use this to use an HTTP server to validate the network
> <network-check-URL-list>http://www.apache.org</network-check-URL-list> -->
> <!-- <network-check-period>10000</network-check-period> -->
> <!-- <network-check-timeout>1000</network-check-timeout> -->
> <!-- this is a comma separated list, no spaces, just DNS or IPs
> it should accept IPV6
> Warning: Make sure you understand your network topology as this is meant to validate if your network is valid.
> Using IPs that could eventually disappear or be partially visible may defeat the purpose.
> You can use a list of multiple IPs, and if any successful ping will make the server OK to continue running -->
> <!-- <network-check-list>10.0.0.1</network-check-list> -->
> <!-- use this to customize the ping used for ipv4 addresses -->
> <!-- <network-check-ping-command>ping -c 1 -t %d %s</network-check-ping-command> -->
> <!-- use this to customize the ping used for ipv6 addresses -->
> <!-- <network-check-ping6-command>ping6 -c 1 %2$s</network-check-ping6-command> -->
> <!-- how often we are looking for how many bytes are being used on the disk in ms -->
> <disk-scan-period>5000</disk-scan-period>
> <!-- once the disk hits this limit the system will block, or close the connection in certain protocols
> that won't support flow control. -->
> <max-disk-usage>90</max-disk-usage>
> <!-- should the broker detect dead locks and other issues -->
> <critical-analyzer>true</critical-analyzer>
> <critical-analyzer-timeout>120000</critical-analyzer-timeout>
> <critical-analyzer-check-period>60000</critical-analyzer-check-period>
> <critical-analyzer-policy>HALT</critical-analyzer-policy>
>
> <page-sync-timeout>2728000</page-sync-timeout>
> <!-- the system will enter into page mode once you hit this limit.
> This is an estimate in bytes of how much the messages are using in memory
> The system will use half of the available memory (-Xmx) by default for the global-max-size.
> You may specify a different value here if you need to customize it to your needs.
> <global-max-size>100Mb</global-max-size>
> -->
> <acceptors>
> <!-- useEpoll means: it will use Netty epoll if you are on a system (Linux) that supports it -->
> <!-- amqpCredits: The number of credits sent to AMQP producers -->
> <!-- amqpLowCredits: The server will send the # credits specified at amqpCredits at this low mark -->
> <!-- amqpDuplicateDetection: If you are not using duplicate detection, set this to false
> as duplicate detection requires applicationProperties to be parsed on the server. -->
> <!-- Note: If an acceptor needs to be compatible with HornetQ and/or Artemis 1.x clients add
> "anycastPrefix=jms.queue.;multicastPrefix=jms.topic." to the acceptor url.
> See https://issues.apache.org/jira/browse/ARTEMIS-1644 for more information. -->
> <!-- Acceptor for every supported protocol -->
> <acceptor name="artemis">tcp://masterip:61616?tcpSendBufferSize=1048576;tcpReceiveBufferSize=1048576;protocols=CORE,AMQP,STOMP,HORNETQ,MQTT,OPENWIRE;useEpoll=true;amqpCredits=1000;amqpLowCredits=300;amqpDuplicateDetection=true;sslEnabled=true;needClientAuth=true;keyStorePath=/opt/mqkeypath;keyStorePassword=ENC(value);trustStorePath=/opt/mqtspath;trustStorePassword=ENC(value1);enabledProtocols=TLSv1.2;enabledCipherSuites=some_cipherlist;supportAdvisory=false;suppressInternalManagementObjects=false;updateClusterClients=true;rebalanceClusterClients=true</acceptor>
>
> </acceptors>
> <security-settings>
> <security-setting match="#">
> <permission type="createNonDurableQueue" roles="amq"/>
> <permission type="deleteNonDurableQueue" roles="amq"/>
> <permission type="createDurableQueue" roles="amq"/>
> <permission type="deleteDurableQueue" roles="amq"/>
> <permission type="createAddress" roles="amq"/>
> <permission type="deleteAddress" roles="amq"/>
> <permission type="consume" roles="amq"/>
> <permission type="browse" roles="amq"/>
> <permission type="send" roles="amq"/>
> <!-- we need this otherwise ./artemis data imp wouldn't work -->
> <permission type="manage" roles="amq"/>
> </security-setting>
> </security-settings>
> <address-settings>
> <!-- if you define auto-create on certain queues, management has to be auto-create -->
> <address-setting match="activemq.management#">
> <dead-letter-address>DLQ</dead-letter-address>
> <expiry-address>ExpiryQueue</expiry-address>
> <redelivery-delay>0</redelivery-delay>
> <!-- with -1 only the global-max-size is in use for limiting -->
> <max-size-bytes>-1</max-size-bytes>
> <message-counter-history-day-limit>10</message-counter-history-day-limit>
> <address-full-policy>PAGE</address-full-policy>
> <auto-create-queues>true</auto-create-queues>
> <auto-create-addresses>true</auto-create-addresses>
> <auto-create-jms-queues>true</auto-create-jms-queues>
> <auto-create-jms-topics>true</auto-create-jms-topics>
> </address-setting>
> <!--default for catch all-->
> <address-setting match="#">
> <dead-letter-address>DLQ</dead-letter-address>
> <expiry-address>ExpiryQueue</expiry-address>
> <redelivery-delay>0</redelivery-delay>
> <!-- with -1 only the global-max-size is in use for limiting -->
> <max-size-bytes>-1</max-size-bytes>
> <message-counter-history-day-limit>10</message-counter-history-day-limit>
> <address-full-policy>PAGE</address-full-policy>
> <auto-create-queues>true</auto-create-queues>
> <auto-create-addresses>true</auto-create-addresses>
> <auto-create-jms-queues>true</auto-create-jms-queues>
> <auto-create-jms-topics>true</auto-create-jms-topics>
> </address-setting>
> </address-settings>
> <addresses>
> <address name="DLQ">
> <anycast>
> <queue name="DLQ" />
> </anycast>
> </address>
> <address name="ExpiryQueue">
> <anycast>
> <queue name="ExpiryQueue" />
> </anycast>
> </address>
> </addresses>
> <metrics-plugin class-name="org.apache.activemq.artemis.core.server.metrics.plugins.ArtemisPrometheusMetricsPlugin"/>
> <!-- Uncomment the following if you want to use the Standard LoggingActiveMQServerPlugin pluging to log in events
> <broker-plugins>
> <broker-plugin class-name="org.apache.activemq.artemis.core.server.plugin.impl.LoggingActiveMQServerPlugin">
> <property key="LOG_ALL_EVENTS" value="true"/>
> <property key="LOG_CONNECTION_EVENTS" value="true"/>
> <property key="LOG_SESSION_EVENTS" value="true"/>
> <property key="LOG_CONSUMER_EVENTS" value="true"/>
> <property key="LOG_DELIVERING_EVENTS" value="true"/>
> <property key="LOG_SENDING_EVENTS" value="true"/>
> <property key="LOG_INTERNAL_EVENTS" value="true"/>
> </broker-plugin>
> </broker-plugins>
> -->
> <connectors>
> <connector name="master-connector">tcp://masterip:61616?sslEnabled=true;keyStorePath=/opt/mqkeypath;keyStorePassword=ENC(value)</connector>
> <connector name="slave-connector">tcp://slaveip:61616?sslEnabled=true;keyStorePath=/opt/mqkeypath;keyStorePassword=ENC(value)</connector>
> </connectors>
> <cluster-user>mycluster</cluster-user>
> <cluster-password>mycluster</cluster-password>
> <cluster-connections>
> <cluster-connection name="mycluster">
> <connector-ref>master-connector</connector-ref>
> <static-connectors>
> <connector-ref>slave-connector</connector-ref>
> </static-connectors>
> </cluster-connection>
> </cluster-connections>
> <ha-policy>
> <shared-store>
> <master>
> <failover-on-shutdown>true</failover-on-shutdown>
> </master>
> </shared-store>
> </ha-policy>
> </core>
> </configuration>
> ==================================
--
This message was sent by Atlassian Jira
(v8.3.4#803005)