You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by Vilius Šumskas <v....@advantes.tech.INVALID> on 2022/01/03 09:00:30 UTC
Artemis cluster topology and external clients
Hello list,
we are trying to use Artemis HA shared storage cluster which our SaaS application. In addition to consumers/producers internal to SaaS application itself, we also have thousands of external consumers/producers which are installed on client’s premises .
As broadcast is not possible on Google Cloud we are using static discovery configuration with these connectors:
<connectors>
<!-- Connector used to be announced through cluster connections and notifications -->
<connector name="artemis-master">tcp://internal-cluster-dns-1:61616</connector>
<connector name="artemis-slave">tcp://internal-cluster-dns2:61616</connector>
</connectors>
Our acceptors are also configured to use internal DNS of the hosts on both cluster nodes:
<acceptor name="artemis">tcp://internal-cluster-dns-1:61616?tcpSendBufferSize=1048576;tcpReceiveBufferSize=1048576;amqpMinLargeMessageSize=102400;protocols=CORE,AMQP,STOMP,HORNETQ,MQTT,OPENWIRE;useEpoll=true;amqpCredits=10
00;amqpLowCredits=300;amqpDuplicateDetection=true;supportAdvisory=false;suppressInternalManagementObjects=false</acceptor>
We don’t have issues with internal consumers/producers, however when we try to connect external consumers (via external IP), they are trying to connect via internal DNS which is probably set in the cluster topology packet.
This is probably expected and by design, but my question is how do we correctly handle such case? We obviously do not want internal clients to be served via external IP because external traffic is expensive in the cloud and the performance would decrease dramatically. Even with static discovery we would like to have a possibility to expand our cluster in the future, i.e. use the topology so that clients are configured automatically.
Do we need to have a split-DNS server so that external and internal clients will see different IP addresses? Or maybe it is possible to have the same node serving different acceptors on different ports and different DNS names?
Any pointers are much appreciated.
--
Best Regards,
Vilius Šumskas
Advantes technologies
IT manager
+370 614 75713
Re: Artemis cluster topology and external clients
Posted by Clebert Suconic <cl...@gmail.com>.
The best option now I think would be to add an interceptor..
long term we could add a translation table on the acceptor? WDYT?
On Thu, Jan 13, 2022 at 9:00 AM Gary Tully <ga...@gmail.com> wrote:
>
> > Why initialConnectAttempts=2?
> That I got from this code:
> https://github.com/apache/activemq-artemis/blob/main/artemis-core-client/src/main/java/org/apache/activemq/artemis/core/client/impl/ServerLocatorImpl.java#L1722
>
> It looks like with 0, it won't ever try the second url, and I don't
> see a randomise option in there
>
> On Wed, 12 Jan 2022 at 21:49, Vilius Šumskas <vi...@rivile.lt> wrote:
> >
> > Thank you! We will try to apply your suggestions.
> >
> > One last thing, as I understood the proper URL string for external clients then would be:
> > (tcp://external-cluster-dns-1:61616,tcp://external-cluster-dns-2:61616)?ha=false&useTopologyForLoadBalancing=false&initialConnectAttempts=2&failoverOnInitialConnection=true
> >
> > Why initialConnectAttempts=2? I saw that default is 0 which means one attempt. Does this mean that 2 will try to reconnect three times (master -> slave -> master)?
> >
> > --
> > Vilius
> >
> > -----Original Message-----
> > From: Gary Tully <ga...@gmail.com>
> > Sent: Wednesday, January 12, 2022 7:30 PM
> > To: users@activemq.apache.org
> > Subject: Re: Artemis cluster topology and external clients
> >
> > that makes sense, any failover reconnect will use the topology information, that will be wrong.
> > you need to have artemis not do any reconnect,
> > initialConnectAttempts=2 and on initial connection, it will choose the first working url from the two external dns names configured.
> > You can get the failover behaviour from camel/jms template - it will create a new connection with the provided url but it too will recreate the connection/session/consumer etc so you won't really need the artemis client to do transparent failover.
> >
> > On Wed, 12 Jan 2022 at 12:55, Vilius Šumskas <vi...@rivile.lt> wrote:
> > >
> > > Thanks, Gary. Ticket created
> > > https://issues.apache.org/jira/browse/ARTEMIS-3640
> > >
> > > We experimented a little bit more with this and found that if we use any of the following parameters failover fails:
> > > retryIntervalMultiplier=2,reconnectAttempts=1000,maxRetryInterval=1200
> > > 00
> > >
> > > However if we remove these parameters the unknown host exception is still present but the failover starts to work (at least from our initial tests). This doesn't make sense to me and goes against what documentation says regarding reconnectAttempt usage in HA.
> > >
> > > Do you have ideas what could be going on? We are using Camel library with Spring Framework for our messaging. My theory is that without reconnectAttempt parameter Artemis client itself doesn't try to reconnect, but Camel (or Spring) somehow takes addresses from URL and reconnects?
> > >
> > > --
> > > Vilius
> > >
> > > -----Original Message-----
> > > From: Gary Tully <ga...@gmail.com>
> > > Sent: Wednesday, January 12, 2022 1:40 PM
> > > To: users@activemq.apache.org
> > > Subject: Re: Artemis cluster topology and external clients
> > >
> > > there is a problem here, currently there is no way to "transform" the host information in the topology update commands. the topology is used for ha and for loadbalancing and the client will always end up with the internal dns name via the topology update.
> > >
> > > This needs an enhancement, some thing along the lines of the what we
> > > have in 5.x via the publishAddressStrategy for Artemis -
> > > https://issues.apache.org/jira/browse/AMQ-6392
> > >
> > > The only workaround is to forsake HA, ha=false and not do reconnects, and have the two urls (with the external dns names) be candidates for reconnection, as it will be an initial connection.
> > > Something above the jms connection (the application, jms template) will have to recreate the connection on failure.
> > >
> > > Please open a JIRA issue to track this need, and we can peek into how difficult it is to resolve or if there are some alternatives that can help this use case.
> > >
> > > On Mon, 10 Jan 2022 at 14:21, Vilius Šumskas <vi...@rivile.lt> wrote:
> > > >
> > > > Hi list,
> > > >
> > > > does anyone have more ideas regarding the issue with external consumers below?
> > > >
> > > > --
> > > > Vilius
> > > >
> > > > -----Original Message-----
> > > > From: Vilius Šumskas
> > > > Sent: Friday, January 7, 2022 11:43 AM
> > > > To: users@activemq.apache.org
> > > > Subject: RE: Artemis cluster topology and external clients
> > > >
> > > > We are still getting these errors on the external clients even when
> > > > useTopologyForLoadBalancing is disabled and using
> > > > (tcp://external-cluster-dns-1:61616,tcp://external-cluster-dns-2:616
> > > > 16
> > > > )
> > > >
> > > > org.apache.activemq.artemis.core.remoting.impl.netty.NettyConnector:
> > > > AMQ214033: Cannot resolve host
> > > > java.net.UnknownHostException: internal-cluster-dns-1
> > > >
> > > > The error is show 2 times every time the external consumer is started. The connection works until we power down cluster-dns-1 machine. Then the consumer just stops receiving messages and never reconnects to cluster-dns-2 node.
> > > >
> > > >
> > > > When using the same consumer internally everything works as expected.
> > > >
> > > > --
> > > > Vilius
> > > >
> > > > -----Original Message-----
> > > > From: Domenico Francesco Bruscino <br...@gmail.com>
> > > > Sent: Tuesday, January 4, 2022 8:02 PM
> > > > To: users@activemq.apache.org
> > > > Subject: Re: Artemis cluster topology and external clients
> > > >
> > > > Hi Vilius,
> > > >
> > > > the clients should disable topology for load balancing and use static connectors, i.e.
> > > >
> > > > (tcp://external-cluster-dns-1:61616,tcp://external-cluster-dns-2:616
> > > > 16 )?ha=true&reconnectAttempts=30&useTopologyForLoadBalancing=false
> > > >
> > > > Regards,
> > > > Domenico
> > > >
> > > > On Mon, 3 Jan 2022 at 10:00, Vilius Šumskas
> > > > <v....@advantes.tech.invalid>
> > > > wrote:
> > > >
> > > > > Hello list,
> > > > >
> > > > > we are trying to use Artemis HA shared storage cluster which our
> > > > > SaaS application. In addition to consumers/producers internal to
> > > > > SaaS application itself, we also have thousands of external
> > > > > consumers/producers which are installed on client’s premises .
> > > > >
> > > > > As broadcast is not possible on Google Cloud we are using static
> > > > > discovery configuration with these connectors:
> > > > >
> > > > > <connectors>
> > > > > <!-- Connector used to be announced through cluster
> > > > > connections and notifications -->
> > > > > <connector
> > > > > name="artemis-master">tcp://internal-cluster-dns-1:61616</connector>
> > > > > <connector
> > > > > name="artemis-slave">tcp://internal-cluster-dns2:61616</connector>
> > > > > </connectors>
> > > > >
> > > > > Our acceptors are also configured to use internal DNS of the hosts
> > > > > on both cluster nodes:
> > > > >
> > > > > <acceptor
> > > > > name="artemis">tcp://internal-cluster-dns-1:61616?tcpSendBufferSiz
> > > > > e=
> > > > > 10
> > > > > 48576;tcpReceiveBufferSize=1048576;amqpMinLargeMessageSize=102400;
> > > > > pr
> > > > > ot
> > > > > ocols=CORE,AMQP,STOMP,HORNETQ,MQTT,OPENWIRE;useEpoll=true;amqpCred
> > > > > it
> > > > > s=
> > > > > 10
> > > > >
> > > > > 00;amqpLowCredits=300;amqpDuplicateDetection=true;supportAdvisory=
> > > > > fa ls e;suppressInternalManagementObjects=false</acceptor>
> > > > >
> > > > > We don’t have issues with internal consumers/producers, however
> > > > > when we try to connect external consumers (via external IP), they
> > > > > are trying to connect via internal DNS which is probably set in
> > > > > the cluster topology packet.
> > > > >
> > > > > This is probably expected and by design, but my question is how do
> > > > > we correctly handle such case? We obviously do not want internal
> > > > > clients to be served via external IP because external traffic is
> > > > > expensive in the cloud and the performance would decrease
> > > > > dramatically. Even with static discovery we would like to have a
> > > > > possibility to expand our cluster in the future, i.e. use the topology so that clients are configured automatically.
> > > > >
> > > > > Do we need to have a split-DNS server so that external and
> > > > > internal clients will see different IP addresses? Or maybe it is
> > > > > possible to have the same node serving different acceptors on
> > > > > different ports and different DNS names?
> > > > >
> > > > > Any pointers are much appreciated.
> > > > >
> > > > > --
> > > > > Best Regards,
> > > > >
> > > > > Vilius Šumskas
> > > > > Advantes technologies
> > > > > IT manager
> > > > > +370 614 75713
> > > > >
> > > > >
--
Clebert Suconic
RE: Artemis cluster topology and external clients
Posted by Vilius Šumskas <vi...@rivile.lt>.
That if statement contains AND so the cycle should do one more loop, no? However, your remark regarding randomisation is interesting. Does this mean that failoverOnInitialConnection doesn't work for static discovery?
BTW, I'm using this documentation as a guidance https://access.redhat.com/documentation/en-us/red_hat_amq/2021.q3/html-single/using_the_amq_core_protocol_jms_client/index#failover_options
--
Vilius
-----Original Message-----
From: Gary Tully <ga...@gmail.com>
Sent: Thursday, January 13, 2022 4:00 PM
To: users@activemq.apache.org
Subject: Re: Artemis cluster topology and external clients
> Why initialConnectAttempts=2?
That I got from this code:
https://github.com/apache/activemq-artemis/blob/main/artemis-core-client/src/main/java/org/apache/activemq/artemis/core/client/impl/ServerLocatorImpl.java#L1722
It looks like with 0, it won't ever try the second url, and I don't see a randomise option in there
On Wed, 12 Jan 2022 at 21:49, Vilius Šumskas <vi...@rivile.lt> wrote:
>
> Thank you! We will try to apply your suggestions.
>
> One last thing, as I understood the proper URL string for external clients then would be:
> (tcp://external-cluster-dns-1:61616,tcp://external-cluster-dns-2:61616
> )?ha=false&useTopologyForLoadBalancing=false&initialConnectAttempts=2&
> failoverOnInitialConnection=true
>
> Why initialConnectAttempts=2? I saw that default is 0 which means one attempt. Does this mean that 2 will try to reconnect three times (master -> slave -> master)?
>
> --
> Vilius
>
> -----Original Message-----
> From: Gary Tully <ga...@gmail.com>
> Sent: Wednesday, January 12, 2022 7:30 PM
> To: users@activemq.apache.org
> Subject: Re: Artemis cluster topology and external clients
>
> that makes sense, any failover reconnect will use the topology information, that will be wrong.
> you need to have artemis not do any reconnect,
> initialConnectAttempts=2 and on initial connection, it will choose the first working url from the two external dns names configured.
> You can get the failover behaviour from camel/jms template - it will create a new connection with the provided url but it too will recreate the connection/session/consumer etc so you won't really need the artemis client to do transparent failover.
>
> On Wed, 12 Jan 2022 at 12:55, Vilius Šumskas <vi...@rivile.lt> wrote:
> >
> > Thanks, Gary. Ticket created
> > https://issues.apache.org/jira/browse/ARTEMIS-3640
> >
> > We experimented a little bit more with this and found that if we use any of the following parameters failover fails:
> > retryIntervalMultiplier=2,reconnectAttempts=1000,maxRetryInterval=12
> > 00
> > 00
> >
> > However if we remove these parameters the unknown host exception is still present but the failover starts to work (at least from our initial tests). This doesn't make sense to me and goes against what documentation says regarding reconnectAttempt usage in HA.
> >
> > Do you have ideas what could be going on? We are using Camel library with Spring Framework for our messaging. My theory is that without reconnectAttempt parameter Artemis client itself doesn't try to reconnect, but Camel (or Spring) somehow takes addresses from URL and reconnects?
> >
> > --
> > Vilius
> >
> > -----Original Message-----
> > From: Gary Tully <ga...@gmail.com>
> > Sent: Wednesday, January 12, 2022 1:40 PM
> > To: users@activemq.apache.org
> > Subject: Re: Artemis cluster topology and external clients
> >
> > there is a problem here, currently there is no way to "transform" the host information in the topology update commands. the topology is used for ha and for loadbalancing and the client will always end up with the internal dns name via the topology update.
> >
> > This needs an enhancement, some thing along the lines of the what we
> > have in 5.x via the publishAddressStrategy for Artemis -
> > https://issues.apache.org/jira/browse/AMQ-6392
> >
> > The only workaround is to forsake HA, ha=false and not do reconnects, and have the two urls (with the external dns names) be candidates for reconnection, as it will be an initial connection.
> > Something above the jms connection (the application, jms template) will have to recreate the connection on failure.
> >
> > Please open a JIRA issue to track this need, and we can peek into how difficult it is to resolve or if there are some alternatives that can help this use case.
> >
> > On Mon, 10 Jan 2022 at 14:21, Vilius Šumskas <vi...@rivile.lt> wrote:
> > >
> > > Hi list,
> > >
> > > does anyone have more ideas regarding the issue with external consumers below?
> > >
> > > --
> > > Vilius
> > >
> > > -----Original Message-----
> > > From: Vilius Šumskas
> > > Sent: Friday, January 7, 2022 11:43 AM
> > > To: users@activemq.apache.org
> > > Subject: RE: Artemis cluster topology and external clients
> > >
> > > We are still getting these errors on the external clients even
> > > when useTopologyForLoadBalancing is disabled and using
> > > (tcp://external-cluster-dns-1:61616,tcp://external-cluster-dns-2:6
> > > 16
> > > 16
> > > )
> > >
> > > org.apache.activemq.artemis.core.remoting.impl.netty.NettyConnector:
> > > AMQ214033: Cannot resolve host
> > > java.net.UnknownHostException: internal-cluster-dns-1
> > >
> > > The error is show 2 times every time the external consumer is started. The connection works until we power down cluster-dns-1 machine. Then the consumer just stops receiving messages and never reconnects to cluster-dns-2 node.
> > >
> > >
> > > When using the same consumer internally everything works as expected.
> > >
> > > --
> > > Vilius
> > >
> > > -----Original Message-----
> > > From: Domenico Francesco Bruscino <br...@gmail.com>
> > > Sent: Tuesday, January 4, 2022 8:02 PM
> > > To: users@activemq.apache.org
> > > Subject: Re: Artemis cluster topology and external clients
> > >
> > > Hi Vilius,
> > >
> > > the clients should disable topology for load balancing and use static connectors, i.e.
> > >
> > > (tcp://external-cluster-dns-1:61616,tcp://external-cluster-dns-2:6
> > > 16
> > > 16
> > > )?ha=true&reconnectAttempts=30&useTopologyForLoadBalancing=false
> > >
> > > Regards,
> > > Domenico
> > >
> > > On Mon, 3 Jan 2022 at 10:00, Vilius Šumskas
> > > <v....@advantes.tech.invalid>
> > > wrote:
> > >
> > > > Hello list,
> > > >
> > > > we are trying to use Artemis HA shared storage cluster which our
> > > > SaaS application. In addition to consumers/producers internal to
> > > > SaaS application itself, we also have thousands of external
> > > > consumers/producers which are installed on client’s premises .
> > > >
> > > > As broadcast is not possible on Google Cloud we are using static
> > > > discovery configuration with these connectors:
> > > >
> > > > <connectors>
> > > > <!-- Connector used to be announced through cluster
> > > > connections and notifications -->
> > > > <connector
> > > > name="artemis-master">tcp://internal-cluster-dns-1:61616</connector>
> > > > <connector
> > > > name="artemis-slave">tcp://internal-cluster-dns2:61616</connector>
> > > > </connectors>
> > > >
> > > > Our acceptors are also configured to use internal DNS of the
> > > > hosts on both cluster nodes:
> > > >
> > > > <acceptor
> > > > name="artemis">tcp://internal-cluster-dns-1:61616?tcpSendBufferS
> > > > iz
> > > > e=
> > > > 10
> > > > 48576;tcpReceiveBufferSize=1048576;amqpMinLargeMessageSize=10240
> > > > 0;
> > > > pr
> > > > ot
> > > > ocols=CORE,AMQP,STOMP,HORNETQ,MQTT,OPENWIRE;useEpoll=true;amqpCr
> > > > ed
> > > > it
> > > > s=
> > > > 10
> > > >
> > > > 00;amqpLowCredits=300;amqpDuplicateDetection=true;supportAdvisor
> > > > y= fa ls e;suppressInternalManagementObjects=false</acceptor>
> > > >
> > > > We don’t have issues with internal consumers/producers, however
> > > > when we try to connect external consumers (via external IP),
> > > > they are trying to connect via internal DNS which is probably
> > > > set in the cluster topology packet.
> > > >
> > > > This is probably expected and by design, but my question is how
> > > > do we correctly handle such case? We obviously do not want
> > > > internal clients to be served via external IP because external
> > > > traffic is expensive in the cloud and the performance would
> > > > decrease dramatically. Even with static discovery we would like
> > > > to have a possibility to expand our cluster in the future, i.e. use the topology so that clients are configured automatically.
> > > >
> > > > Do we need to have a split-DNS server so that external and
> > > > internal clients will see different IP addresses? Or maybe it is
> > > > possible to have the same node serving different acceptors on
> > > > different ports and different DNS names?
> > > >
> > > > Any pointers are much appreciated.
> > > >
> > > > --
> > > > Best Regards,
> > > >
> > > > Vilius Šumskas
> > > > Advantes technologies
> > > > IT manager
> > > > +370 614 75713
> > > >
> > > >
Re: Artemis cluster topology and external clients
Posted by Gary Tully <ga...@gmail.com>.
> Why initialConnectAttempts=2?
That I got from this code:
https://github.com/apache/activemq-artemis/blob/main/artemis-core-client/src/main/java/org/apache/activemq/artemis/core/client/impl/ServerLocatorImpl.java#L1722
It looks like with 0, it won't ever try the second url, and I don't
see a randomise option in there
On Wed, 12 Jan 2022 at 21:49, Vilius Šumskas <vi...@rivile.lt> wrote:
>
> Thank you! We will try to apply your suggestions.
>
> One last thing, as I understood the proper URL string for external clients then would be:
> (tcp://external-cluster-dns-1:61616,tcp://external-cluster-dns-2:61616)?ha=false&useTopologyForLoadBalancing=false&initialConnectAttempts=2&failoverOnInitialConnection=true
>
> Why initialConnectAttempts=2? I saw that default is 0 which means one attempt. Does this mean that 2 will try to reconnect three times (master -> slave -> master)?
>
> --
> Vilius
>
> -----Original Message-----
> From: Gary Tully <ga...@gmail.com>
> Sent: Wednesday, January 12, 2022 7:30 PM
> To: users@activemq.apache.org
> Subject: Re: Artemis cluster topology and external clients
>
> that makes sense, any failover reconnect will use the topology information, that will be wrong.
> you need to have artemis not do any reconnect,
> initialConnectAttempts=2 and on initial connection, it will choose the first working url from the two external dns names configured.
> You can get the failover behaviour from camel/jms template - it will create a new connection with the provided url but it too will recreate the connection/session/consumer etc so you won't really need the artemis client to do transparent failover.
>
> On Wed, 12 Jan 2022 at 12:55, Vilius Šumskas <vi...@rivile.lt> wrote:
> >
> > Thanks, Gary. Ticket created
> > https://issues.apache.org/jira/browse/ARTEMIS-3640
> >
> > We experimented a little bit more with this and found that if we use any of the following parameters failover fails:
> > retryIntervalMultiplier=2,reconnectAttempts=1000,maxRetryInterval=1200
> > 00
> >
> > However if we remove these parameters the unknown host exception is still present but the failover starts to work (at least from our initial tests). This doesn't make sense to me and goes against what documentation says regarding reconnectAttempt usage in HA.
> >
> > Do you have ideas what could be going on? We are using Camel library with Spring Framework for our messaging. My theory is that without reconnectAttempt parameter Artemis client itself doesn't try to reconnect, but Camel (or Spring) somehow takes addresses from URL and reconnects?
> >
> > --
> > Vilius
> >
> > -----Original Message-----
> > From: Gary Tully <ga...@gmail.com>
> > Sent: Wednesday, January 12, 2022 1:40 PM
> > To: users@activemq.apache.org
> > Subject: Re: Artemis cluster topology and external clients
> >
> > there is a problem here, currently there is no way to "transform" the host information in the topology update commands. the topology is used for ha and for loadbalancing and the client will always end up with the internal dns name via the topology update.
> >
> > This needs an enhancement, some thing along the lines of the what we
> > have in 5.x via the publishAddressStrategy for Artemis -
> > https://issues.apache.org/jira/browse/AMQ-6392
> >
> > The only workaround is to forsake HA, ha=false and not do reconnects, and have the two urls (with the external dns names) be candidates for reconnection, as it will be an initial connection.
> > Something above the jms connection (the application, jms template) will have to recreate the connection on failure.
> >
> > Please open a JIRA issue to track this need, and we can peek into how difficult it is to resolve or if there are some alternatives that can help this use case.
> >
> > On Mon, 10 Jan 2022 at 14:21, Vilius Šumskas <vi...@rivile.lt> wrote:
> > >
> > > Hi list,
> > >
> > > does anyone have more ideas regarding the issue with external consumers below?
> > >
> > > --
> > > Vilius
> > >
> > > -----Original Message-----
> > > From: Vilius Šumskas
> > > Sent: Friday, January 7, 2022 11:43 AM
> > > To: users@activemq.apache.org
> > > Subject: RE: Artemis cluster topology and external clients
> > >
> > > We are still getting these errors on the external clients even when
> > > useTopologyForLoadBalancing is disabled and using
> > > (tcp://external-cluster-dns-1:61616,tcp://external-cluster-dns-2:616
> > > 16
> > > )
> > >
> > > org.apache.activemq.artemis.core.remoting.impl.netty.NettyConnector:
> > > AMQ214033: Cannot resolve host
> > > java.net.UnknownHostException: internal-cluster-dns-1
> > >
> > > The error is show 2 times every time the external consumer is started. The connection works until we power down cluster-dns-1 machine. Then the consumer just stops receiving messages and never reconnects to cluster-dns-2 node.
> > >
> > >
> > > When using the same consumer internally everything works as expected.
> > >
> > > --
> > > Vilius
> > >
> > > -----Original Message-----
> > > From: Domenico Francesco Bruscino <br...@gmail.com>
> > > Sent: Tuesday, January 4, 2022 8:02 PM
> > > To: users@activemq.apache.org
> > > Subject: Re: Artemis cluster topology and external clients
> > >
> > > Hi Vilius,
> > >
> > > the clients should disable topology for load balancing and use static connectors, i.e.
> > >
> > > (tcp://external-cluster-dns-1:61616,tcp://external-cluster-dns-2:616
> > > 16 )?ha=true&reconnectAttempts=30&useTopologyForLoadBalancing=false
> > >
> > > Regards,
> > > Domenico
> > >
> > > On Mon, 3 Jan 2022 at 10:00, Vilius Šumskas
> > > <v....@advantes.tech.invalid>
> > > wrote:
> > >
> > > > Hello list,
> > > >
> > > > we are trying to use Artemis HA shared storage cluster which our
> > > > SaaS application. In addition to consumers/producers internal to
> > > > SaaS application itself, we also have thousands of external
> > > > consumers/producers which are installed on client’s premises .
> > > >
> > > > As broadcast is not possible on Google Cloud we are using static
> > > > discovery configuration with these connectors:
> > > >
> > > > <connectors>
> > > > <!-- Connector used to be announced through cluster
> > > > connections and notifications -->
> > > > <connector
> > > > name="artemis-master">tcp://internal-cluster-dns-1:61616</connector>
> > > > <connector
> > > > name="artemis-slave">tcp://internal-cluster-dns2:61616</connector>
> > > > </connectors>
> > > >
> > > > Our acceptors are also configured to use internal DNS of the hosts
> > > > on both cluster nodes:
> > > >
> > > > <acceptor
> > > > name="artemis">tcp://internal-cluster-dns-1:61616?tcpSendBufferSiz
> > > > e=
> > > > 10
> > > > 48576;tcpReceiveBufferSize=1048576;amqpMinLargeMessageSize=102400;
> > > > pr
> > > > ot
> > > > ocols=CORE,AMQP,STOMP,HORNETQ,MQTT,OPENWIRE;useEpoll=true;amqpCred
> > > > it
> > > > s=
> > > > 10
> > > >
> > > > 00;amqpLowCredits=300;amqpDuplicateDetection=true;supportAdvisory=
> > > > fa ls e;suppressInternalManagementObjects=false</acceptor>
> > > >
> > > > We don’t have issues with internal consumers/producers, however
> > > > when we try to connect external consumers (via external IP), they
> > > > are trying to connect via internal DNS which is probably set in
> > > > the cluster topology packet.
> > > >
> > > > This is probably expected and by design, but my question is how do
> > > > we correctly handle such case? We obviously do not want internal
> > > > clients to be served via external IP because external traffic is
> > > > expensive in the cloud and the performance would decrease
> > > > dramatically. Even with static discovery we would like to have a
> > > > possibility to expand our cluster in the future, i.e. use the topology so that clients are configured automatically.
> > > >
> > > > Do we need to have a split-DNS server so that external and
> > > > internal clients will see different IP addresses? Or maybe it is
> > > > possible to have the same node serving different acceptors on
> > > > different ports and different DNS names?
> > > >
> > > > Any pointers are much appreciated.
> > > >
> > > > --
> > > > Best Regards,
> > > >
> > > > Vilius Šumskas
> > > > Advantes technologies
> > > > IT manager
> > > > +370 614 75713
> > > >
> > > >
RE: Artemis cluster topology and external clients
Posted by Vilius Šumskas <vi...@rivile.lt>.
Thank you! We will try to apply your suggestions.
One last thing, as I understood the proper URL string for external clients then would be:
(tcp://external-cluster-dns-1:61616,tcp://external-cluster-dns-2:61616)?ha=false&useTopologyForLoadBalancing=false&initialConnectAttempts=2&failoverOnInitialConnection=true
Why initialConnectAttempts=2? I saw that default is 0 which means one attempt. Does this mean that 2 will try to reconnect three times (master -> slave -> master)?
--
Vilius
-----Original Message-----
From: Gary Tully <ga...@gmail.com>
Sent: Wednesday, January 12, 2022 7:30 PM
To: users@activemq.apache.org
Subject: Re: Artemis cluster topology and external clients
that makes sense, any failover reconnect will use the topology information, that will be wrong.
you need to have artemis not do any reconnect,
initialConnectAttempts=2 and on initial connection, it will choose the first working url from the two external dns names configured.
You can get the failover behaviour from camel/jms template - it will create a new connection with the provided url but it too will recreate the connection/session/consumer etc so you won't really need the artemis client to do transparent failover.
On Wed, 12 Jan 2022 at 12:55, Vilius Šumskas <vi...@rivile.lt> wrote:
>
> Thanks, Gary. Ticket created
> https://issues.apache.org/jira/browse/ARTEMIS-3640
>
> We experimented a little bit more with this and found that if we use any of the following parameters failover fails:
> retryIntervalMultiplier=2,reconnectAttempts=1000,maxRetryInterval=1200
> 00
>
> However if we remove these parameters the unknown host exception is still present but the failover starts to work (at least from our initial tests). This doesn't make sense to me and goes against what documentation says regarding reconnectAttempt usage in HA.
>
> Do you have ideas what could be going on? We are using Camel library with Spring Framework for our messaging. My theory is that without reconnectAttempt parameter Artemis client itself doesn't try to reconnect, but Camel (or Spring) somehow takes addresses from URL and reconnects?
>
> --
> Vilius
>
> -----Original Message-----
> From: Gary Tully <ga...@gmail.com>
> Sent: Wednesday, January 12, 2022 1:40 PM
> To: users@activemq.apache.org
> Subject: Re: Artemis cluster topology and external clients
>
> there is a problem here, currently there is no way to "transform" the host information in the topology update commands. the topology is used for ha and for loadbalancing and the client will always end up with the internal dns name via the topology update.
>
> This needs an enhancement, some thing along the lines of the what we
> have in 5.x via the publishAddressStrategy for Artemis -
> https://issues.apache.org/jira/browse/AMQ-6392
>
> The only workaround is to forsake HA, ha=false and not do reconnects, and have the two urls (with the external dns names) be candidates for reconnection, as it will be an initial connection.
> Something above the jms connection (the application, jms template) will have to recreate the connection on failure.
>
> Please open a JIRA issue to track this need, and we can peek into how difficult it is to resolve or if there are some alternatives that can help this use case.
>
> On Mon, 10 Jan 2022 at 14:21, Vilius Šumskas <vi...@rivile.lt> wrote:
> >
> > Hi list,
> >
> > does anyone have more ideas regarding the issue with external consumers below?
> >
> > --
> > Vilius
> >
> > -----Original Message-----
> > From: Vilius Šumskas
> > Sent: Friday, January 7, 2022 11:43 AM
> > To: users@activemq.apache.org
> > Subject: RE: Artemis cluster topology and external clients
> >
> > We are still getting these errors on the external clients even when
> > useTopologyForLoadBalancing is disabled and using
> > (tcp://external-cluster-dns-1:61616,tcp://external-cluster-dns-2:616
> > 16
> > )
> >
> > org.apache.activemq.artemis.core.remoting.impl.netty.NettyConnector:
> > AMQ214033: Cannot resolve host
> > java.net.UnknownHostException: internal-cluster-dns-1
> >
> > The error is show 2 times every time the external consumer is started. The connection works until we power down cluster-dns-1 machine. Then the consumer just stops receiving messages and never reconnects to cluster-dns-2 node.
> >
> >
> > When using the same consumer internally everything works as expected.
> >
> > --
> > Vilius
> >
> > -----Original Message-----
> > From: Domenico Francesco Bruscino <br...@gmail.com>
> > Sent: Tuesday, January 4, 2022 8:02 PM
> > To: users@activemq.apache.org
> > Subject: Re: Artemis cluster topology and external clients
> >
> > Hi Vilius,
> >
> > the clients should disable topology for load balancing and use static connectors, i.e.
> >
> > (tcp://external-cluster-dns-1:61616,tcp://external-cluster-dns-2:616
> > 16 )?ha=true&reconnectAttempts=30&useTopologyForLoadBalancing=false
> >
> > Regards,
> > Domenico
> >
> > On Mon, 3 Jan 2022 at 10:00, Vilius Šumskas
> > <v....@advantes.tech.invalid>
> > wrote:
> >
> > > Hello list,
> > >
> > > we are trying to use Artemis HA shared storage cluster which our
> > > SaaS application. In addition to consumers/producers internal to
> > > SaaS application itself, we also have thousands of external
> > > consumers/producers which are installed on client’s premises .
> > >
> > > As broadcast is not possible on Google Cloud we are using static
> > > discovery configuration with these connectors:
> > >
> > > <connectors>
> > > <!-- Connector used to be announced through cluster
> > > connections and notifications -->
> > > <connector
> > > name="artemis-master">tcp://internal-cluster-dns-1:61616</connector>
> > > <connector
> > > name="artemis-slave">tcp://internal-cluster-dns2:61616</connector>
> > > </connectors>
> > >
> > > Our acceptors are also configured to use internal DNS of the hosts
> > > on both cluster nodes:
> > >
> > > <acceptor
> > > name="artemis">tcp://internal-cluster-dns-1:61616?tcpSendBufferSiz
> > > e=
> > > 10
> > > 48576;tcpReceiveBufferSize=1048576;amqpMinLargeMessageSize=102400;
> > > pr
> > > ot
> > > ocols=CORE,AMQP,STOMP,HORNETQ,MQTT,OPENWIRE;useEpoll=true;amqpCred
> > > it
> > > s=
> > > 10
> > >
> > > 00;amqpLowCredits=300;amqpDuplicateDetection=true;supportAdvisory=
> > > fa ls e;suppressInternalManagementObjects=false</acceptor>
> > >
> > > We don’t have issues with internal consumers/producers, however
> > > when we try to connect external consumers (via external IP), they
> > > are trying to connect via internal DNS which is probably set in
> > > the cluster topology packet.
> > >
> > > This is probably expected and by design, but my question is how do
> > > we correctly handle such case? We obviously do not want internal
> > > clients to be served via external IP because external traffic is
> > > expensive in the cloud and the performance would decrease
> > > dramatically. Even with static discovery we would like to have a
> > > possibility to expand our cluster in the future, i.e. use the topology so that clients are configured automatically.
> > >
> > > Do we need to have a split-DNS server so that external and
> > > internal clients will see different IP addresses? Or maybe it is
> > > possible to have the same node serving different acceptors on
> > > different ports and different DNS names?
> > >
> > > Any pointers are much appreciated.
> > >
> > > --
> > > Best Regards,
> > >
> > > Vilius Šumskas
> > > Advantes technologies
> > > IT manager
> > > +370 614 75713
> > >
> > >
Re: Artemis cluster topology and external clients
Posted by Gary Tully <ga...@gmail.com>.
that makes sense, any failover reconnect will use the topology
information, that will be wrong.
you need to have artemis not do any reconnect,
initialConnectAttempts=2 and on initial connection, it will choose the
first working url from the two external dns names configured.
You can get the failover behaviour from camel/jms template - it will
create a new connection with the provided url but it too will recreate
the connection/session/consumer etc so you won't really need the
artemis client to do transparent failover.
On Wed, 12 Jan 2022 at 12:55, Vilius Šumskas <vi...@rivile.lt> wrote:
>
> Thanks, Gary. Ticket created https://issues.apache.org/jira/browse/ARTEMIS-3640
>
> We experimented a little bit more with this and found that if we use any of the following parameters failover fails:
> retryIntervalMultiplier=2,reconnectAttempts=1000,maxRetryInterval=120000
>
> However if we remove these parameters the unknown host exception is still present but the failover starts to work (at least from our initial tests). This doesn't make sense to me and goes against what documentation says regarding reconnectAttempt usage in HA.
>
> Do you have ideas what could be going on? We are using Camel library with Spring Framework for our messaging. My theory is that without reconnectAttempt parameter Artemis client itself doesn't try to reconnect, but Camel (or Spring) somehow takes addresses from URL and reconnects?
>
> --
> Vilius
>
> -----Original Message-----
> From: Gary Tully <ga...@gmail.com>
> Sent: Wednesday, January 12, 2022 1:40 PM
> To: users@activemq.apache.org
> Subject: Re: Artemis cluster topology and external clients
>
> there is a problem here, currently there is no way to "transform" the host information in the topology update commands. the topology is used for ha and for loadbalancing and the client will always end up with the internal dns name via the topology update.
>
> This needs an enhancement, some thing along the lines of the what we have in 5.x via the publishAddressStrategy for Artemis -
> https://issues.apache.org/jira/browse/AMQ-6392
>
> The only workaround is to forsake HA, ha=false and not do reconnects, and have the two urls (with the external dns names) be candidates for reconnection, as it will be an initial connection.
> Something above the jms connection (the application, jms template) will have to recreate the connection on failure.
>
> Please open a JIRA issue to track this need, and we can peek into how difficult it is to resolve or if there are some alternatives that can help this use case.
>
> On Mon, 10 Jan 2022 at 14:21, Vilius Šumskas <vi...@rivile.lt> wrote:
> >
> > Hi list,
> >
> > does anyone have more ideas regarding the issue with external consumers below?
> >
> > --
> > Vilius
> >
> > -----Original Message-----
> > From: Vilius Šumskas
> > Sent: Friday, January 7, 2022 11:43 AM
> > To: users@activemq.apache.org
> > Subject: RE: Artemis cluster topology and external clients
> >
> > We are still getting these errors on the external clients even when
> > useTopologyForLoadBalancing is disabled and using
> > (tcp://external-cluster-dns-1:61616,tcp://external-cluster-dns-2:61616
> > )
> >
> > org.apache.activemq.artemis.core.remoting.impl.netty.NettyConnector:
> > AMQ214033: Cannot resolve host
> > java.net.UnknownHostException: internal-cluster-dns-1
> >
> > The error is show 2 times every time the external consumer is started. The connection works until we power down cluster-dns-1 machine. Then the consumer just stops receiving messages and never reconnects to cluster-dns-2 node.
> >
> >
> > When using the same consumer internally everything works as expected.
> >
> > --
> > Vilius
> >
> > -----Original Message-----
> > From: Domenico Francesco Bruscino <br...@gmail.com>
> > Sent: Tuesday, January 4, 2022 8:02 PM
> > To: users@activemq.apache.org
> > Subject: Re: Artemis cluster topology and external clients
> >
> > Hi Vilius,
> >
> > the clients should disable topology for load balancing and use static connectors, i.e.
> >
> > (tcp://external-cluster-dns-1:61616,tcp://external-cluster-dns-2:61616
> > )?ha=true&reconnectAttempts=30&useTopologyForLoadBalancing=false
> >
> > Regards,
> > Domenico
> >
> > On Mon, 3 Jan 2022 at 10:00, Vilius Šumskas
> > <v....@advantes.tech.invalid>
> > wrote:
> >
> > > Hello list,
> > >
> > > we are trying to use Artemis HA shared storage cluster which our
> > > SaaS application. In addition to consumers/producers internal to
> > > SaaS application itself, we also have thousands of external
> > > consumers/producers which are installed on client’s premises .
> > >
> > > As broadcast is not possible on Google Cloud we are using static
> > > discovery configuration with these connectors:
> > >
> > > <connectors>
> > > <!-- Connector used to be announced through cluster
> > > connections and notifications -->
> > > <connector
> > > name="artemis-master">tcp://internal-cluster-dns-1:61616</connector>
> > > <connector
> > > name="artemis-slave">tcp://internal-cluster-dns2:61616</connector>
> > > </connectors>
> > >
> > > Our acceptors are also configured to use internal DNS of the hosts
> > > on both cluster nodes:
> > >
> > > <acceptor
> > > name="artemis">tcp://internal-cluster-dns-1:61616?tcpSendBufferSize=
> > > 10
> > > 48576;tcpReceiveBufferSize=1048576;amqpMinLargeMessageSize=102400;pr
> > > ot
> > > ocols=CORE,AMQP,STOMP,HORNETQ,MQTT,OPENWIRE;useEpoll=true;amqpCredit
> > > s=
> > > 10
> > >
> > > 00;amqpLowCredits=300;amqpDuplicateDetection=true;supportAdvisory=fa
> > > ls e;suppressInternalManagementObjects=false</acceptor>
> > >
> > > We don’t have issues with internal consumers/producers, however when
> > > we try to connect external consumers (via external IP), they are
> > > trying to connect via internal DNS which is probably set in the
> > > cluster topology packet.
> > >
> > > This is probably expected and by design, but my question is how do
> > > we correctly handle such case? We obviously do not want internal
> > > clients to be served via external IP because external traffic is
> > > expensive in the cloud and the performance would decrease
> > > dramatically. Even with static discovery we would like to have a
> > > possibility to expand our cluster in the future, i.e. use the topology so that clients are configured automatically.
> > >
> > > Do we need to have a split-DNS server so that external and internal
> > > clients will see different IP addresses? Or maybe it is possible to
> > > have the same node serving different acceptors on different ports
> > > and different DNS names?
> > >
> > > Any pointers are much appreciated.
> > >
> > > --
> > > Best Regards,
> > >
> > > Vilius Šumskas
> > > Advantes technologies
> > > IT manager
> > > +370 614 75713
> > >
> > >
RE: Artemis cluster topology and external clients
Posted by Vilius Šumskas <vi...@rivile.lt>.
Thanks, Gary. Ticket created https://issues.apache.org/jira/browse/ARTEMIS-3640
We experimented a little bit more with this and found that if we use any of the following parameters failover fails:
retryIntervalMultiplier=2,reconnectAttempts=1000,maxRetryInterval=120000
However if we remove these parameters the unknown host exception is still present but the failover starts to work (at least from our initial tests). This doesn't make sense to me and goes against what documentation says regarding reconnectAttempt usage in HA.
Do you have ideas what could be going on? We are using Camel library with Spring Framework for our messaging. My theory is that without reconnectAttempt parameter Artemis client itself doesn't try to reconnect, but Camel (or Spring) somehow takes addresses from URL and reconnects?
--
Vilius
-----Original Message-----
From: Gary Tully <ga...@gmail.com>
Sent: Wednesday, January 12, 2022 1:40 PM
To: users@activemq.apache.org
Subject: Re: Artemis cluster topology and external clients
there is a problem here, currently there is no way to "transform" the host information in the topology update commands. the topology is used for ha and for loadbalancing and the client will always end up with the internal dns name via the topology update.
This needs an enhancement, some thing along the lines of the what we have in 5.x via the publishAddressStrategy for Artemis -
https://issues.apache.org/jira/browse/AMQ-6392
The only workaround is to forsake HA, ha=false and not do reconnects, and have the two urls (with the external dns names) be candidates for reconnection, as it will be an initial connection.
Something above the jms connection (the application, jms template) will have to recreate the connection on failure.
Please open a JIRA issue to track this need, and we can peek into how difficult it is to resolve or if there are some alternatives that can help this use case.
On Mon, 10 Jan 2022 at 14:21, Vilius Šumskas <vi...@rivile.lt> wrote:
>
> Hi list,
>
> does anyone have more ideas regarding the issue with external consumers below?
>
> --
> Vilius
>
> -----Original Message-----
> From: Vilius Šumskas
> Sent: Friday, January 7, 2022 11:43 AM
> To: users@activemq.apache.org
> Subject: RE: Artemis cluster topology and external clients
>
> We are still getting these errors on the external clients even when
> useTopologyForLoadBalancing is disabled and using
> (tcp://external-cluster-dns-1:61616,tcp://external-cluster-dns-2:61616
> )
>
> org.apache.activemq.artemis.core.remoting.impl.netty.NettyConnector:
> AMQ214033: Cannot resolve host
> java.net.UnknownHostException: internal-cluster-dns-1
>
> The error is show 2 times every time the external consumer is started. The connection works until we power down cluster-dns-1 machine. Then the consumer just stops receiving messages and never reconnects to cluster-dns-2 node.
>
>
> When using the same consumer internally everything works as expected.
>
> --
> Vilius
>
> -----Original Message-----
> From: Domenico Francesco Bruscino <br...@gmail.com>
> Sent: Tuesday, January 4, 2022 8:02 PM
> To: users@activemq.apache.org
> Subject: Re: Artemis cluster topology and external clients
>
> Hi Vilius,
>
> the clients should disable topology for load balancing and use static connectors, i.e.
>
> (tcp://external-cluster-dns-1:61616,tcp://external-cluster-dns-2:61616
> )?ha=true&reconnectAttempts=30&useTopologyForLoadBalancing=false
>
> Regards,
> Domenico
>
> On Mon, 3 Jan 2022 at 10:00, Vilius Šumskas
> <v....@advantes.tech.invalid>
> wrote:
>
> > Hello list,
> >
> > we are trying to use Artemis HA shared storage cluster which our
> > SaaS application. In addition to consumers/producers internal to
> > SaaS application itself, we also have thousands of external
> > consumers/producers which are installed on client’s premises .
> >
> > As broadcast is not possible on Google Cloud we are using static
> > discovery configuration with these connectors:
> >
> > <connectors>
> > <!-- Connector used to be announced through cluster
> > connections and notifications -->
> > <connector
> > name="artemis-master">tcp://internal-cluster-dns-1:61616</connector>
> > <connector
> > name="artemis-slave">tcp://internal-cluster-dns2:61616</connector>
> > </connectors>
> >
> > Our acceptors are also configured to use internal DNS of the hosts
> > on both cluster nodes:
> >
> > <acceptor
> > name="artemis">tcp://internal-cluster-dns-1:61616?tcpSendBufferSize=
> > 10
> > 48576;tcpReceiveBufferSize=1048576;amqpMinLargeMessageSize=102400;pr
> > ot
> > ocols=CORE,AMQP,STOMP,HORNETQ,MQTT,OPENWIRE;useEpoll=true;amqpCredit
> > s=
> > 10
> >
> > 00;amqpLowCredits=300;amqpDuplicateDetection=true;supportAdvisory=fa
> > ls e;suppressInternalManagementObjects=false</acceptor>
> >
> > We don’t have issues with internal consumers/producers, however when
> > we try to connect external consumers (via external IP), they are
> > trying to connect via internal DNS which is probably set in the
> > cluster topology packet.
> >
> > This is probably expected and by design, but my question is how do
> > we correctly handle such case? We obviously do not want internal
> > clients to be served via external IP because external traffic is
> > expensive in the cloud and the performance would decrease
> > dramatically. Even with static discovery we would like to have a
> > possibility to expand our cluster in the future, i.e. use the topology so that clients are configured automatically.
> >
> > Do we need to have a split-DNS server so that external and internal
> > clients will see different IP addresses? Or maybe it is possible to
> > have the same node serving different acceptors on different ports
> > and different DNS names?
> >
> > Any pointers are much appreciated.
> >
> > --
> > Best Regards,
> >
> > Vilius Šumskas
> > Advantes technologies
> > IT manager
> > +370 614 75713
> >
> >
Re: Artemis cluster topology and external clients
Posted by Gary Tully <ga...@gmail.com>.
there is a problem here, currently there is no way to "transform" the
host information in the topology update commands. the topology is used
for ha and for loadbalancing and the client will always end up with
the internal dns name via the topology update.
This needs an enhancement, some thing along the lines of the what we
have in 5.x via the publishAddressStrategy for Artemis -
https://issues.apache.org/jira/browse/AMQ-6392
The only workaround is to forsake HA, ha=false and not do reconnects,
and have the two urls (with the external dns names) be candidates for
reconnection, as it will be an initial connection.
Something above the jms connection (the application, jms template)
will have to recreate the connection on failure.
Please open a JIRA issue to track this need, and we can peek into how
difficult it is to resolve or if there are some alternatives that can
help this use case.
On Mon, 10 Jan 2022 at 14:21, Vilius Šumskas <vi...@rivile.lt> wrote:
>
> Hi list,
>
> does anyone have more ideas regarding the issue with external consumers below?
>
> --
> Vilius
>
> -----Original Message-----
> From: Vilius Šumskas
> Sent: Friday, January 7, 2022 11:43 AM
> To: users@activemq.apache.org
> Subject: RE: Artemis cluster topology and external clients
>
> We are still getting these errors on the external clients even when useTopologyForLoadBalancing is disabled and using (tcp://external-cluster-dns-1:61616,tcp://external-cluster-dns-2:61616)
>
> org.apache.activemq.artemis.core.remoting.impl.netty.NettyConnector: AMQ214033: Cannot resolve host
> java.net.UnknownHostException: internal-cluster-dns-1
>
> The error is show 2 times every time the external consumer is started. The connection works until we power down cluster-dns-1 machine. Then the consumer just stops receiving messages and never reconnects to cluster-dns-2 node.
>
>
> When using the same consumer internally everything works as expected.
>
> --
> Vilius
>
> -----Original Message-----
> From: Domenico Francesco Bruscino <br...@gmail.com>
> Sent: Tuesday, January 4, 2022 8:02 PM
> To: users@activemq.apache.org
> Subject: Re: Artemis cluster topology and external clients
>
> Hi Vilius,
>
> the clients should disable topology for load balancing and use static connectors, i.e.
>
> (tcp://external-cluster-dns-1:61616,tcp://external-cluster-dns-2:61616)?ha=true&reconnectAttempts=30&useTopologyForLoadBalancing=false
>
> Regards,
> Domenico
>
> On Mon, 3 Jan 2022 at 10:00, Vilius Šumskas <v....@advantes.tech.invalid>
> wrote:
>
> > Hello list,
> >
> > we are trying to use Artemis HA shared storage cluster which our SaaS
> > application. In addition to consumers/producers internal to SaaS
> > application itself, we also have thousands of external
> > consumers/producers which are installed on client’s premises .
> >
> > As broadcast is not possible on Google Cloud we are using static
> > discovery configuration with these connectors:
> >
> > <connectors>
> > <!-- Connector used to be announced through cluster
> > connections and notifications -->
> > <connector
> > name="artemis-master">tcp://internal-cluster-dns-1:61616</connector>
> > <connector
> > name="artemis-slave">tcp://internal-cluster-dns2:61616</connector>
> > </connectors>
> >
> > Our acceptors are also configured to use internal DNS of the hosts on
> > both cluster nodes:
> >
> > <acceptor
> > name="artemis">tcp://internal-cluster-dns-1:61616?tcpSendBufferSize=10
> > 48576;tcpReceiveBufferSize=1048576;amqpMinLargeMessageSize=102400;prot
> > ocols=CORE,AMQP,STOMP,HORNETQ,MQTT,OPENWIRE;useEpoll=true;amqpCredits=
> > 10
> >
> > 00;amqpLowCredits=300;amqpDuplicateDetection=true;supportAdvisory=fals
> > e;suppressInternalManagementObjects=false</acceptor>
> >
> > We don’t have issues with internal consumers/producers, however when
> > we try to connect external consumers (via external IP), they are
> > trying to connect via internal DNS which is probably set in the
> > cluster topology packet.
> >
> > This is probably expected and by design, but my question is how do we
> > correctly handle such case? We obviously do not want internal clients
> > to be served via external IP because external traffic is expensive in
> > the cloud and the performance would decrease dramatically. Even with
> > static discovery we would like to have a possibility to expand our
> > cluster in the future, i.e. use the topology so that clients are configured automatically.
> >
> > Do we need to have a split-DNS server so that external and internal
> > clients will see different IP addresses? Or maybe it is possible to
> > have the same node serving different acceptors on different ports and
> > different DNS names?
> >
> > Any pointers are much appreciated.
> >
> > --
> > Best Regards,
> >
> > Vilius Šumskas
> > Advantes technologies
> > IT manager
> > +370 614 75713
> >
> >
RE: Artemis cluster topology and external clients
Posted by Vilius Šumskas <vi...@rivile.lt>.
Hi list,
does anyone have more ideas regarding the issue with external consumers below?
--
Vilius
-----Original Message-----
From: Vilius Šumskas
Sent: Friday, January 7, 2022 11:43 AM
To: users@activemq.apache.org
Subject: RE: Artemis cluster topology and external clients
We are still getting these errors on the external clients even when useTopologyForLoadBalancing is disabled and using (tcp://external-cluster-dns-1:61616,tcp://external-cluster-dns-2:61616)
org.apache.activemq.artemis.core.remoting.impl.netty.NettyConnector: AMQ214033: Cannot resolve host
java.net.UnknownHostException: internal-cluster-dns-1
The error is show 2 times every time the external consumer is started. The connection works until we power down cluster-dns-1 machine. Then the consumer just stops receiving messages and never reconnects to cluster-dns-2 node.
When using the same consumer internally everything works as expected.
--
Vilius
-----Original Message-----
From: Domenico Francesco Bruscino <br...@gmail.com>
Sent: Tuesday, January 4, 2022 8:02 PM
To: users@activemq.apache.org
Subject: Re: Artemis cluster topology and external clients
Hi Vilius,
the clients should disable topology for load balancing and use static connectors, i.e.
(tcp://external-cluster-dns-1:61616,tcp://external-cluster-dns-2:61616)?ha=true&reconnectAttempts=30&useTopologyForLoadBalancing=false
Regards,
Domenico
On Mon, 3 Jan 2022 at 10:00, Vilius Šumskas <v....@advantes.tech.invalid>
wrote:
> Hello list,
>
> we are trying to use Artemis HA shared storage cluster which our SaaS
> application. In addition to consumers/producers internal to SaaS
> application itself, we also have thousands of external
> consumers/producers which are installed on client’s premises .
>
> As broadcast is not possible on Google Cloud we are using static
> discovery configuration with these connectors:
>
> <connectors>
> <!-- Connector used to be announced through cluster
> connections and notifications -->
> <connector
> name="artemis-master">tcp://internal-cluster-dns-1:61616</connector>
> <connector
> name="artemis-slave">tcp://internal-cluster-dns2:61616</connector>
> </connectors>
>
> Our acceptors are also configured to use internal DNS of the hosts on
> both cluster nodes:
>
> <acceptor
> name="artemis">tcp://internal-cluster-dns-1:61616?tcpSendBufferSize=10
> 48576;tcpReceiveBufferSize=1048576;amqpMinLargeMessageSize=102400;prot
> ocols=CORE,AMQP,STOMP,HORNETQ,MQTT,OPENWIRE;useEpoll=true;amqpCredits=
> 10
>
> 00;amqpLowCredits=300;amqpDuplicateDetection=true;supportAdvisory=fals
> e;suppressInternalManagementObjects=false</acceptor>
>
> We don’t have issues with internal consumers/producers, however when
> we try to connect external consumers (via external IP), they are
> trying to connect via internal DNS which is probably set in the
> cluster topology packet.
>
> This is probably expected and by design, but my question is how do we
> correctly handle such case? We obviously do not want internal clients
> to be served via external IP because external traffic is expensive in
> the cloud and the performance would decrease dramatically. Even with
> static discovery we would like to have a possibility to expand our
> cluster in the future, i.e. use the topology so that clients are configured automatically.
>
> Do we need to have a split-DNS server so that external and internal
> clients will see different IP addresses? Or maybe it is possible to
> have the same node serving different acceptors on different ports and
> different DNS names?
>
> Any pointers are much appreciated.
>
> --
> Best Regards,
>
> Vilius Šumskas
> Advantes technologies
> IT manager
> +370 614 75713
>
>
RE: Artemis cluster topology and external clients
Posted by Vilius Šumskas <v....@advantes.tech.INVALID>.
We are still getting these errors on the external clients even when useTopologyForLoadBalancing is disabled and using (tcp://external-cluster-dns-1:61616,tcp://external-cluster-dns-2:61616)
org.apache.activemq.artemis.core.remoting.impl.netty.NettyConnector: AMQ214033: Cannot resolve host
java.net.UnknownHostException: internal-cluster-dns-1
The error is show 2 times every time the external consumer is started. The connection works until we power down cluster-dns-1 machine. Then the consumer just stops receiving messages and never reconnects to cluster-dns-2 node.
When using the same consumer internally everything works as expected.
--
Vilius
-----Original Message-----
From: Domenico Francesco Bruscino <br...@gmail.com>
Sent: Tuesday, January 4, 2022 8:02 PM
To: users@activemq.apache.org
Subject: Re: Artemis cluster topology and external clients
Hi Vilius,
the clients should disable topology for load balancing and use static connectors, i.e.
(tcp://external-cluster-dns-1:61616,tcp://external-cluster-dns-2:61616)?ha=true&reconnectAttempts=30&useTopologyForLoadBalancing=false
Regards,
Domenico
On Mon, 3 Jan 2022 at 10:00, Vilius Šumskas <v....@advantes.tech.invalid>
wrote:
> Hello list,
>
> we are trying to use Artemis HA shared storage cluster which our SaaS
> application. In addition to consumers/producers internal to SaaS
> application itself, we also have thousands of external
> consumers/producers which are installed on client’s premises .
>
> As broadcast is not possible on Google Cloud we are using static
> discovery configuration with these connectors:
>
> <connectors>
> <!-- Connector used to be announced through cluster
> connections and notifications -->
> <connector
> name="artemis-master">tcp://internal-cluster-dns-1:61616</connector>
> <connector
> name="artemis-slave">tcp://internal-cluster-dns2:61616</connector>
> </connectors>
>
> Our acceptors are also configured to use internal DNS of the hosts on
> both cluster nodes:
>
> <acceptor
> name="artemis">tcp://internal-cluster-dns-1:61616?tcpSendBufferSize=10
> 48576;tcpReceiveBufferSize=1048576;amqpMinLargeMessageSize=102400;prot
> ocols=CORE,AMQP,STOMP,HORNETQ,MQTT,OPENWIRE;useEpoll=true;amqpCredits=
> 10
>
> 00;amqpLowCredits=300;amqpDuplicateDetection=true;supportAdvisory=fals
> e;suppressInternalManagementObjects=false</acceptor>
>
> We don’t have issues with internal consumers/producers, however when
> we try to connect external consumers (via external IP), they are
> trying to connect via internal DNS which is probably set in the
> cluster topology packet.
>
> This is probably expected and by design, but my question is how do we
> correctly handle such case? We obviously do not want internal clients
> to be served via external IP because external traffic is expensive in
> the cloud and the performance would decrease dramatically. Even with
> static discovery we would like to have a possibility to expand our
> cluster in the future, i.e. use the topology so that clients are configured automatically.
>
> Do we need to have a split-DNS server so that external and internal
> clients will see different IP addresses? Or maybe it is possible to
> have the same node serving different acceptors on different ports and
> different DNS names?
>
> Any pointers are much appreciated.
>
> --
> Best Regards,
>
> Vilius Šumskas
> Advantes technologies
> IT manager
> +370 614 75713
>
>
RE: Artemis cluster topology and external clients
Posted by Vilius Šumskas <v....@advantes.tech.INVALID>.
Does this mean that I we have to plan for a possible future topology in advance? As far as I understand, we won't be able to add more DNS names once these external clients are deployed.
If static connectors with disabled topology are the only option I'm thinking to create 6 or maybe even 12 external DNS names which at the moment would resolve to the same 2 IP addresses our cluster currently uses. All external clients will be preconfigured with these DNS names. Later, when we will need to increase node count, we just add new node and adjust DNS accordingly. Does this sound like a feasible plan?
--
Vilius
-----Original Message-----
From: Domenico Francesco Bruscino <br...@gmail.com>
Sent: Tuesday, January 4, 2022 8:02 PM
To: users@activemq.apache.org
Subject: Re: Artemis cluster topology and external clients
Hi Vilius,
the clients should disable topology for load balancing and use static connectors, i.e.
(tcp://external-cluster-dns-1:61616,tcp://external-cluster-dns-2:61616)?ha=true&reconnectAttempts=30&useTopologyForLoadBalancing=false
Regards,
Domenico
On Mon, 3 Jan 2022 at 10:00, Vilius Šumskas <v....@advantes.tech.invalid>
wrote:
> Hello list,
>
> we are trying to use Artemis HA shared storage cluster which our SaaS
> application. In addition to consumers/producers internal to SaaS
> application itself, we also have thousands of external
> consumers/producers which are installed on client’s premises .
>
> As broadcast is not possible on Google Cloud we are using static
> discovery configuration with these connectors:
>
> <connectors>
> <!-- Connector used to be announced through cluster
> connections and notifications -->
> <connector
> name="artemis-master">tcp://internal-cluster-dns-1:61616</connector>
> <connector
> name="artemis-slave">tcp://internal-cluster-dns2:61616</connector>
> </connectors>
>
> Our acceptors are also configured to use internal DNS of the hosts on
> both cluster nodes:
>
> <acceptor
> name="artemis">tcp://internal-cluster-dns-1:61616?tcpSendBufferSize=10
> 48576;tcpReceiveBufferSize=1048576;amqpMinLargeMessageSize=102400;prot
> ocols=CORE,AMQP,STOMP,HORNETQ,MQTT,OPENWIRE;useEpoll=true;amqpCredits=
> 10
>
> 00;amqpLowCredits=300;amqpDuplicateDetection=true;supportAdvisory=fals
> e;suppressInternalManagementObjects=false</acceptor>
>
> We don’t have issues with internal consumers/producers, however when
> we try to connect external consumers (via external IP), they are
> trying to connect via internal DNS which is probably set in the
> cluster topology packet.
>
> This is probably expected and by design, but my question is how do we
> correctly handle such case? We obviously do not want internal clients
> to be served via external IP because external traffic is expensive in
> the cloud and the performance would decrease dramatically. Even with
> static discovery we would like to have a possibility to expand our
> cluster in the future, i.e. use the topology so that clients are configured automatically.
>
> Do we need to have a split-DNS server so that external and internal
> clients will see different IP addresses? Or maybe it is possible to
> have the same node serving different acceptors on different ports and
> different DNS names?
>
> Any pointers are much appreciated.
>
> --
> Best Regards,
>
> Vilius Šumskas
> Advantes technologies
> IT manager
> +370 614 75713
>
>
Re: Artemis cluster topology and external clients
Posted by Domenico Francesco Bruscino <br...@gmail.com>.
Hi Vilius,
the clients should disable topology for load balancing and use static
connectors, i.e.
(tcp://external-cluster-dns-1:61616,tcp://external-cluster-dns-2:61616)?ha=true&reconnectAttempts=30&useTopologyForLoadBalancing=false
Regards,
Domenico
On Mon, 3 Jan 2022 at 10:00, Vilius Šumskas <v....@advantes.tech.invalid>
wrote:
> Hello list,
>
> we are trying to use Artemis HA shared storage cluster which our SaaS
> application. In addition to consumers/producers internal to SaaS
> application itself, we also have thousands of external consumers/producers
> which are installed on client’s premises .
>
> As broadcast is not possible on Google Cloud we are using static discovery
> configuration with these connectors:
>
> <connectors>
> <!-- Connector used to be announced through cluster connections
> and notifications -->
> <connector
> name="artemis-master">tcp://internal-cluster-dns-1:61616</connector>
> <connector
> name="artemis-slave">tcp://internal-cluster-dns2:61616</connector>
> </connectors>
>
> Our acceptors are also configured to use internal DNS of the hosts on both
> cluster nodes:
>
> <acceptor
> name="artemis">tcp://internal-cluster-dns-1:61616?tcpSendBufferSize=1048576;tcpReceiveBufferSize=1048576;amqpMinLargeMessageSize=102400;protocols=CORE,AMQP,STOMP,HORNETQ,MQTT,OPENWIRE;useEpoll=true;amqpCredits=10
>
> 00;amqpLowCredits=300;amqpDuplicateDetection=true;supportAdvisory=false;suppressInternalManagementObjects=false</acceptor>
>
> We don’t have issues with internal consumers/producers, however when we
> try to connect external consumers (via external IP), they are trying to
> connect via internal DNS which is probably set in the cluster topology
> packet.
>
> This is probably expected and by design, but my question is how do we
> correctly handle such case? We obviously do not want internal clients to be
> served via external IP because external traffic is expensive in the cloud
> and the performance would decrease dramatically. Even with static discovery
> we would like to have a possibility to expand our cluster in the future,
> i.e. use the topology so that clients are configured automatically.
>
> Do we need to have a split-DNS server so that external and internal
> clients will see different IP addresses? Or maybe it is possible to have
> the same node serving different acceptors on different ports and different
> DNS names?
>
> Any pointers are much appreciated.
>
> --
> Best Regards,
>
> Vilius Šumskas
> Advantes technologies
> IT manager
> +370 614 75713
>
>