You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by Lachezar Dobrev <l....@gmail.com> on 2016/01/25 12:22:16 UTC
Cluster/Federated Artemis problems
Hello group members.
I'm having problems with clustering/federating an application's Artemis
embedded server.
The application is a .WAR with Springframework 4 and Embedded Artemis
1.1.0 (from Spring).
Multiple instances of the application are expected to be deployed in
multiple spots. The Artemis component is expected to cluster a JMS Topic
between nodes so that if any node sends a message on the topic all
listeners on all nodes should receive the message.
With a few problems I was able to make the Embedded Artemis Server handle
topic in a single deployment.
Every application connects to the Embedded Artemis using InVM connector.
Trying to cluster instances does not work!
My configuration contains:
- A NettyAcceptor on (host):(random-high-port)
- A NettyConnector on (host):(random-high-port) named "cluster-connector"
- A BroadcastGroupConfiguration named "cluster-broadcast"
= UDPBroadcastEndpointFactory
* group-host 239.1.2.3
* group-port 12345
* local-host (host)
* local-port (random-high-port)
= ConnectorInfos
* "cluster-connector"
- DiscoveryGroupConfiguration named "cluster-discovery"
= UDPBroadcastEndpointFactory
* group-host 239.1.2.3
* group-port 12345
* local-host (host)
* local-port (random-high-port)
- ClusterConnectionConfiguration named "cluster"
= address: "cluster-address"
= connectorName: "cluster-connector"
= discoveryGroupName: "cluster-discovery"
The configuration is done in Java (not XML) via
o.a.a.a.core.config.Configuration
As already noted this does not work, even worse when a second application
instance is brought up the VMs on both instances hang on attempt to
shut-down.
I noticed a possible problem: the network monitor showed, that the
applications keep open UDP socket that has a Send-Q that continuously grows
until about 200K pending, and then it seizes. Further using a tcpdump I
noticed, that the packages being sent by Artemis look invalid, as they're
really BIG: 4096 bytes broadcast datagrams!
Looking further I found out a possible BUG in
…cluster.impl.BroadcastGroupImpl in the broadcastConnectors() method. It
seems the method works incorrectly with the ActiveMQBuffer and the
underlying NIO ByteBuffer, and instead of sending a package with the needed
data it sends the whole Buffer of nearly 4K zeros and only a few hundred
bytes of actual payload. A package of 4K is next to impossible to send as a
datagram packet.
I've tried to perform a hot-fix for this issue and succeeded (the
datagrams fell to about 250 bytes), datagrams are sent and received, but
the cluster still does not form.
Please advise!
Is there a way to create a cluster without using discovery? Assuming I
know every instance of the application at initialisation time is it
possible to create a cluster of pre-defined nodes?
Hope I get help.
Re: Cluster/Federated Artemis problems
Posted by Lachezar Dobrev <l....@gmail.com>.
Passed.
I was having "minor" arguments with checkstyle.
2016-01-26 13:14 GMT+02:00 Lachezar Dobrev <l....@gmail.com>:
> Well, not that you have explained this it is really "Obvious".
> I'm using Artemis as JMS client+server, and internals are a bit obscure
> for me. I'll have to wrap my head around the fact, that there is more than
> JMS to that.
>
> Package size issue:
> https://issues.apache.org/jira/browse/ARTEMIS-362
>
> Git Pull-Request with fix and test case:
> https://github.com/apache/activemq-artemis/pull/348
>
> Jenkins does not like me however, The build passed, but the pull request
> is marked erroneous:
> https://builds.apache.org/job/ActiveMQ-Artemis-PR-Build/965/
>
> Please advise.
>
> Lachezar
>
> 2016-01-25 18:59 GMT+02:00 Justin Bertram <jb...@apache.com>:
>
>> The documentation is pretty clear (IMO) about the purpose of the
>> "address" property of the cluster connection. There's nothing "magic"
>> about it. In short, "Each cluster connection only applies to addresses
>> that match the specified address field. An address is matched on the
>> cluster connection when it begins with the string specified in this
>> field." The documentation itself [1] includes a more thorough explanation
>> (including a few simple examples), but I didn't want to paste the whole
>> thing here when you can simply go there and read it yourself.
>>
>> In your particular case, the value of "jms" works for you because you're
>> ostensibly using a JMS topic and all addresses for JMS topics are queues
>> are prefixed with "jms.". You can read more about how JMS maps to the
>> Artemis "core" API here [2].
>>
>> If you could open a JIRA [3] regarding the datagram issue you observed
>> and outline a way I can observe that myself that would go a long way in
>> making sure the issue is resolved. Thanks!
>>
>>
>> Justin
>>
>> [1] http://activemq.apache.org/artemis/docs/1.1.0/clusters.html (see the
>> "Configuring Cluster Connections" section)
>> [2] http://activemq.apache.org/artemis/docs/1.1.0/jms-core-mapping.html
>> [3] https://issues.apache.org/jira/browse/ARTEMIS
>>
>> ----- Original Message -----
>> From: "Lachezar Dobrev" <l....@gmail.com>
>> To: users@activemq.apache.org
>> Sent: Monday, January 25, 2016 10:35:36 AM
>> Subject: Re: Cluster/Federated Artemis problems
>>
>> Your help is appreciated, and effective.
>>
>> I have misunderstood the configuration elements. I was not clear that
>> the
>> acceptor and the connector describe the same end-point. After fixing that
>> I
>> saw a few logging messages about bridges being built.
>> The configuration help that you pointed me to showed an inexplicable (to
>> me) address for the cluster "jms".
>> After putting this *magic* address the cluster is now up and running.
>>
>> Now I'm trying to implement the fixed-peer cluster. I'm having trouble
>> with that, but the progress is good.
>>
>> I still believe sending 4K datagrams is a bug though.
>>
>>
>> 2016-01-25 17:02 GMT+02:00 Justin Bertram <jb...@apache.com>:
>>
>> > I imagine this is just a configuration issue somewhere. We ship lots of
>> > examples which use clustering (although not embedded), and the
>> test-suite
>> > is full of embedded clustering tests. As far as I know all these are
>> > working properly.
>> >
>> > When you say the NettyAcceptor and NettyConnector use a
>> "random-high-port"
>> > are they both using the same port? If not, that would be a problem.
>> >
>> > You can configure a cluster "statically" (i.e. without UDP discovery).
>> > Check out the "clustered-static-discovery" example and also see the
>> > documentation [1].
>> >
>> > I'd like to understand a bit more about how/why your current
>> configuration
>> > is failing. Could you provide a reproducible test-case?
>> >
>> > Lastly, I think you'd be best served by being on Artemis 1.2 which we
>> > recently released.
>> >
>> >
>> > Justin
>> >
>> > [1] http://activemq.apache.org/artemis/docs/1.1.0/clusters.html (see
>> the
>> > "Discovery using static Connectors" section)
>> >
>> > ----- Original Message -----
>> > From: "Lachezar Dobrev" <l....@gmail.com>
>> > To: users@activemq.apache.org
>> > Sent: Monday, January 25, 2016 5:22:16 AM
>> > Subject: Cluster/Federated Artemis problems
>> >
>> > Hello group members.
>> >
>> > I'm having problems with clustering/federating an application's
>> Artemis
>> > embedded server.
>> > The application is a .WAR with Springframework 4 and Embedded Artemis
>> > 1.1.0 (from Spring).
>> >
>> > Multiple instances of the application are expected to be deployed in
>> > multiple spots. The Artemis component is expected to cluster a JMS Topic
>> > between nodes so that if any node sends a message on the topic all
>> > listeners on all nodes should receive the message.
>> > With a few problems I was able to make the Embedded Artemis Server
>> handle
>> > topic in a single deployment.
>> >
>> > Every application connects to the Embedded Artemis using InVM
>> connector.
>> >
>> > Trying to cluster instances does not work!
>> >
>> > My configuration contains:
>> > - A NettyAcceptor on (host):(random-high-port)
>> > - A NettyConnector on (host):(random-high-port) named
>> "cluster-connector"
>> > - A BroadcastGroupConfiguration named "cluster-broadcast"
>> > = UDPBroadcastEndpointFactory
>> > * group-host 239.1.2.3
>> > * group-port 12345
>> > * local-host (host)
>> > * local-port (random-high-port)
>> > = ConnectorInfos
>> > * "cluster-connector"
>> > - DiscoveryGroupConfiguration named "cluster-discovery"
>> > = UDPBroadcastEndpointFactory
>> > * group-host 239.1.2.3
>> > * group-port 12345
>> > * local-host (host)
>> > * local-port (random-high-port)
>> > - ClusterConnectionConfiguration named "cluster"
>> > = address: "cluster-address"
>> > = connectorName: "cluster-connector"
>> > = discoveryGroupName: "cluster-discovery"
>> >
>> > The configuration is done in Java (not XML) via
>> > o.a.a.a.core.config.Configuration
>> >
>> > As already noted this does not work, even worse when a second
>> application
>> > instance is brought up the VMs on both instances hang on attempt to
>> > shut-down.
>> >
>> > I noticed a possible problem: the network monitor showed, that the
>> > applications keep open UDP socket that has a Send-Q that continuously
>> grows
>> > until about 200K pending, and then it seizes. Further using a tcpdump I
>> > noticed, that the packages being sent by Artemis look invalid, as
>> they're
>> > really BIG: 4096 bytes broadcast datagrams!
>> >
>> > Looking further I found out a possible BUG in
>> > …cluster.impl.BroadcastGroupImpl in the broadcastConnectors() method. It
>> > seems the method works incorrectly with the ActiveMQBuffer and the
>> > underlying NIO ByteBuffer, and instead of sending a package with the
>> needed
>> > data it sends the whole Buffer of nearly 4K zeros and only a few hundred
>> > bytes of actual payload. A package of 4K is next to impossible to send
>> as a
>> > datagram packet.
>> >
>> > I've tried to perform a hot-fix for this issue and succeeded (the
>> > datagrams fell to about 250 bytes), datagrams are sent and received, but
>> > the cluster still does not form.
>> >
>> > Please advise!
>> > Is there a way to create a cluster without using discovery? Assuming I
>> > know every instance of the application at initialisation time is it
>> > possible to create a cluster of pre-defined nodes?
>> >
>> > Hope I get help.
>> >
>>
>
>
Re: Cluster/Federated Artemis problems
Posted by Clebert Suconic <cl...@gmail.com>.
I believe Eclipse can import the checkstyle.xml natively as a setting.
I haven't been using eclipse for some time as Idea has been dealing
with maven projects in a better way.
If you use Idea we have the settings /etc... There as some Eclipse
settings there as well.
On Tue, Jan 26, 2016 at 10:47 AM, Lachezar Dobrev <l....@gmail.com> wrote:
> Thank you. I'll try to remember this.
> BTW. Is there any code style settings for ActiveMQ/Artemis for developers
> using Eclipse? The 3 space indentation got me broad-side…
>
> 2016-01-26 17:17 GMT+02:00 Clebert Suconic <cl...@gmail.com>:
>
>> just to let you know.. to build with checkstyle activated, you have to
>> build with the dev profile:
>>
>> mvn -Pdev install (etc).
>>
>>
>>
>> On Tue, Jan 26, 2016 at 6:14 AM, Lachezar Dobrev <l....@gmail.com>
>> wrote:
>> > Well, not that you have explained this it is really "Obvious".
>> > I'm using Artemis as JMS client+server, and internals are a bit obscure
>> > for me. I'll have to wrap my head around the fact, that there is more
>> than
>> > JMS to that.
>> >
>> > Package size issue:
>> > https://issues.apache.org/jira/browse/ARTEMIS-362
>> >
>> > Git Pull-Request with fix and test case:
>> > https://github.com/apache/activemq-artemis/pull/348
>> >
>> > Jenkins does not like me however, The build passed, but the pull
>> request
>> > is marked erroneous:
>> > https://builds.apache.org/job/ActiveMQ-Artemis-PR-Build/965/
>> >
>> > Please advise.
>> >
>> > Lachezar
>> >
>> > 2016-01-25 18:59 GMT+02:00 Justin Bertram <jb...@apache.com>:
>> >
>> >> The documentation is pretty clear (IMO) about the purpose of the
>> "address"
>> >> property of the cluster connection. There's nothing "magic" about it.
>> In
>> >> short, "Each cluster connection only applies to addresses that match the
>> >> specified address field. An address is matched on the cluster connection
>> >> when it begins with the string specified in this field." The
>> documentation
>> >> itself [1] includes a more thorough explanation (including a few simple
>> >> examples), but I didn't want to paste the whole thing here when you can
>> >> simply go there and read it yourself.
>> >>
>> >> In your particular case, the value of "jms" works for you because you're
>> >> ostensibly using a JMS topic and all addresses for JMS topics are queues
>> >> are prefixed with "jms.". You can read more about how JMS maps to the
>> >> Artemis "core" API here [2].
>> >>
>> >> If you could open a JIRA [3] regarding the datagram issue you observed
>> and
>> >> outline a way I can observe that myself that would go a long way in
>> making
>> >> sure the issue is resolved. Thanks!
>> >>
>> >>
>> >> Justin
>> >>
>> >> [1] http://activemq.apache.org/artemis/docs/1.1.0/clusters.html (see
>> the
>> >> "Configuring Cluster Connections" section)
>> >> [2] http://activemq.apache.org/artemis/docs/1.1.0/jms-core-mapping.html
>> >> [3] https://issues.apache.org/jira/browse/ARTEMIS
>> >>
>> >> ----- Original Message -----
>> >> From: "Lachezar Dobrev" <l....@gmail.com>
>> >> To: users@activemq.apache.org
>> >> Sent: Monday, January 25, 2016 10:35:36 AM
>> >> Subject: Re: Cluster/Federated Artemis problems
>> >>
>> >> Your help is appreciated, and effective.
>> >>
>> >> I have misunderstood the configuration elements. I was not clear that
>> the
>> >> acceptor and the connector describe the same end-point. After fixing
>> that I
>> >> saw a few logging messages about bridges being built.
>> >> The configuration help that you pointed me to showed an inexplicable
>> (to
>> >> me) address for the cluster "jms".
>> >> After putting this *magic* address the cluster is now up and running.
>> >>
>> >> Now I'm trying to implement the fixed-peer cluster. I'm having trouble
>> >> with that, but the progress is good.
>> >>
>> >> I still believe sending 4K datagrams is a bug though.
>> >>
>> >>
>> >> 2016-01-25 17:02 GMT+02:00 Justin Bertram <jb...@apache.com>:
>> >>
>> >> > I imagine this is just a configuration issue somewhere. We ship lots
>> of
>> >> > examples which use clustering (although not embedded), and the
>> test-suite
>> >> > is full of embedded clustering tests. As far as I know all these are
>> >> > working properly.
>> >> >
>> >> > When you say the NettyAcceptor and NettyConnector use a
>> >> "random-high-port"
>> >> > are they both using the same port? If not, that would be a problem.
>> >> >
>> >> > You can configure a cluster "statically" (i.e. without UDP discovery).
>> >> > Check out the "clustered-static-discovery" example and also see the
>> >> > documentation [1].
>> >> >
>> >> > I'd like to understand a bit more about how/why your current
>> >> configuration
>> >> > is failing. Could you provide a reproducible test-case?
>> >> >
>> >> > Lastly, I think you'd be best served by being on Artemis 1.2 which we
>> >> > recently released.
>> >> >
>> >> >
>> >> > Justin
>> >> >
>> >> > [1] http://activemq.apache.org/artemis/docs/1.1.0/clusters.html (see
>> the
>> >> > "Discovery using static Connectors" section)
>> >> >
>> >> > ----- Original Message -----
>> >> > From: "Lachezar Dobrev" <l....@gmail.com>
>> >> > To: users@activemq.apache.org
>> >> > Sent: Monday, January 25, 2016 5:22:16 AM
>> >> > Subject: Cluster/Federated Artemis problems
>> >> >
>> >> > Hello group members.
>> >> >
>> >> > I'm having problems with clustering/federating an application's
>> Artemis
>> >> > embedded server.
>> >> > The application is a .WAR with Springframework 4 and Embedded
>> Artemis
>> >> > 1.1.0 (from Spring).
>> >> >
>> >> > Multiple instances of the application are expected to be deployed in
>> >> > multiple spots. The Artemis component is expected to cluster a JMS
>> Topic
>> >> > between nodes so that if any node sends a message on the topic all
>> >> > listeners on all nodes should receive the message.
>> >> > With a few problems I was able to make the Embedded Artemis Server
>> >> handle
>> >> > topic in a single deployment.
>> >> >
>> >> > Every application connects to the Embedded Artemis using InVM
>> >> connector.
>> >> >
>> >> > Trying to cluster instances does not work!
>> >> >
>> >> > My configuration contains:
>> >> > - A NettyAcceptor on (host):(random-high-port)
>> >> > - A NettyConnector on (host):(random-high-port) named
>> >> "cluster-connector"
>> >> > - A BroadcastGroupConfiguration named "cluster-broadcast"
>> >> > = UDPBroadcastEndpointFactory
>> >> > * group-host 239.1.2.3
>> >> > * group-port 12345
>> >> > * local-host (host)
>> >> > * local-port (random-high-port)
>> >> > = ConnectorInfos
>> >> > * "cluster-connector"
>> >> > - DiscoveryGroupConfiguration named "cluster-discovery"
>> >> > = UDPBroadcastEndpointFactory
>> >> > * group-host 239.1.2.3
>> >> > * group-port 12345
>> >> > * local-host (host)
>> >> > * local-port (random-high-port)
>> >> > - ClusterConnectionConfiguration named "cluster"
>> >> > = address: "cluster-address"
>> >> > = connectorName: "cluster-connector"
>> >> > = discoveryGroupName: "cluster-discovery"
>> >> >
>> >> > The configuration is done in Java (not XML) via
>> >> > o.a.a.a.core.config.Configuration
>> >> >
>> >> > As already noted this does not work, even worse when a second
>> >> application
>> >> > instance is brought up the VMs on both instances hang on attempt to
>> >> > shut-down.
>> >> >
>> >> > I noticed a possible problem: the network monitor showed, that the
>> >> > applications keep open UDP socket that has a Send-Q that continuously
>> >> grows
>> >> > until about 200K pending, and then it seizes. Further using a tcpdump
>> I
>> >> > noticed, that the packages being sent by Artemis look invalid, as
>> they're
>> >> > really BIG: 4096 bytes broadcast datagrams!
>> >> >
>> >> > Looking further I found out a possible BUG in
>> >> > …cluster.impl.BroadcastGroupImpl in the broadcastConnectors() method.
>> It
>> >> > seems the method works incorrectly with the ActiveMQBuffer and the
>> >> > underlying NIO ByteBuffer, and instead of sending a package with the
>> >> needed
>> >> > data it sends the whole Buffer of nearly 4K zeros and only a few
>> hundred
>> >> > bytes of actual payload. A package of 4K is next to impossible to send
>> >> as a
>> >> > datagram packet.
>> >> >
>> >> > I've tried to perform a hot-fix for this issue and succeeded (the
>> >> > datagrams fell to about 250 bytes), datagrams are sent and received,
>> but
>> >> > the cluster still does not form.
>> >> >
>> >> > Please advise!
>> >> > Is there a way to create a cluster without using discovery?
>> Assuming I
>> >> > know every instance of the application at initialisation time is it
>> >> > possible to create a cluster of pre-defined nodes?
>> >> >
>> >> > Hope I get help.
>> >> >
>> >>
>>
>>
>>
>> --
>> Clebert Suconic
>>
--
Clebert Suconic
Re: Cluster/Federated Artemis problems
Posted by Justin Bertram <jb...@apache.com>.
I believe Artemis ships Eclipse preferences under the "etc" directory. I don't use Eclipse so I'm not 100% certain.
Justin
----- Original Message -----
From: "Lachezar Dobrev" <l....@gmail.com>
To: users@activemq.apache.org
Sent: Tuesday, January 26, 2016 9:47:20 AM
Subject: Re: Cluster/Federated Artemis problems
Thank you. I'll try to remember this.
BTW. Is there any code style settings for ActiveMQ/Artemis for developers
using Eclipse? The 3 space indentation got me broad-side…
2016-01-26 17:17 GMT+02:00 Clebert Suconic <cl...@gmail.com>:
> just to let you know.. to build with checkstyle activated, you have to
> build with the dev profile:
>
> mvn -Pdev install (etc).
>
>
>
> On Tue, Jan 26, 2016 at 6:14 AM, Lachezar Dobrev <l....@gmail.com>
> wrote:
> > Well, not that you have explained this it is really "Obvious".
> > I'm using Artemis as JMS client+server, and internals are a bit obscure
> > for me. I'll have to wrap my head around the fact, that there is more
> than
> > JMS to that.
> >
> > Package size issue:
> > https://issues.apache.org/jira/browse/ARTEMIS-362
> >
> > Git Pull-Request with fix and test case:
> > https://github.com/apache/activemq-artemis/pull/348
> >
> > Jenkins does not like me however, The build passed, but the pull
> request
> > is marked erroneous:
> > https://builds.apache.org/job/ActiveMQ-Artemis-PR-Build/965/
> >
> > Please advise.
> >
> > Lachezar
> >
> > 2016-01-25 18:59 GMT+02:00 Justin Bertram <jb...@apache.com>:
> >
> >> The documentation is pretty clear (IMO) about the purpose of the
> "address"
> >> property of the cluster connection. There's nothing "magic" about it.
> In
> >> short, "Each cluster connection only applies to addresses that match the
> >> specified address field. An address is matched on the cluster connection
> >> when it begins with the string specified in this field." The
> documentation
> >> itself [1] includes a more thorough explanation (including a few simple
> >> examples), but I didn't want to paste the whole thing here when you can
> >> simply go there and read it yourself.
> >>
> >> In your particular case, the value of "jms" works for you because you're
> >> ostensibly using a JMS topic and all addresses for JMS topics are queues
> >> are prefixed with "jms.". You can read more about how JMS maps to the
> >> Artemis "core" API here [2].
> >>
> >> If you could open a JIRA [3] regarding the datagram issue you observed
> and
> >> outline a way I can observe that myself that would go a long way in
> making
> >> sure the issue is resolved. Thanks!
> >>
> >>
> >> Justin
> >>
> >> [1] http://activemq.apache.org/artemis/docs/1.1.0/clusters.html (see
> the
> >> "Configuring Cluster Connections" section)
> >> [2] http://activemq.apache.org/artemis/docs/1.1.0/jms-core-mapping.html
> >> [3] https://issues.apache.org/jira/browse/ARTEMIS
> >>
> >> ----- Original Message -----
> >> From: "Lachezar Dobrev" <l....@gmail.com>
> >> To: users@activemq.apache.org
> >> Sent: Monday, January 25, 2016 10:35:36 AM
> >> Subject: Re: Cluster/Federated Artemis problems
> >>
> >> Your help is appreciated, and effective.
> >>
> >> I have misunderstood the configuration elements. I was not clear that
> the
> >> acceptor and the connector describe the same end-point. After fixing
> that I
> >> saw a few logging messages about bridges being built.
> >> The configuration help that you pointed me to showed an inexplicable
> (to
> >> me) address for the cluster "jms".
> >> After putting this *magic* address the cluster is now up and running.
> >>
> >> Now I'm trying to implement the fixed-peer cluster. I'm having trouble
> >> with that, but the progress is good.
> >>
> >> I still believe sending 4K datagrams is a bug though.
> >>
> >>
> >> 2016-01-25 17:02 GMT+02:00 Justin Bertram <jb...@apache.com>:
> >>
> >> > I imagine this is just a configuration issue somewhere. We ship lots
> of
> >> > examples which use clustering (although not embedded), and the
> test-suite
> >> > is full of embedded clustering tests. As far as I know all these are
> >> > working properly.
> >> >
> >> > When you say the NettyAcceptor and NettyConnector use a
> >> "random-high-port"
> >> > are they both using the same port? If not, that would be a problem.
> >> >
> >> > You can configure a cluster "statically" (i.e. without UDP discovery).
> >> > Check out the "clustered-static-discovery" example and also see the
> >> > documentation [1].
> >> >
> >> > I'd like to understand a bit more about how/why your current
> >> configuration
> >> > is failing. Could you provide a reproducible test-case?
> >> >
> >> > Lastly, I think you'd be best served by being on Artemis 1.2 which we
> >> > recently released.
> >> >
> >> >
> >> > Justin
> >> >
> >> > [1] http://activemq.apache.org/artemis/docs/1.1.0/clusters.html (see
> the
> >> > "Discovery using static Connectors" section)
> >> >
> >> > ----- Original Message -----
> >> > From: "Lachezar Dobrev" <l....@gmail.com>
> >> > To: users@activemq.apache.org
> >> > Sent: Monday, January 25, 2016 5:22:16 AM
> >> > Subject: Cluster/Federated Artemis problems
> >> >
> >> > Hello group members.
> >> >
> >> > I'm having problems with clustering/federating an application's
> Artemis
> >> > embedded server.
> >> > The application is a .WAR with Springframework 4 and Embedded
> Artemis
> >> > 1.1.0 (from Spring).
> >> >
> >> > Multiple instances of the application are expected to be deployed in
> >> > multiple spots. The Artemis component is expected to cluster a JMS
> Topic
> >> > between nodes so that if any node sends a message on the topic all
> >> > listeners on all nodes should receive the message.
> >> > With a few problems I was able to make the Embedded Artemis Server
> >> handle
> >> > topic in a single deployment.
> >> >
> >> > Every application connects to the Embedded Artemis using InVM
> >> connector.
> >> >
> >> > Trying to cluster instances does not work!
> >> >
> >> > My configuration contains:
> >> > - A NettyAcceptor on (host):(random-high-port)
> >> > - A NettyConnector on (host):(random-high-port) named
> >> "cluster-connector"
> >> > - A BroadcastGroupConfiguration named "cluster-broadcast"
> >> > = UDPBroadcastEndpointFactory
> >> > * group-host 239.1.2.3
> >> > * group-port 12345
> >> > * local-host (host)
> >> > * local-port (random-high-port)
> >> > = ConnectorInfos
> >> > * "cluster-connector"
> >> > - DiscoveryGroupConfiguration named "cluster-discovery"
> >> > = UDPBroadcastEndpointFactory
> >> > * group-host 239.1.2.3
> >> > * group-port 12345
> >> > * local-host (host)
> >> > * local-port (random-high-port)
> >> > - ClusterConnectionConfiguration named "cluster"
> >> > = address: "cluster-address"
> >> > = connectorName: "cluster-connector"
> >> > = discoveryGroupName: "cluster-discovery"
> >> >
> >> > The configuration is done in Java (not XML) via
> >> > o.a.a.a.core.config.Configuration
> >> >
> >> > As already noted this does not work, even worse when a second
> >> application
> >> > instance is brought up the VMs on both instances hang on attempt to
> >> > shut-down.
> >> >
> >> > I noticed a possible problem: the network monitor showed, that the
> >> > applications keep open UDP socket that has a Send-Q that continuously
> >> grows
> >> > until about 200K pending, and then it seizes. Further using a tcpdump
> I
> >> > noticed, that the packages being sent by Artemis look invalid, as
> they're
> >> > really BIG: 4096 bytes broadcast datagrams!
> >> >
> >> > Looking further I found out a possible BUG in
> >> > …cluster.impl.BroadcastGroupImpl in the broadcastConnectors() method.
> It
> >> > seems the method works incorrectly with the ActiveMQBuffer and the
> >> > underlying NIO ByteBuffer, and instead of sending a package with the
> >> needed
> >> > data it sends the whole Buffer of nearly 4K zeros and only a few
> hundred
> >> > bytes of actual payload. A package of 4K is next to impossible to send
> >> as a
> >> > datagram packet.
> >> >
> >> > I've tried to perform a hot-fix for this issue and succeeded (the
> >> > datagrams fell to about 250 bytes), datagrams are sent and received,
> but
> >> > the cluster still does not form.
> >> >
> >> > Please advise!
> >> > Is there a way to create a cluster without using discovery?
> Assuming I
> >> > know every instance of the application at initialisation time is it
> >> > possible to create a cluster of pre-defined nodes?
> >> >
> >> > Hope I get help.
> >> >
> >>
>
>
>
> --
> Clebert Suconic
>
Re: Cluster/Federated Artemis problems
Posted by Lachezar Dobrev <l....@gmail.com>.
Thank you. I'll try to remember this.
BTW. Is there any code style settings for ActiveMQ/Artemis for developers
using Eclipse? The 3 space indentation got me broad-side…
2016-01-26 17:17 GMT+02:00 Clebert Suconic <cl...@gmail.com>:
> just to let you know.. to build with checkstyle activated, you have to
> build with the dev profile:
>
> mvn -Pdev install (etc).
>
>
>
> On Tue, Jan 26, 2016 at 6:14 AM, Lachezar Dobrev <l....@gmail.com>
> wrote:
> > Well, not that you have explained this it is really "Obvious".
> > I'm using Artemis as JMS client+server, and internals are a bit obscure
> > for me. I'll have to wrap my head around the fact, that there is more
> than
> > JMS to that.
> >
> > Package size issue:
> > https://issues.apache.org/jira/browse/ARTEMIS-362
> >
> > Git Pull-Request with fix and test case:
> > https://github.com/apache/activemq-artemis/pull/348
> >
> > Jenkins does not like me however, The build passed, but the pull
> request
> > is marked erroneous:
> > https://builds.apache.org/job/ActiveMQ-Artemis-PR-Build/965/
> >
> > Please advise.
> >
> > Lachezar
> >
> > 2016-01-25 18:59 GMT+02:00 Justin Bertram <jb...@apache.com>:
> >
> >> The documentation is pretty clear (IMO) about the purpose of the
> "address"
> >> property of the cluster connection. There's nothing "magic" about it.
> In
> >> short, "Each cluster connection only applies to addresses that match the
> >> specified address field. An address is matched on the cluster connection
> >> when it begins with the string specified in this field." The
> documentation
> >> itself [1] includes a more thorough explanation (including a few simple
> >> examples), but I didn't want to paste the whole thing here when you can
> >> simply go there and read it yourself.
> >>
> >> In your particular case, the value of "jms" works for you because you're
> >> ostensibly using a JMS topic and all addresses for JMS topics are queues
> >> are prefixed with "jms.". You can read more about how JMS maps to the
> >> Artemis "core" API here [2].
> >>
> >> If you could open a JIRA [3] regarding the datagram issue you observed
> and
> >> outline a way I can observe that myself that would go a long way in
> making
> >> sure the issue is resolved. Thanks!
> >>
> >>
> >> Justin
> >>
> >> [1] http://activemq.apache.org/artemis/docs/1.1.0/clusters.html (see
> the
> >> "Configuring Cluster Connections" section)
> >> [2] http://activemq.apache.org/artemis/docs/1.1.0/jms-core-mapping.html
> >> [3] https://issues.apache.org/jira/browse/ARTEMIS
> >>
> >> ----- Original Message -----
> >> From: "Lachezar Dobrev" <l....@gmail.com>
> >> To: users@activemq.apache.org
> >> Sent: Monday, January 25, 2016 10:35:36 AM
> >> Subject: Re: Cluster/Federated Artemis problems
> >>
> >> Your help is appreciated, and effective.
> >>
> >> I have misunderstood the configuration elements. I was not clear that
> the
> >> acceptor and the connector describe the same end-point. After fixing
> that I
> >> saw a few logging messages about bridges being built.
> >> The configuration help that you pointed me to showed an inexplicable
> (to
> >> me) address for the cluster "jms".
> >> After putting this *magic* address the cluster is now up and running.
> >>
> >> Now I'm trying to implement the fixed-peer cluster. I'm having trouble
> >> with that, but the progress is good.
> >>
> >> I still believe sending 4K datagrams is a bug though.
> >>
> >>
> >> 2016-01-25 17:02 GMT+02:00 Justin Bertram <jb...@apache.com>:
> >>
> >> > I imagine this is just a configuration issue somewhere. We ship lots
> of
> >> > examples which use clustering (although not embedded), and the
> test-suite
> >> > is full of embedded clustering tests. As far as I know all these are
> >> > working properly.
> >> >
> >> > When you say the NettyAcceptor and NettyConnector use a
> >> "random-high-port"
> >> > are they both using the same port? If not, that would be a problem.
> >> >
> >> > You can configure a cluster "statically" (i.e. without UDP discovery).
> >> > Check out the "clustered-static-discovery" example and also see the
> >> > documentation [1].
> >> >
> >> > I'd like to understand a bit more about how/why your current
> >> configuration
> >> > is failing. Could you provide a reproducible test-case?
> >> >
> >> > Lastly, I think you'd be best served by being on Artemis 1.2 which we
> >> > recently released.
> >> >
> >> >
> >> > Justin
> >> >
> >> > [1] http://activemq.apache.org/artemis/docs/1.1.0/clusters.html (see
> the
> >> > "Discovery using static Connectors" section)
> >> >
> >> > ----- Original Message -----
> >> > From: "Lachezar Dobrev" <l....@gmail.com>
> >> > To: users@activemq.apache.org
> >> > Sent: Monday, January 25, 2016 5:22:16 AM
> >> > Subject: Cluster/Federated Artemis problems
> >> >
> >> > Hello group members.
> >> >
> >> > I'm having problems with clustering/federating an application's
> Artemis
> >> > embedded server.
> >> > The application is a .WAR with Springframework 4 and Embedded
> Artemis
> >> > 1.1.0 (from Spring).
> >> >
> >> > Multiple instances of the application are expected to be deployed in
> >> > multiple spots. The Artemis component is expected to cluster a JMS
> Topic
> >> > between nodes so that if any node sends a message on the topic all
> >> > listeners on all nodes should receive the message.
> >> > With a few problems I was able to make the Embedded Artemis Server
> >> handle
> >> > topic in a single deployment.
> >> >
> >> > Every application connects to the Embedded Artemis using InVM
> >> connector.
> >> >
> >> > Trying to cluster instances does not work!
> >> >
> >> > My configuration contains:
> >> > - A NettyAcceptor on (host):(random-high-port)
> >> > - A NettyConnector on (host):(random-high-port) named
> >> "cluster-connector"
> >> > - A BroadcastGroupConfiguration named "cluster-broadcast"
> >> > = UDPBroadcastEndpointFactory
> >> > * group-host 239.1.2.3
> >> > * group-port 12345
> >> > * local-host (host)
> >> > * local-port (random-high-port)
> >> > = ConnectorInfos
> >> > * "cluster-connector"
> >> > - DiscoveryGroupConfiguration named "cluster-discovery"
> >> > = UDPBroadcastEndpointFactory
> >> > * group-host 239.1.2.3
> >> > * group-port 12345
> >> > * local-host (host)
> >> > * local-port (random-high-port)
> >> > - ClusterConnectionConfiguration named "cluster"
> >> > = address: "cluster-address"
> >> > = connectorName: "cluster-connector"
> >> > = discoveryGroupName: "cluster-discovery"
> >> >
> >> > The configuration is done in Java (not XML) via
> >> > o.a.a.a.core.config.Configuration
> >> >
> >> > As already noted this does not work, even worse when a second
> >> application
> >> > instance is brought up the VMs on both instances hang on attempt to
> >> > shut-down.
> >> >
> >> > I noticed a possible problem: the network monitor showed, that the
> >> > applications keep open UDP socket that has a Send-Q that continuously
> >> grows
> >> > until about 200K pending, and then it seizes. Further using a tcpdump
> I
> >> > noticed, that the packages being sent by Artemis look invalid, as
> they're
> >> > really BIG: 4096 bytes broadcast datagrams!
> >> >
> >> > Looking further I found out a possible BUG in
> >> > …cluster.impl.BroadcastGroupImpl in the broadcastConnectors() method.
> It
> >> > seems the method works incorrectly with the ActiveMQBuffer and the
> >> > underlying NIO ByteBuffer, and instead of sending a package with the
> >> needed
> >> > data it sends the whole Buffer of nearly 4K zeros and only a few
> hundred
> >> > bytes of actual payload. A package of 4K is next to impossible to send
> >> as a
> >> > datagram packet.
> >> >
> >> > I've tried to perform a hot-fix for this issue and succeeded (the
> >> > datagrams fell to about 250 bytes), datagrams are sent and received,
> but
> >> > the cluster still does not form.
> >> >
> >> > Please advise!
> >> > Is there a way to create a cluster without using discovery?
> Assuming I
> >> > know every instance of the application at initialisation time is it
> >> > possible to create a cluster of pre-defined nodes?
> >> >
> >> > Hope I get help.
> >> >
> >>
>
>
>
> --
> Clebert Suconic
>
Re: Cluster/Federated Artemis problems
Posted by Clebert Suconic <cl...@gmail.com>.
just to let you know.. to build with checkstyle activated, you have to
build with the dev profile:
mvn -Pdev install (etc).
On Tue, Jan 26, 2016 at 6:14 AM, Lachezar Dobrev <l....@gmail.com> wrote:
> Well, not that you have explained this it is really "Obvious".
> I'm using Artemis as JMS client+server, and internals are a bit obscure
> for me. I'll have to wrap my head around the fact, that there is more than
> JMS to that.
>
> Package size issue:
> https://issues.apache.org/jira/browse/ARTEMIS-362
>
> Git Pull-Request with fix and test case:
> https://github.com/apache/activemq-artemis/pull/348
>
> Jenkins does not like me however, The build passed, but the pull request
> is marked erroneous:
> https://builds.apache.org/job/ActiveMQ-Artemis-PR-Build/965/
>
> Please advise.
>
> Lachezar
>
> 2016-01-25 18:59 GMT+02:00 Justin Bertram <jb...@apache.com>:
>
>> The documentation is pretty clear (IMO) about the purpose of the "address"
>> property of the cluster connection. There's nothing "magic" about it. In
>> short, "Each cluster connection only applies to addresses that match the
>> specified address field. An address is matched on the cluster connection
>> when it begins with the string specified in this field." The documentation
>> itself [1] includes a more thorough explanation (including a few simple
>> examples), but I didn't want to paste the whole thing here when you can
>> simply go there and read it yourself.
>>
>> In your particular case, the value of "jms" works for you because you're
>> ostensibly using a JMS topic and all addresses for JMS topics are queues
>> are prefixed with "jms.". You can read more about how JMS maps to the
>> Artemis "core" API here [2].
>>
>> If you could open a JIRA [3] regarding the datagram issue you observed and
>> outline a way I can observe that myself that would go a long way in making
>> sure the issue is resolved. Thanks!
>>
>>
>> Justin
>>
>> [1] http://activemq.apache.org/artemis/docs/1.1.0/clusters.html (see the
>> "Configuring Cluster Connections" section)
>> [2] http://activemq.apache.org/artemis/docs/1.1.0/jms-core-mapping.html
>> [3] https://issues.apache.org/jira/browse/ARTEMIS
>>
>> ----- Original Message -----
>> From: "Lachezar Dobrev" <l....@gmail.com>
>> To: users@activemq.apache.org
>> Sent: Monday, January 25, 2016 10:35:36 AM
>> Subject: Re: Cluster/Federated Artemis problems
>>
>> Your help is appreciated, and effective.
>>
>> I have misunderstood the configuration elements. I was not clear that the
>> acceptor and the connector describe the same end-point. After fixing that I
>> saw a few logging messages about bridges being built.
>> The configuration help that you pointed me to showed an inexplicable (to
>> me) address for the cluster "jms".
>> After putting this *magic* address the cluster is now up and running.
>>
>> Now I'm trying to implement the fixed-peer cluster. I'm having trouble
>> with that, but the progress is good.
>>
>> I still believe sending 4K datagrams is a bug though.
>>
>>
>> 2016-01-25 17:02 GMT+02:00 Justin Bertram <jb...@apache.com>:
>>
>> > I imagine this is just a configuration issue somewhere. We ship lots of
>> > examples which use clustering (although not embedded), and the test-suite
>> > is full of embedded clustering tests. As far as I know all these are
>> > working properly.
>> >
>> > When you say the NettyAcceptor and NettyConnector use a
>> "random-high-port"
>> > are they both using the same port? If not, that would be a problem.
>> >
>> > You can configure a cluster "statically" (i.e. without UDP discovery).
>> > Check out the "clustered-static-discovery" example and also see the
>> > documentation [1].
>> >
>> > I'd like to understand a bit more about how/why your current
>> configuration
>> > is failing. Could you provide a reproducible test-case?
>> >
>> > Lastly, I think you'd be best served by being on Artemis 1.2 which we
>> > recently released.
>> >
>> >
>> > Justin
>> >
>> > [1] http://activemq.apache.org/artemis/docs/1.1.0/clusters.html (see the
>> > "Discovery using static Connectors" section)
>> >
>> > ----- Original Message -----
>> > From: "Lachezar Dobrev" <l....@gmail.com>
>> > To: users@activemq.apache.org
>> > Sent: Monday, January 25, 2016 5:22:16 AM
>> > Subject: Cluster/Federated Artemis problems
>> >
>> > Hello group members.
>> >
>> > I'm having problems with clustering/federating an application's Artemis
>> > embedded server.
>> > The application is a .WAR with Springframework 4 and Embedded Artemis
>> > 1.1.0 (from Spring).
>> >
>> > Multiple instances of the application are expected to be deployed in
>> > multiple spots. The Artemis component is expected to cluster a JMS Topic
>> > between nodes so that if any node sends a message on the topic all
>> > listeners on all nodes should receive the message.
>> > With a few problems I was able to make the Embedded Artemis Server
>> handle
>> > topic in a single deployment.
>> >
>> > Every application connects to the Embedded Artemis using InVM
>> connector.
>> >
>> > Trying to cluster instances does not work!
>> >
>> > My configuration contains:
>> > - A NettyAcceptor on (host):(random-high-port)
>> > - A NettyConnector on (host):(random-high-port) named
>> "cluster-connector"
>> > - A BroadcastGroupConfiguration named "cluster-broadcast"
>> > = UDPBroadcastEndpointFactory
>> > * group-host 239.1.2.3
>> > * group-port 12345
>> > * local-host (host)
>> > * local-port (random-high-port)
>> > = ConnectorInfos
>> > * "cluster-connector"
>> > - DiscoveryGroupConfiguration named "cluster-discovery"
>> > = UDPBroadcastEndpointFactory
>> > * group-host 239.1.2.3
>> > * group-port 12345
>> > * local-host (host)
>> > * local-port (random-high-port)
>> > - ClusterConnectionConfiguration named "cluster"
>> > = address: "cluster-address"
>> > = connectorName: "cluster-connector"
>> > = discoveryGroupName: "cluster-discovery"
>> >
>> > The configuration is done in Java (not XML) via
>> > o.a.a.a.core.config.Configuration
>> >
>> > As already noted this does not work, even worse when a second
>> application
>> > instance is brought up the VMs on both instances hang on attempt to
>> > shut-down.
>> >
>> > I noticed a possible problem: the network monitor showed, that the
>> > applications keep open UDP socket that has a Send-Q that continuously
>> grows
>> > until about 200K pending, and then it seizes. Further using a tcpdump I
>> > noticed, that the packages being sent by Artemis look invalid, as they're
>> > really BIG: 4096 bytes broadcast datagrams!
>> >
>> > Looking further I found out a possible BUG in
>> > …cluster.impl.BroadcastGroupImpl in the broadcastConnectors() method. It
>> > seems the method works incorrectly with the ActiveMQBuffer and the
>> > underlying NIO ByteBuffer, and instead of sending a package with the
>> needed
>> > data it sends the whole Buffer of nearly 4K zeros and only a few hundred
>> > bytes of actual payload. A package of 4K is next to impossible to send
>> as a
>> > datagram packet.
>> >
>> > I've tried to perform a hot-fix for this issue and succeeded (the
>> > datagrams fell to about 250 bytes), datagrams are sent and received, but
>> > the cluster still does not form.
>> >
>> > Please advise!
>> > Is there a way to create a cluster without using discovery? Assuming I
>> > know every instance of the application at initialisation time is it
>> > possible to create a cluster of pre-defined nodes?
>> >
>> > Hope I get help.
>> >
>>
--
Clebert Suconic
Re: Cluster/Federated Artemis problems
Posted by Lachezar Dobrev <l....@gmail.com>.
Well, not that you have explained this it is really "Obvious".
I'm using Artemis as JMS client+server, and internals are a bit obscure
for me. I'll have to wrap my head around the fact, that there is more than
JMS to that.
Package size issue:
https://issues.apache.org/jira/browse/ARTEMIS-362
Git Pull-Request with fix and test case:
https://github.com/apache/activemq-artemis/pull/348
Jenkins does not like me however, The build passed, but the pull request
is marked erroneous:
https://builds.apache.org/job/ActiveMQ-Artemis-PR-Build/965/
Please advise.
Lachezar
2016-01-25 18:59 GMT+02:00 Justin Bertram <jb...@apache.com>:
> The documentation is pretty clear (IMO) about the purpose of the "address"
> property of the cluster connection. There's nothing "magic" about it. In
> short, "Each cluster connection only applies to addresses that match the
> specified address field. An address is matched on the cluster connection
> when it begins with the string specified in this field." The documentation
> itself [1] includes a more thorough explanation (including a few simple
> examples), but I didn't want to paste the whole thing here when you can
> simply go there and read it yourself.
>
> In your particular case, the value of "jms" works for you because you're
> ostensibly using a JMS topic and all addresses for JMS topics are queues
> are prefixed with "jms.". You can read more about how JMS maps to the
> Artemis "core" API here [2].
>
> If you could open a JIRA [3] regarding the datagram issue you observed and
> outline a way I can observe that myself that would go a long way in making
> sure the issue is resolved. Thanks!
>
>
> Justin
>
> [1] http://activemq.apache.org/artemis/docs/1.1.0/clusters.html (see the
> "Configuring Cluster Connections" section)
> [2] http://activemq.apache.org/artemis/docs/1.1.0/jms-core-mapping.html
> [3] https://issues.apache.org/jira/browse/ARTEMIS
>
> ----- Original Message -----
> From: "Lachezar Dobrev" <l....@gmail.com>
> To: users@activemq.apache.org
> Sent: Monday, January 25, 2016 10:35:36 AM
> Subject: Re: Cluster/Federated Artemis problems
>
> Your help is appreciated, and effective.
>
> I have misunderstood the configuration elements. I was not clear that the
> acceptor and the connector describe the same end-point. After fixing that I
> saw a few logging messages about bridges being built.
> The configuration help that you pointed me to showed an inexplicable (to
> me) address for the cluster "jms".
> After putting this *magic* address the cluster is now up and running.
>
> Now I'm trying to implement the fixed-peer cluster. I'm having trouble
> with that, but the progress is good.
>
> I still believe sending 4K datagrams is a bug though.
>
>
> 2016-01-25 17:02 GMT+02:00 Justin Bertram <jb...@apache.com>:
>
> > I imagine this is just a configuration issue somewhere. We ship lots of
> > examples which use clustering (although not embedded), and the test-suite
> > is full of embedded clustering tests. As far as I know all these are
> > working properly.
> >
> > When you say the NettyAcceptor and NettyConnector use a
> "random-high-port"
> > are they both using the same port? If not, that would be a problem.
> >
> > You can configure a cluster "statically" (i.e. without UDP discovery).
> > Check out the "clustered-static-discovery" example and also see the
> > documentation [1].
> >
> > I'd like to understand a bit more about how/why your current
> configuration
> > is failing. Could you provide a reproducible test-case?
> >
> > Lastly, I think you'd be best served by being on Artemis 1.2 which we
> > recently released.
> >
> >
> > Justin
> >
> > [1] http://activemq.apache.org/artemis/docs/1.1.0/clusters.html (see the
> > "Discovery using static Connectors" section)
> >
> > ----- Original Message -----
> > From: "Lachezar Dobrev" <l....@gmail.com>
> > To: users@activemq.apache.org
> > Sent: Monday, January 25, 2016 5:22:16 AM
> > Subject: Cluster/Federated Artemis problems
> >
> > Hello group members.
> >
> > I'm having problems with clustering/federating an application's Artemis
> > embedded server.
> > The application is a .WAR with Springframework 4 and Embedded Artemis
> > 1.1.0 (from Spring).
> >
> > Multiple instances of the application are expected to be deployed in
> > multiple spots. The Artemis component is expected to cluster a JMS Topic
> > between nodes so that if any node sends a message on the topic all
> > listeners on all nodes should receive the message.
> > With a few problems I was able to make the Embedded Artemis Server
> handle
> > topic in a single deployment.
> >
> > Every application connects to the Embedded Artemis using InVM
> connector.
> >
> > Trying to cluster instances does not work!
> >
> > My configuration contains:
> > - A NettyAcceptor on (host):(random-high-port)
> > - A NettyConnector on (host):(random-high-port) named
> "cluster-connector"
> > - A BroadcastGroupConfiguration named "cluster-broadcast"
> > = UDPBroadcastEndpointFactory
> > * group-host 239.1.2.3
> > * group-port 12345
> > * local-host (host)
> > * local-port (random-high-port)
> > = ConnectorInfos
> > * "cluster-connector"
> > - DiscoveryGroupConfiguration named "cluster-discovery"
> > = UDPBroadcastEndpointFactory
> > * group-host 239.1.2.3
> > * group-port 12345
> > * local-host (host)
> > * local-port (random-high-port)
> > - ClusterConnectionConfiguration named "cluster"
> > = address: "cluster-address"
> > = connectorName: "cluster-connector"
> > = discoveryGroupName: "cluster-discovery"
> >
> > The configuration is done in Java (not XML) via
> > o.a.a.a.core.config.Configuration
> >
> > As already noted this does not work, even worse when a second
> application
> > instance is brought up the VMs on both instances hang on attempt to
> > shut-down.
> >
> > I noticed a possible problem: the network monitor showed, that the
> > applications keep open UDP socket that has a Send-Q that continuously
> grows
> > until about 200K pending, and then it seizes. Further using a tcpdump I
> > noticed, that the packages being sent by Artemis look invalid, as they're
> > really BIG: 4096 bytes broadcast datagrams!
> >
> > Looking further I found out a possible BUG in
> > …cluster.impl.BroadcastGroupImpl in the broadcastConnectors() method. It
> > seems the method works incorrectly with the ActiveMQBuffer and the
> > underlying NIO ByteBuffer, and instead of sending a package with the
> needed
> > data it sends the whole Buffer of nearly 4K zeros and only a few hundred
> > bytes of actual payload. A package of 4K is next to impossible to send
> as a
> > datagram packet.
> >
> > I've tried to perform a hot-fix for this issue and succeeded (the
> > datagrams fell to about 250 bytes), datagrams are sent and received, but
> > the cluster still does not form.
> >
> > Please advise!
> > Is there a way to create a cluster without using discovery? Assuming I
> > know every instance of the application at initialisation time is it
> > possible to create a cluster of pre-defined nodes?
> >
> > Hope I get help.
> >
>
Re: Cluster/Federated Artemis problems
Posted by Justin Bertram <jb...@apache.com>.
The documentation is pretty clear (IMO) about the purpose of the "address" property of the cluster connection. There's nothing "magic" about it. In short, "Each cluster connection only applies to addresses that match the specified address field. An address is matched on the cluster connection when it begins with the string specified in this field." The documentation itself [1] includes a more thorough explanation (including a few simple examples), but I didn't want to paste the whole thing here when you can simply go there and read it yourself.
In your particular case, the value of "jms" works for you because you're ostensibly using a JMS topic and all addresses for JMS topics are queues are prefixed with "jms.". You can read more about how JMS maps to the Artemis "core" API here [2].
If you could open a JIRA [3] regarding the datagram issue you observed and outline a way I can observe that myself that would go a long way in making sure the issue is resolved. Thanks!
Justin
[1] http://activemq.apache.org/artemis/docs/1.1.0/clusters.html (see the "Configuring Cluster Connections" section)
[2] http://activemq.apache.org/artemis/docs/1.1.0/jms-core-mapping.html
[3] https://issues.apache.org/jira/browse/ARTEMIS
----- Original Message -----
From: "Lachezar Dobrev" <l....@gmail.com>
To: users@activemq.apache.org
Sent: Monday, January 25, 2016 10:35:36 AM
Subject: Re: Cluster/Federated Artemis problems
Your help is appreciated, and effective.
I have misunderstood the configuration elements. I was not clear that the
acceptor and the connector describe the same end-point. After fixing that I
saw a few logging messages about bridges being built.
The configuration help that you pointed me to showed an inexplicable (to
me) address for the cluster "jms".
After putting this *magic* address the cluster is now up and running.
Now I'm trying to implement the fixed-peer cluster. I'm having trouble
with that, but the progress is good.
I still believe sending 4K datagrams is a bug though.
2016-01-25 17:02 GMT+02:00 Justin Bertram <jb...@apache.com>:
> I imagine this is just a configuration issue somewhere. We ship lots of
> examples which use clustering (although not embedded), and the test-suite
> is full of embedded clustering tests. As far as I know all these are
> working properly.
>
> When you say the NettyAcceptor and NettyConnector use a "random-high-port"
> are they both using the same port? If not, that would be a problem.
>
> You can configure a cluster "statically" (i.e. without UDP discovery).
> Check out the "clustered-static-discovery" example and also see the
> documentation [1].
>
> I'd like to understand a bit more about how/why your current configuration
> is failing. Could you provide a reproducible test-case?
>
> Lastly, I think you'd be best served by being on Artemis 1.2 which we
> recently released.
>
>
> Justin
>
> [1] http://activemq.apache.org/artemis/docs/1.1.0/clusters.html (see the
> "Discovery using static Connectors" section)
>
> ----- Original Message -----
> From: "Lachezar Dobrev" <l....@gmail.com>
> To: users@activemq.apache.org
> Sent: Monday, January 25, 2016 5:22:16 AM
> Subject: Cluster/Federated Artemis problems
>
> Hello group members.
>
> I'm having problems with clustering/federating an application's Artemis
> embedded server.
> The application is a .WAR with Springframework 4 and Embedded Artemis
> 1.1.0 (from Spring).
>
> Multiple instances of the application are expected to be deployed in
> multiple spots. The Artemis component is expected to cluster a JMS Topic
> between nodes so that if any node sends a message on the topic all
> listeners on all nodes should receive the message.
> With a few problems I was able to make the Embedded Artemis Server handle
> topic in a single deployment.
>
> Every application connects to the Embedded Artemis using InVM connector.
>
> Trying to cluster instances does not work!
>
> My configuration contains:
> - A NettyAcceptor on (host):(random-high-port)
> - A NettyConnector on (host):(random-high-port) named "cluster-connector"
> - A BroadcastGroupConfiguration named "cluster-broadcast"
> = UDPBroadcastEndpointFactory
> * group-host 239.1.2.3
> * group-port 12345
> * local-host (host)
> * local-port (random-high-port)
> = ConnectorInfos
> * "cluster-connector"
> - DiscoveryGroupConfiguration named "cluster-discovery"
> = UDPBroadcastEndpointFactory
> * group-host 239.1.2.3
> * group-port 12345
> * local-host (host)
> * local-port (random-high-port)
> - ClusterConnectionConfiguration named "cluster"
> = address: "cluster-address"
> = connectorName: "cluster-connector"
> = discoveryGroupName: "cluster-discovery"
>
> The configuration is done in Java (not XML) via
> o.a.a.a.core.config.Configuration
>
> As already noted this does not work, even worse when a second application
> instance is brought up the VMs on both instances hang on attempt to
> shut-down.
>
> I noticed a possible problem: the network monitor showed, that the
> applications keep open UDP socket that has a Send-Q that continuously grows
> until about 200K pending, and then it seizes. Further using a tcpdump I
> noticed, that the packages being sent by Artemis look invalid, as they're
> really BIG: 4096 bytes broadcast datagrams!
>
> Looking further I found out a possible BUG in
> …cluster.impl.BroadcastGroupImpl in the broadcastConnectors() method. It
> seems the method works incorrectly with the ActiveMQBuffer and the
> underlying NIO ByteBuffer, and instead of sending a package with the needed
> data it sends the whole Buffer of nearly 4K zeros and only a few hundred
> bytes of actual payload. A package of 4K is next to impossible to send as a
> datagram packet.
>
> I've tried to perform a hot-fix for this issue and succeeded (the
> datagrams fell to about 250 bytes), datagrams are sent and received, but
> the cluster still does not form.
>
> Please advise!
> Is there a way to create a cluster without using discovery? Assuming I
> know every instance of the application at initialisation time is it
> possible to create a cluster of pre-defined nodes?
>
> Hope I get help.
>
Re: Cluster/Federated Artemis problems
Posted by Lachezar Dobrev <l....@gmail.com>.
Your help is appreciated, and effective.
I have misunderstood the configuration elements. I was not clear that the
acceptor and the connector describe the same end-point. After fixing that I
saw a few logging messages about bridges being built.
The configuration help that you pointed me to showed an inexplicable (to
me) address for the cluster "jms".
After putting this *magic* address the cluster is now up and running.
Now I'm trying to implement the fixed-peer cluster. I'm having trouble
with that, but the progress is good.
I still believe sending 4K datagrams is a bug though.
2016-01-25 17:02 GMT+02:00 Justin Bertram <jb...@apache.com>:
> I imagine this is just a configuration issue somewhere. We ship lots of
> examples which use clustering (although not embedded), and the test-suite
> is full of embedded clustering tests. As far as I know all these are
> working properly.
>
> When you say the NettyAcceptor and NettyConnector use a "random-high-port"
> are they both using the same port? If not, that would be a problem.
>
> You can configure a cluster "statically" (i.e. without UDP discovery).
> Check out the "clustered-static-discovery" example and also see the
> documentation [1].
>
> I'd like to understand a bit more about how/why your current configuration
> is failing. Could you provide a reproducible test-case?
>
> Lastly, I think you'd be best served by being on Artemis 1.2 which we
> recently released.
>
>
> Justin
>
> [1] http://activemq.apache.org/artemis/docs/1.1.0/clusters.html (see the
> "Discovery using static Connectors" section)
>
> ----- Original Message -----
> From: "Lachezar Dobrev" <l....@gmail.com>
> To: users@activemq.apache.org
> Sent: Monday, January 25, 2016 5:22:16 AM
> Subject: Cluster/Federated Artemis problems
>
> Hello group members.
>
> I'm having problems with clustering/federating an application's Artemis
> embedded server.
> The application is a .WAR with Springframework 4 and Embedded Artemis
> 1.1.0 (from Spring).
>
> Multiple instances of the application are expected to be deployed in
> multiple spots. The Artemis component is expected to cluster a JMS Topic
> between nodes so that if any node sends a message on the topic all
> listeners on all nodes should receive the message.
> With a few problems I was able to make the Embedded Artemis Server handle
> topic in a single deployment.
>
> Every application connects to the Embedded Artemis using InVM connector.
>
> Trying to cluster instances does not work!
>
> My configuration contains:
> - A NettyAcceptor on (host):(random-high-port)
> - A NettyConnector on (host):(random-high-port) named "cluster-connector"
> - A BroadcastGroupConfiguration named "cluster-broadcast"
> = UDPBroadcastEndpointFactory
> * group-host 239.1.2.3
> * group-port 12345
> * local-host (host)
> * local-port (random-high-port)
> = ConnectorInfos
> * "cluster-connector"
> - DiscoveryGroupConfiguration named "cluster-discovery"
> = UDPBroadcastEndpointFactory
> * group-host 239.1.2.3
> * group-port 12345
> * local-host (host)
> * local-port (random-high-port)
> - ClusterConnectionConfiguration named "cluster"
> = address: "cluster-address"
> = connectorName: "cluster-connector"
> = discoveryGroupName: "cluster-discovery"
>
> The configuration is done in Java (not XML) via
> o.a.a.a.core.config.Configuration
>
> As already noted this does not work, even worse when a second application
> instance is brought up the VMs on both instances hang on attempt to
> shut-down.
>
> I noticed a possible problem: the network monitor showed, that the
> applications keep open UDP socket that has a Send-Q that continuously grows
> until about 200K pending, and then it seizes. Further using a tcpdump I
> noticed, that the packages being sent by Artemis look invalid, as they're
> really BIG: 4096 bytes broadcast datagrams!
>
> Looking further I found out a possible BUG in
> …cluster.impl.BroadcastGroupImpl in the broadcastConnectors() method. It
> seems the method works incorrectly with the ActiveMQBuffer and the
> underlying NIO ByteBuffer, and instead of sending a package with the needed
> data it sends the whole Buffer of nearly 4K zeros and only a few hundred
> bytes of actual payload. A package of 4K is next to impossible to send as a
> datagram packet.
>
> I've tried to perform a hot-fix for this issue and succeeded (the
> datagrams fell to about 250 bytes), datagrams are sent and received, but
> the cluster still does not form.
>
> Please advise!
> Is there a way to create a cluster without using discovery? Assuming I
> know every instance of the application at initialisation time is it
> possible to create a cluster of pre-defined nodes?
>
> Hope I get help.
>
Re: Cluster/Federated Artemis problems
Posted by Justin Bertram <jb...@apache.com>.
I imagine this is just a configuration issue somewhere. We ship lots of examples which use clustering (although not embedded), and the test-suite is full of embedded clustering tests. As far as I know all these are working properly.
When you say the NettyAcceptor and NettyConnector use a "random-high-port" are they both using the same port? If not, that would be a problem.
You can configure a cluster "statically" (i.e. without UDP discovery). Check out the "clustered-static-discovery" example and also see the documentation [1].
I'd like to understand a bit more about how/why your current configuration is failing. Could you provide a reproducible test-case?
Lastly, I think you'd be best served by being on Artemis 1.2 which we recently released.
Justin
[1] http://activemq.apache.org/artemis/docs/1.1.0/clusters.html (see the "Discovery using static Connectors" section)
----- Original Message -----
From: "Lachezar Dobrev" <l....@gmail.com>
To: users@activemq.apache.org
Sent: Monday, January 25, 2016 5:22:16 AM
Subject: Cluster/Federated Artemis problems
Hello group members.
I'm having problems with clustering/federating an application's Artemis
embedded server.
The application is a .WAR with Springframework 4 and Embedded Artemis
1.1.0 (from Spring).
Multiple instances of the application are expected to be deployed in
multiple spots. The Artemis component is expected to cluster a JMS Topic
between nodes so that if any node sends a message on the topic all
listeners on all nodes should receive the message.
With a few problems I was able to make the Embedded Artemis Server handle
topic in a single deployment.
Every application connects to the Embedded Artemis using InVM connector.
Trying to cluster instances does not work!
My configuration contains:
- A NettyAcceptor on (host):(random-high-port)
- A NettyConnector on (host):(random-high-port) named "cluster-connector"
- A BroadcastGroupConfiguration named "cluster-broadcast"
= UDPBroadcastEndpointFactory
* group-host 239.1.2.3
* group-port 12345
* local-host (host)
* local-port (random-high-port)
= ConnectorInfos
* "cluster-connector"
- DiscoveryGroupConfiguration named "cluster-discovery"
= UDPBroadcastEndpointFactory
* group-host 239.1.2.3
* group-port 12345
* local-host (host)
* local-port (random-high-port)
- ClusterConnectionConfiguration named "cluster"
= address: "cluster-address"
= connectorName: "cluster-connector"
= discoveryGroupName: "cluster-discovery"
The configuration is done in Java (not XML) via
o.a.a.a.core.config.Configuration
As already noted this does not work, even worse when a second application
instance is brought up the VMs on both instances hang on attempt to
shut-down.
I noticed a possible problem: the network monitor showed, that the
applications keep open UDP socket that has a Send-Q that continuously grows
until about 200K pending, and then it seizes. Further using a tcpdump I
noticed, that the packages being sent by Artemis look invalid, as they're
really BIG: 4096 bytes broadcast datagrams!
Looking further I found out a possible BUG in
…cluster.impl.BroadcastGroupImpl in the broadcastConnectors() method. It
seems the method works incorrectly with the ActiveMQBuffer and the
underlying NIO ByteBuffer, and instead of sending a package with the needed
data it sends the whole Buffer of nearly 4K zeros and only a few hundred
bytes of actual payload. A package of 4K is next to impossible to send as a
datagram packet.
I've tried to perform a hot-fix for this issue and succeeded (the
datagrams fell to about 250 bytes), datagrams are sent and received, but
the cluster still does not form.
Please advise!
Is there a way to create a cluster without using discovery? Assuming I
know every instance of the application at initialisation time is it
possible to create a cluster of pre-defined nodes?
Hope I get help.