You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by Mohan Kumar <mk...@netsurion.com.INVALID> on 2020/12/14 07:57:11 UTC
ActiveMQ Artemis Client interface (AMQP vs REST)
Hi,
Based on the suggestion from Justin Bertram I am posting my query here.
We are new to ActiveMQ Artemis world.
When we started in ActiveMQ Artemis Broker we choose AMQP protocol to produce and consume data from broker.
But as per the suggestion we received from ActiveMQ Artemis consultant we switched from AMQP to REST interface.
Reason to switch from AMQP to REST
* As per the suggestion
Using AMQP, one of the problem we run into managing the connections come and go it is hard to get lot of insight into what's going on the broker.
So it is challenging to manage and troubleshoot.
broker aren't built with the intension of serving very large numbers of endpoints they built with the intension of moving messages quickly between endpoints.
Rest: The tooling which are available in HTTP, and for scaling and for frontend and it is really for superior to broker itself
* When we create 10000 connection using AMQP, It creates 10000 connections in Artemis broker(i.e. 10000 clients : 10000 connection in Artemis broker)
But using rest, 10000 clients connecting to HTTP server, creates 10000 connection at HTTP server and there is only one connection from HTTP server to REST interface.
So there is less load in broker(less number of connection in broker) and connection management comes to REST layer.
Our requirement is,
sensors (client which connect to Artemis server) in concurrent way we are using AMQP acceptor in broker We would like to know:
1. What would be maximum concurrent connection could be handled by Artemis broker (includes both publisher and subscriber)
2. Does broker connection has any dependency with acceptor (STOMP, AMQP, HTTP etc...)
Thanks,
Mohan
Re: ActiveMQ Artemis Client interface (AMQP vs REST)
Posted by Clebert Suconic <cl...@gmail.com>.
Each one will keep a socket open.. it's really an OS question...
I would create some sort of pool though.
On Wed, Dec 16, 2020 at 5:49 AM Mohan Kumar
<mk...@netsurion.com.invalid> wrote:
>
> Hi Clebert,
>
> Thanks for your reply.
>
> Is there any limit on number of concurrent connection at Artemis Broker level?
> Ex : If we connect 10000 persistent connection, does Artemis broker handle without any problem?
>
> Thanks,
> Mohan
>
> -----Original Message-----
> From: Clebert Suconic <cl...@gmail.com>
> Sent: Wednesday, December 16, 2020 7:41 AM
> To: users@activemq.apache.org
> Subject: Re: ActiveMQ Artemis Client interface (AMQP vs REST)
>
> CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe.
>
>
> I think you would be best on deploying your own Rest Interface.
>
> Like if you have a REST method that would then produce, have the producer on the Rest server.. and do your own facade.
>
> I haven't seen any commits on the REst Interface for Artemis in a while.
>
> That is, you use AMQP or whatever protocol at your REST endpoint...
> and you talk to the server through your implementation.
>
> On Mon, Dec 14, 2020 at 2:58 AM Mohan Kumar <mk...@netsurion.com.invalid> wrote:
> >
> > Hi,
> >
> > Based on the suggestion from Justin Bertram I am posting my query here.
> >
> > We are new to ActiveMQ Artemis world.
> > When we started in ActiveMQ Artemis Broker we choose AMQP protocol to produce and consume data from broker.
> > But as per the suggestion we received from ActiveMQ Artemis consultant we switched from AMQP to REST interface.
> >
> > Reason to switch from AMQP to REST
> >
> > * As per the suggestion
> >
> > Using AMQP, one of the problem we run into managing the connections come and go it is hard to get lot of insight into what's going on the broker.
> >
> > So it is challenging to manage and troubleshoot.
> >
> > broker aren't built with the intension of serving very large numbers of endpoints they built with the intension of moving messages quickly between endpoints.
> >
> > Rest: The tooling which are available in HTTP, and for scaling and for
> > frontend and it is really for superior to broker itself
> >
> >
> >
> >
> >
> > * When we create 10000 connection using AMQP, It creates 10000 connections in Artemis broker(i.e. 10000 clients : 10000 connection in Artemis broker)
> >
> > But using rest, 10000 clients connecting to HTTP server, creates 10000 connection at HTTP server and there is only one connection from HTTP server to REST interface.
> >
> > So there is less load in broker(less number of connection in broker) and connection management comes to REST layer.
> >
> >
> >
> >
> > Our requirement is,
> > sensors (client which connect to Artemis server) in concurrent way we are using AMQP acceptor in broker We would like to know:
> > 1. What would be maximum concurrent connection could be handled by
> > Artemis broker (includes both publisher and subscriber) 2. Does broker
> > connection has any dependency with acceptor (STOMP, AMQP, HTTP etc...)
> >
> > Thanks,
> > Mohan
> >
>
>
> --
> Clebert Suconic
--
Clebert Suconic
RE: ActiveMQ Artemis Client interface (AMQP vs REST)
Posted by Mohan Kumar <mk...@netsurion.com.INVALID>.
Hi Clebert,
Thanks for your reply.
Is there any limit on number of concurrent connection at Artemis Broker level?
Ex : If we connect 10000 persistent connection, does Artemis broker handle without any problem?
Thanks,
Mohan
-----Original Message-----
From: Clebert Suconic <cl...@gmail.com>
Sent: Wednesday, December 16, 2020 7:41 AM
To: users@activemq.apache.org
Subject: Re: ActiveMQ Artemis Client interface (AMQP vs REST)
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe.
I think you would be best on deploying your own Rest Interface.
Like if you have a REST method that would then produce, have the producer on the Rest server.. and do your own facade.
I haven't seen any commits on the REst Interface for Artemis in a while.
That is, you use AMQP or whatever protocol at your REST endpoint...
and you talk to the server through your implementation.
On Mon, Dec 14, 2020 at 2:58 AM Mohan Kumar <mk...@netsurion.com.invalid> wrote:
>
> Hi,
>
> Based on the suggestion from Justin Bertram I am posting my query here.
>
> We are new to ActiveMQ Artemis world.
> When we started in ActiveMQ Artemis Broker we choose AMQP protocol to produce and consume data from broker.
> But as per the suggestion we received from ActiveMQ Artemis consultant we switched from AMQP to REST interface.
>
> Reason to switch from AMQP to REST
>
> * As per the suggestion
>
> Using AMQP, one of the problem we run into managing the connections come and go it is hard to get lot of insight into what's going on the broker.
>
> So it is challenging to manage and troubleshoot.
>
> broker aren't built with the intension of serving very large numbers of endpoints they built with the intension of moving messages quickly between endpoints.
>
> Rest: The tooling which are available in HTTP, and for scaling and for
> frontend and it is really for superior to broker itself
>
>
>
>
>
> * When we create 10000 connection using AMQP, It creates 10000 connections in Artemis broker(i.e. 10000 clients : 10000 connection in Artemis broker)
>
> But using rest, 10000 clients connecting to HTTP server, creates 10000 connection at HTTP server and there is only one connection from HTTP server to REST interface.
>
> So there is less load in broker(less number of connection in broker) and connection management comes to REST layer.
>
>
>
>
> Our requirement is,
> sensors (client which connect to Artemis server) in concurrent way we are using AMQP acceptor in broker We would like to know:
> 1. What would be maximum concurrent connection could be handled by
> Artemis broker (includes both publisher and subscriber) 2. Does broker
> connection has any dependency with acceptor (STOMP, AMQP, HTTP etc...)
>
> Thanks,
> Mohan
>
--
Clebert Suconic
Re: ActiveMQ Artemis Client interface (AMQP vs REST)
Posted by Clebert Suconic <cl...@gmail.com>.
I think you would be best on deploying your own Rest Interface.
Like if you have a REST method that would then produce, have the
producer on the Rest server.. and do your own facade.
I haven't seen any commits on the REst Interface for Artemis in a while.
That is, you use AMQP or whatever protocol at your REST endpoint...
and you talk to the server through your implementation.
On Mon, Dec 14, 2020 at 2:58 AM Mohan Kumar
<mk...@netsurion.com.invalid> wrote:
>
> Hi,
>
> Based on the suggestion from Justin Bertram I am posting my query here.
>
> We are new to ActiveMQ Artemis world.
> When we started in ActiveMQ Artemis Broker we choose AMQP protocol to produce and consume data from broker.
> But as per the suggestion we received from ActiveMQ Artemis consultant we switched from AMQP to REST interface.
>
> Reason to switch from AMQP to REST
>
> * As per the suggestion
>
> Using AMQP, one of the problem we run into managing the connections come and go it is hard to get lot of insight into what's going on the broker.
>
> So it is challenging to manage and troubleshoot.
>
> broker aren't built with the intension of serving very large numbers of endpoints they built with the intension of moving messages quickly between endpoints.
>
> Rest: The tooling which are available in HTTP, and for scaling and for frontend and it is really for superior to broker itself
>
>
>
>
>
> * When we create 10000 connection using AMQP, It creates 10000 connections in Artemis broker(i.e. 10000 clients : 10000 connection in Artemis broker)
>
> But using rest, 10000 clients connecting to HTTP server, creates 10000 connection at HTTP server and there is only one connection from HTTP server to REST interface.
>
> So there is less load in broker(less number of connection in broker) and connection management comes to REST layer.
>
>
>
>
> Our requirement is,
> sensors (client which connect to Artemis server) in concurrent way we are using AMQP acceptor in broker We would like to know:
> 1. What would be maximum concurrent connection could be handled by Artemis broker (includes both publisher and subscriber)
> 2. Does broker connection has any dependency with acceptor (STOMP, AMQP, HTTP etc...)
>
> Thanks,
> Mohan
>
--
Clebert Suconic
Re: ActiveMQ Artemis Client interface (AMQP vs REST)
Posted by Francesco Nigro <ni...@gmail.com>.
As the exception suggest, the heap size seems not enough. I suggest to
raise it and print some GC logs to check the amount of stop of the world
pauses.
Il mer 23 dic 2020, 20:25 Mohan Kumar <mk...@netsurion.com.invalid> ha
scritto:
> Hi
>
> I have tried with 2200+ client application connects to Artemis server and
> publishing data in an interval of 30 sec(1100 messages per minute).
>
> At some point Artemis server was not responsive.
> Find the below logs.
>
> 2020-12-23 13:01:33,245 WARN [org.apache.activemq.artemis.core.server]
> AMQ222214: Destination *.Ax1.* has an inconsistent and negative address
> size=-227,980,495.
> 2020-12-23 13:01:33,518 WARN [org.apache.activemq.artemis.core.server]
> AMQ222214: Destination *.Ax1.* has an inconsistent and negative address
> size=-227,980,559.
> 2020-12-23 13:01:33,707 WARN [org.apache.activemq.artemis.core.server]
> AMQ222214: Destination *.Ax1.* has an inconsistent and negative address
> size=-227,980,495.
> 2020-12-23 13:01:33,707 WARN [org.apache.activemq.artemis.core.server]
> AMQ222214: Destination *.Ax1.* has an inconsistent and negative address
> size=-227,980,431.
> 2020-12-23 13:01:36,815 WARN
> [org.apache.activemq.artemis.utils.actors.OrderedExecutor] GC overhead
> limit exceeded: java.lang.OutOfMemoryError: GC overhead limit exceeded
>
> 2020-12-23 13:02:01,856 WARN
> [org.apache.activemq.artemis.utils.actors.OrderedExecutor] GC overhead
> limit exceeded: java.lang.OutOfMemoryError: GC overhead limit exceeded
>
> 2020-12-23 13:02:04,724 WARN
> [org.eclipse.jetty.util.thread.strategy.EatWhatYouKill] :
> java.lang.OutOfMemoryError: GC overhead limit exceeded
>
> 2020-12-23 13:01:45,404 WARN [org.eclipse.jetty.server.HttpChannel]
> /console/jolokia/: java.lang.OutOfMemoryError: GC overhead limit exceeded
>
> 2020-12-23 13:01:55,386 WARN
> [io.netty.util.concurrent.SingleThreadEventExecutor] Unexpected exception
> from an event executor: : java.lang.OutOfMemoryError: GC overhead limit
> exceeded
>
> 2020-12-23 13:01:52,191 WARN
> [org.apache.activemq.artemis.utils.actors.OrderedExecutor] GC overhead
> limit exceeded: java.lang.OutOfMemoryError: GC overhead limit exceeded
>
> 2020-12-23 13:01:35,482 WARN [org.apache.activemq.artemis.core.server]
> AMQ222214: Destination *.Ax1.* has an inconsistent and negative address
> size=-227,980,495.
> 2020-12-23 13:01:42,381 WARN [io.netty.channel.nio.NioEventLoop]
> Unexpected exception in the selector loop.: java.lang.OutOfMemoryError: GC
> overhead limit exceeded
>
> 2020-12-23 13:01:50,721 WARN [org.apache.activemq.artemis.core.server]
> AMQ222082: error on connection failure check: java.lang.OutOfMemoryError:
> GC overhead limit exceeded
>
> 2020-12-23 13:02:30,180 WARN
> [io.netty.channel.AbstractChannelHandlerContext] An exception
> 'java.lang.OutOfMemoryError: GC overhead limit exceeded' [enable DEBUG
> level for full stacktrace] was thrown by a user handler's exceptionCaught()
> method while handling the following exception:: java.lang.OutOfMemoryError:
> GC overhead limit exceeded
>
> 2020-12-23 13:02:03,396 WARN
> [org.apache.activemq.artemis.utils.actors.OrderedExecutor] GC overhead
> limit exceeded: java.lang.OutOfMemoryError: GC overhead limit exceeded
>
> 2020-12-23 13:01:55,649 WARN
> [io.netty.channel.AbstractChannelHandlerContext] An exception
> 'java.lang.OutOfMemoryError: GC overhead limit exceeded' [enable DEBUG
> level for full stacktrace] was thrown by a user handler's exceptionCaught()
> method while handling the following exception:: java.lang.OutOfMemoryError:
> GC overhead limit exceeded
>
> 2020-12-23 13:01:50,721 WARN
> [org.apache.activemq.artemis.utils.actors.OrderedExecutor] GC overhead
> limit exceeded: java.lang.OutOfMemoryError: GC overhead limit exceeded
>
> 2020-12-23 13:02:11,042 WARN
> [org.apache.activemq.artemis.utils.actors.OrderedExecutor] GC overhead
> limit exceeded: java.lang.OutOfMemoryError: GC overhead limit exceeded
>
> 2020-12-23 13:02:13,987 WARN [org.apache.activemq.artemis.core.server]
> AMQ222231: Failed to flush outstanding data from the connection:
> java.lang.OutOfMemoryError: GC overhead limit exceeded
>
> 2020-12-23 13:02:08,870 WARN
> [io.netty.channel.AbstractChannelHandlerContext] An exception
> 'java.lang.OutOfMemoryError: GC overhead limit exceeded' [enable DEBUG
> level for full stacktrace] was thrown by a user handler's exceptionCaught()
> method while handling the following exception:: java.lang.OutOfMemoryError:
> GC overhead limit exceeded
>
> 2020-12-23 13:02:31,963 WARN
> [org.apache.activemq.artemis.utils.actors.OrderedExecutor] GC overhead
> limit exceeded: java.lang.OutOfMemoryError: GC overhead limit exceeded
>
> 2020-12-23 13:01:54,186 WARN
> [io.netty.util.concurrent.AbstractEventExecutor] A task raised an
> exception. Task:
> org.apache.activemq.artemis.protocol.amqp.proton.AMQPConnectionContext$TickerRunnable@be67394:
> java.lang.OutOfMemoryError: GC overhead limit exceeded
>
> 2020-12-23 13:01:52,811 WARN
> [org.apache.activemq.artemis.utils.actors.OrderedExecutor] GC overhead
> limit exceeded: java.lang.OutOfMemoryError: GC overhead limit exceeded
>
> 2020-12-23 13:01:52,895 WARN [io.netty.channel.nio.NioEventLoop]
> Unexpected exception in the selector loop.: java.lang.OutOfMemoryError: GC
> overhead limit exceeded
>
> 2020-12-23 13:02:32,050 WARN
> [org.apache.activemq.artemis.utils.actors.OrderedExecutor] GC overhead
> limit exceeded: java.lang.OutOfMemoryError: GC overhead limit exceeded
>
> 2020-12-23 13:03:26,702 WARN
> [org.apache.activemq.artemis.utils.actors.OrderedExecutor] GC overhead
> limit exceeded: java.lang.OutOfMemoryError: GC overhead limit exceeded
>
> 2020-12-23 13:03:27,348 WARN
> [org.apache.activemq.artemis.utils.actors.OrderedExecutor] GC overhead
> limit exceeded: java.lang.OutOfMemoryError: GC overhead limit exceeded
>
>
>
> Thanks,
> Mohan
>
> ________________________________
> From: Justin Bertram <jb...@apache.org>
> Sent: Thursday, December 17, 2020 9:52 PM
> To: users@activemq.apache.org <us...@activemq.apache.org>
> Subject: Re: ActiveMQ Artemis Client interface (AMQP vs REST)
>
> CAUTION: This email originated from outside of the organization. Do not
> click links or open attachments unless you recognize the sender and know
> the content is safe.
>
>
> I recommend you ask about building on Windows on the Qpid user list [1].
>
> That said, even if you can't build on Windows you could potentially run it
> under WSL or even a full-blown VM.
>
>
> Justin
>
> [1] https://qpid.apache.org/discussion.html
>
> On Thu, Dec 17, 2020 at 2:49 AM Mohan Kumar <mk...@netsurion.com.invalid>
> wrote:
>
> > Thanks for your reply.
> >
> > As per the documentation QPID Dispatcher "Note : Dispatch Router will not
> > build on Windows."
> > Our applications mainly runs on Windows OS, is there any way to build and
> > run dispatcher application in Windows OS.
> >
> > Thanks,
> > Mohan
> >
> >
> >
> > -----Original Message-----
> > From: Justin Bertram <jb...@apache.org>
> > Sent: Thursday, December 17, 2020 7:11 AM
> > To: users@activemq.apache.org
> > Subject: Re: ActiveMQ Artemis Client interface (AMQP vs REST)
> >
> > CAUTION: This email originated from outside of the organization. Do not
> > click links or open attachments unless you recognize the sender and know
> > the content is safe.
> >
> >
> > You don't really provide any meaningful info about what exactly is
> > challenging to manage and troubleshoot with 10,000 connections.
> Therefore,
> > your idea of moving to REST in the first place could be fundamentally
> > misguided. The trouble (whatever it was) could be tied to something other
> > than connection management which may not be solved by offloading that
> work
> > to an HTTP server.
> >
> > Also, you flatly assert that Artemis wasn't built with the intention of
> > serving very large numbers of endpoints but simply with the intention of
> > moving messages quickly between endpoints. However, Artemis was designed
> > with high concurrency and vertical scalability in mind so I would push
> back
> > on your assertion here.
> >
> > There is no arbitrary limit for the number of connections which a broker
> > can support. The functional limit will depend on your hardware and your
> > use-case.
> >
> > I don't really understand your question about if the "broker connection
> > has any dependency with acceptor." Can you clarify this?
> >
> > Ultimately I would recommend against using the REST interface as a way to
> > offload connection management. It wasn't designed for this purpose so I
> > wouldn't expect it to scale well. In other words, I would expect
> > performance to be poor. Keep in mind that messaging connections (unlike
> > HTTP connections) are *stateful* and that state must be tracked
> somewhere.
> > You can't just switch to REST and expect that to go away.
> >
> > If you really need to scale up the number of AMQP connections beyond what
> > the broker is able to support I would recommend using Qpid Dispatch
> Router
> > [1] as a connection concentrator.
> >
> >
> > Justin
> >
> > [1] https://qpid.apache.org/components/dispatch-router/index.html
> >
> >
> >
> > On Mon, Dec 14, 2020 at 1:57 AM Mohan Kumar <mkumar@netsurion.com.invalid
> >
> > wrote:
> >
> > > Hi,
> > >
> > > Based on the suggestion from Justin Bertram I am posting my query here.
> > >
> > > We are new to ActiveMQ Artemis world.
> > > When we started in ActiveMQ Artemis Broker we choose AMQP protocol to
> > > produce and consume data from broker.
> > > But as per the suggestion we received from ActiveMQ Artemis consultant
> > > we switched from AMQP to REST interface.
> > >
> > > Reason to switch from AMQP to REST
> > >
> > > * As per the suggestion
> > >
> > > Using AMQP, one of the problem we run into managing the connections
> > > come and go it is hard to get lot of insight into what's going on the
> > broker.
> > >
> > > So it is challenging to manage and troubleshoot.
> > >
> > > broker aren't built with the intension of serving very large numbers
> > > of endpoints they built with the intension of moving messages quickly
> > > between endpoints.
> > >
> > > Rest: The tooling which are available in HTTP, and for scaling and for
> > > frontend and it is really for superior to broker itself
> > >
> > >
> > >
> > >
> > >
> > > * When we create 10000 connection using AMQP, It creates 10000
> > > connections in Artemis broker(i.e. 10000 clients : 10000 connection in
> > > Artemis broker)
> > >
> > > But using rest, 10000 clients connecting to HTTP server, creates 10000
> > > connection at HTTP server and there is only one connection from HTTP
> > > server to REST interface.
> > >
> > > So there is less load in broker(less number of connection in broker)
> > > and connection management comes to REST layer.
> > >
> > >
> > >
> > >
> > > Our requirement is,
> > > sensors (client which connect to Artemis server) in concurrent way we
> > > are using AMQP acceptor in broker We would like to know:
> > > 1. What would be maximum concurrent connection could be handled by
> > > Artemis broker (includes both publisher and subscriber) 2. Does broker
> > > connection has any dependency with acceptor (STOMP, AMQP, HTTP etc...)
> > >
> > > Thanks,
> > > Mohan
> > >
> > >
> >
>
Re: ActiveMQ Artemis Client interface (AMQP vs REST)
Posted by Mohan Kumar <mk...@netsurion.com.INVALID>.
Hi
I have tried with 2200+ client application connects to Artemis server and publishing data in an interval of 30 sec(1100 messages per minute).
At some point Artemis server was not responsive.
Find the below logs.
2020-12-23 13:01:33,245 WARN [org.apache.activemq.artemis.core.server] AMQ222214: Destination *.Ax1.* has an inconsistent and negative address size=-227,980,495.
2020-12-23 13:01:33,518 WARN [org.apache.activemq.artemis.core.server] AMQ222214: Destination *.Ax1.* has an inconsistent and negative address size=-227,980,559.
2020-12-23 13:01:33,707 WARN [org.apache.activemq.artemis.core.server] AMQ222214: Destination *.Ax1.* has an inconsistent and negative address size=-227,980,495.
2020-12-23 13:01:33,707 WARN [org.apache.activemq.artemis.core.server] AMQ222214: Destination *.Ax1.* has an inconsistent and negative address size=-227,980,431.
2020-12-23 13:01:36,815 WARN [org.apache.activemq.artemis.utils.actors.OrderedExecutor] GC overhead limit exceeded: java.lang.OutOfMemoryError: GC overhead limit exceeded
2020-12-23 13:02:01,856 WARN [org.apache.activemq.artemis.utils.actors.OrderedExecutor] GC overhead limit exceeded: java.lang.OutOfMemoryError: GC overhead limit exceeded
2020-12-23 13:02:04,724 WARN [org.eclipse.jetty.util.thread.strategy.EatWhatYouKill] : java.lang.OutOfMemoryError: GC overhead limit exceeded
2020-12-23 13:01:45,404 WARN [org.eclipse.jetty.server.HttpChannel] /console/jolokia/: java.lang.OutOfMemoryError: GC overhead limit exceeded
2020-12-23 13:01:55,386 WARN [io.netty.util.concurrent.SingleThreadEventExecutor] Unexpected exception from an event executor: : java.lang.OutOfMemoryError: GC overhead limit exceeded
2020-12-23 13:01:52,191 WARN [org.apache.activemq.artemis.utils.actors.OrderedExecutor] GC overhead limit exceeded: java.lang.OutOfMemoryError: GC overhead limit exceeded
2020-12-23 13:01:35,482 WARN [org.apache.activemq.artemis.core.server] AMQ222214: Destination *.Ax1.* has an inconsistent and negative address size=-227,980,495.
2020-12-23 13:01:42,381 WARN [io.netty.channel.nio.NioEventLoop] Unexpected exception in the selector loop.: java.lang.OutOfMemoryError: GC overhead limit exceeded
2020-12-23 13:01:50,721 WARN [org.apache.activemq.artemis.core.server] AMQ222082: error on connection failure check: java.lang.OutOfMemoryError: GC overhead limit exceeded
2020-12-23 13:02:30,180 WARN [io.netty.channel.AbstractChannelHandlerContext] An exception 'java.lang.OutOfMemoryError: GC overhead limit exceeded' [enable DEBUG level for full stacktrace] was thrown by a user handler's exceptionCaught() method while handling the following exception:: java.lang.OutOfMemoryError: GC overhead limit exceeded
2020-12-23 13:02:03,396 WARN [org.apache.activemq.artemis.utils.actors.OrderedExecutor] GC overhead limit exceeded: java.lang.OutOfMemoryError: GC overhead limit exceeded
2020-12-23 13:01:55,649 WARN [io.netty.channel.AbstractChannelHandlerContext] An exception 'java.lang.OutOfMemoryError: GC overhead limit exceeded' [enable DEBUG level for full stacktrace] was thrown by a user handler's exceptionCaught() method while handling the following exception:: java.lang.OutOfMemoryError: GC overhead limit exceeded
2020-12-23 13:01:50,721 WARN [org.apache.activemq.artemis.utils.actors.OrderedExecutor] GC overhead limit exceeded: java.lang.OutOfMemoryError: GC overhead limit exceeded
2020-12-23 13:02:11,042 WARN [org.apache.activemq.artemis.utils.actors.OrderedExecutor] GC overhead limit exceeded: java.lang.OutOfMemoryError: GC overhead limit exceeded
2020-12-23 13:02:13,987 WARN [org.apache.activemq.artemis.core.server] AMQ222231: Failed to flush outstanding data from the connection: java.lang.OutOfMemoryError: GC overhead limit exceeded
2020-12-23 13:02:08,870 WARN [io.netty.channel.AbstractChannelHandlerContext] An exception 'java.lang.OutOfMemoryError: GC overhead limit exceeded' [enable DEBUG level for full stacktrace] was thrown by a user handler's exceptionCaught() method while handling the following exception:: java.lang.OutOfMemoryError: GC overhead limit exceeded
2020-12-23 13:02:31,963 WARN [org.apache.activemq.artemis.utils.actors.OrderedExecutor] GC overhead limit exceeded: java.lang.OutOfMemoryError: GC overhead limit exceeded
2020-12-23 13:01:54,186 WARN [io.netty.util.concurrent.AbstractEventExecutor] A task raised an exception. Task: org.apache.activemq.artemis.protocol.amqp.proton.AMQPConnectionContext$TickerRunnable@be67394: java.lang.OutOfMemoryError: GC overhead limit exceeded
2020-12-23 13:01:52,811 WARN [org.apache.activemq.artemis.utils.actors.OrderedExecutor] GC overhead limit exceeded: java.lang.OutOfMemoryError: GC overhead limit exceeded
2020-12-23 13:01:52,895 WARN [io.netty.channel.nio.NioEventLoop] Unexpected exception in the selector loop.: java.lang.OutOfMemoryError: GC overhead limit exceeded
2020-12-23 13:02:32,050 WARN [org.apache.activemq.artemis.utils.actors.OrderedExecutor] GC overhead limit exceeded: java.lang.OutOfMemoryError: GC overhead limit exceeded
2020-12-23 13:03:26,702 WARN [org.apache.activemq.artemis.utils.actors.OrderedExecutor] GC overhead limit exceeded: java.lang.OutOfMemoryError: GC overhead limit exceeded
2020-12-23 13:03:27,348 WARN [org.apache.activemq.artemis.utils.actors.OrderedExecutor] GC overhead limit exceeded: java.lang.OutOfMemoryError: GC overhead limit exceeded
Thanks,
Mohan
________________________________
From: Justin Bertram <jb...@apache.org>
Sent: Thursday, December 17, 2020 9:52 PM
To: users@activemq.apache.org <us...@activemq.apache.org>
Subject: Re: ActiveMQ Artemis Client interface (AMQP vs REST)
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe.
I recommend you ask about building on Windows on the Qpid user list [1].
That said, even if you can't build on Windows you could potentially run it
under WSL or even a full-blown VM.
Justin
[1] https://qpid.apache.org/discussion.html
On Thu, Dec 17, 2020 at 2:49 AM Mohan Kumar <mk...@netsurion.com.invalid>
wrote:
> Thanks for your reply.
>
> As per the documentation QPID Dispatcher "Note : Dispatch Router will not
> build on Windows."
> Our applications mainly runs on Windows OS, is there any way to build and
> run dispatcher application in Windows OS.
>
> Thanks,
> Mohan
>
>
>
> -----Original Message-----
> From: Justin Bertram <jb...@apache.org>
> Sent: Thursday, December 17, 2020 7:11 AM
> To: users@activemq.apache.org
> Subject: Re: ActiveMQ Artemis Client interface (AMQP vs REST)
>
> CAUTION: This email originated from outside of the organization. Do not
> click links or open attachments unless you recognize the sender and know
> the content is safe.
>
>
> You don't really provide any meaningful info about what exactly is
> challenging to manage and troubleshoot with 10,000 connections. Therefore,
> your idea of moving to REST in the first place could be fundamentally
> misguided. The trouble (whatever it was) could be tied to something other
> than connection management which may not be solved by offloading that work
> to an HTTP server.
>
> Also, you flatly assert that Artemis wasn't built with the intention of
> serving very large numbers of endpoints but simply with the intention of
> moving messages quickly between endpoints. However, Artemis was designed
> with high concurrency and vertical scalability in mind so I would push back
> on your assertion here.
>
> There is no arbitrary limit for the number of connections which a broker
> can support. The functional limit will depend on your hardware and your
> use-case.
>
> I don't really understand your question about if the "broker connection
> has any dependency with acceptor." Can you clarify this?
>
> Ultimately I would recommend against using the REST interface as a way to
> offload connection management. It wasn't designed for this purpose so I
> wouldn't expect it to scale well. In other words, I would expect
> performance to be poor. Keep in mind that messaging connections (unlike
> HTTP connections) are *stateful* and that state must be tracked somewhere.
> You can't just switch to REST and expect that to go away.
>
> If you really need to scale up the number of AMQP connections beyond what
> the broker is able to support I would recommend using Qpid Dispatch Router
> [1] as a connection concentrator.
>
>
> Justin
>
> [1] https://qpid.apache.org/components/dispatch-router/index.html
>
>
>
> On Mon, Dec 14, 2020 at 1:57 AM Mohan Kumar <mk...@netsurion.com.invalid>
> wrote:
>
> > Hi,
> >
> > Based on the suggestion from Justin Bertram I am posting my query here.
> >
> > We are new to ActiveMQ Artemis world.
> > When we started in ActiveMQ Artemis Broker we choose AMQP protocol to
> > produce and consume data from broker.
> > But as per the suggestion we received from ActiveMQ Artemis consultant
> > we switched from AMQP to REST interface.
> >
> > Reason to switch from AMQP to REST
> >
> > * As per the suggestion
> >
> > Using AMQP, one of the problem we run into managing the connections
> > come and go it is hard to get lot of insight into what's going on the
> broker.
> >
> > So it is challenging to manage and troubleshoot.
> >
> > broker aren't built with the intension of serving very large numbers
> > of endpoints they built with the intension of moving messages quickly
> > between endpoints.
> >
> > Rest: The tooling which are available in HTTP, and for scaling and for
> > frontend and it is really for superior to broker itself
> >
> >
> >
> >
> >
> > * When we create 10000 connection using AMQP, It creates 10000
> > connections in Artemis broker(i.e. 10000 clients : 10000 connection in
> > Artemis broker)
> >
> > But using rest, 10000 clients connecting to HTTP server, creates 10000
> > connection at HTTP server and there is only one connection from HTTP
> > server to REST interface.
> >
> > So there is less load in broker(less number of connection in broker)
> > and connection management comes to REST layer.
> >
> >
> >
> >
> > Our requirement is,
> > sensors (client which connect to Artemis server) in concurrent way we
> > are using AMQP acceptor in broker We would like to know:
> > 1. What would be maximum concurrent connection could be handled by
> > Artemis broker (includes both publisher and subscriber) 2. Does broker
> > connection has any dependency with acceptor (STOMP, AMQP, HTTP etc...)
> >
> > Thanks,
> > Mohan
> >
> >
>
Re: ActiveMQ Artemis Client interface (AMQP vs REST)
Posted by Justin Bertram <jb...@apache.org>.
I recommend you ask about building on Windows on the Qpid user list [1].
That said, even if you can't build on Windows you could potentially run it
under WSL or even a full-blown VM.
Justin
[1] https://qpid.apache.org/discussion.html
On Thu, Dec 17, 2020 at 2:49 AM Mohan Kumar <mk...@netsurion.com.invalid>
wrote:
> Thanks for your reply.
>
> As per the documentation QPID Dispatcher "Note : Dispatch Router will not
> build on Windows."
> Our applications mainly runs on Windows OS, is there any way to build and
> run dispatcher application in Windows OS.
>
> Thanks,
> Mohan
>
>
>
> -----Original Message-----
> From: Justin Bertram <jb...@apache.org>
> Sent: Thursday, December 17, 2020 7:11 AM
> To: users@activemq.apache.org
> Subject: Re: ActiveMQ Artemis Client interface (AMQP vs REST)
>
> CAUTION: This email originated from outside of the organization. Do not
> click links or open attachments unless you recognize the sender and know
> the content is safe.
>
>
> You don't really provide any meaningful info about what exactly is
> challenging to manage and troubleshoot with 10,000 connections. Therefore,
> your idea of moving to REST in the first place could be fundamentally
> misguided. The trouble (whatever it was) could be tied to something other
> than connection management which may not be solved by offloading that work
> to an HTTP server.
>
> Also, you flatly assert that Artemis wasn't built with the intention of
> serving very large numbers of endpoints but simply with the intention of
> moving messages quickly between endpoints. However, Artemis was designed
> with high concurrency and vertical scalability in mind so I would push back
> on your assertion here.
>
> There is no arbitrary limit for the number of connections which a broker
> can support. The functional limit will depend on your hardware and your
> use-case.
>
> I don't really understand your question about if the "broker connection
> has any dependency with acceptor." Can you clarify this?
>
> Ultimately I would recommend against using the REST interface as a way to
> offload connection management. It wasn't designed for this purpose so I
> wouldn't expect it to scale well. In other words, I would expect
> performance to be poor. Keep in mind that messaging connections (unlike
> HTTP connections) are *stateful* and that state must be tracked somewhere.
> You can't just switch to REST and expect that to go away.
>
> If you really need to scale up the number of AMQP connections beyond what
> the broker is able to support I would recommend using Qpid Dispatch Router
> [1] as a connection concentrator.
>
>
> Justin
>
> [1] https://qpid.apache.org/components/dispatch-router/index.html
>
>
>
> On Mon, Dec 14, 2020 at 1:57 AM Mohan Kumar <mk...@netsurion.com.invalid>
> wrote:
>
> > Hi,
> >
> > Based on the suggestion from Justin Bertram I am posting my query here.
> >
> > We are new to ActiveMQ Artemis world.
> > When we started in ActiveMQ Artemis Broker we choose AMQP protocol to
> > produce and consume data from broker.
> > But as per the suggestion we received from ActiveMQ Artemis consultant
> > we switched from AMQP to REST interface.
> >
> > Reason to switch from AMQP to REST
> >
> > * As per the suggestion
> >
> > Using AMQP, one of the problem we run into managing the connections
> > come and go it is hard to get lot of insight into what's going on the
> broker.
> >
> > So it is challenging to manage and troubleshoot.
> >
> > broker aren't built with the intension of serving very large numbers
> > of endpoints they built with the intension of moving messages quickly
> > between endpoints.
> >
> > Rest: The tooling which are available in HTTP, and for scaling and for
> > frontend and it is really for superior to broker itself
> >
> >
> >
> >
> >
> > * When we create 10000 connection using AMQP, It creates 10000
> > connections in Artemis broker(i.e. 10000 clients : 10000 connection in
> > Artemis broker)
> >
> > But using rest, 10000 clients connecting to HTTP server, creates 10000
> > connection at HTTP server and there is only one connection from HTTP
> > server to REST interface.
> >
> > So there is less load in broker(less number of connection in broker)
> > and connection management comes to REST layer.
> >
> >
> >
> >
> > Our requirement is,
> > sensors (client which connect to Artemis server) in concurrent way we
> > are using AMQP acceptor in broker We would like to know:
> > 1. What would be maximum concurrent connection could be handled by
> > Artemis broker (includes both publisher and subscriber) 2. Does broker
> > connection has any dependency with acceptor (STOMP, AMQP, HTTP etc...)
> >
> > Thanks,
> > Mohan
> >
> >
>
RE: ActiveMQ Artemis Client interface (AMQP vs REST)
Posted by Mohan Kumar <mk...@netsurion.com.INVALID>.
Thanks for your reply.
As per the documentation QPID Dispatcher "Note : Dispatch Router will not build on Windows."
Our applications mainly runs on Windows OS, is there any way to build and run dispatcher application in Windows OS.
Thanks,
Mohan
-----Original Message-----
From: Justin Bertram <jb...@apache.org>
Sent: Thursday, December 17, 2020 7:11 AM
To: users@activemq.apache.org
Subject: Re: ActiveMQ Artemis Client interface (AMQP vs REST)
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe.
You don't really provide any meaningful info about what exactly is challenging to manage and troubleshoot with 10,000 connections. Therefore, your idea of moving to REST in the first place could be fundamentally misguided. The trouble (whatever it was) could be tied to something other than connection management which may not be solved by offloading that work to an HTTP server.
Also, you flatly assert that Artemis wasn't built with the intention of serving very large numbers of endpoints but simply with the intention of moving messages quickly between endpoints. However, Artemis was designed with high concurrency and vertical scalability in mind so I would push back on your assertion here.
There is no arbitrary limit for the number of connections which a broker can support. The functional limit will depend on your hardware and your use-case.
I don't really understand your question about if the "broker connection has any dependency with acceptor." Can you clarify this?
Ultimately I would recommend against using the REST interface as a way to offload connection management. It wasn't designed for this purpose so I wouldn't expect it to scale well. In other words, I would expect performance to be poor. Keep in mind that messaging connections (unlike HTTP connections) are *stateful* and that state must be tracked somewhere.
You can't just switch to REST and expect that to go away.
If you really need to scale up the number of AMQP connections beyond what the broker is able to support I would recommend using Qpid Dispatch Router [1] as a connection concentrator.
Justin
[1] https://qpid.apache.org/components/dispatch-router/index.html
On Mon, Dec 14, 2020 at 1:57 AM Mohan Kumar <mk...@netsurion.com.invalid>
wrote:
> Hi,
>
> Based on the suggestion from Justin Bertram I am posting my query here.
>
> We are new to ActiveMQ Artemis world.
> When we started in ActiveMQ Artemis Broker we choose AMQP protocol to
> produce and consume data from broker.
> But as per the suggestion we received from ActiveMQ Artemis consultant
> we switched from AMQP to REST interface.
>
> Reason to switch from AMQP to REST
>
> * As per the suggestion
>
> Using AMQP, one of the problem we run into managing the connections
> come and go it is hard to get lot of insight into what's going on the broker.
>
> So it is challenging to manage and troubleshoot.
>
> broker aren't built with the intension of serving very large numbers
> of endpoints they built with the intension of moving messages quickly
> between endpoints.
>
> Rest: The tooling which are available in HTTP, and for scaling and for
> frontend and it is really for superior to broker itself
>
>
>
>
>
> * When we create 10000 connection using AMQP, It creates 10000
> connections in Artemis broker(i.e. 10000 clients : 10000 connection in
> Artemis broker)
>
> But using rest, 10000 clients connecting to HTTP server, creates 10000
> connection at HTTP server and there is only one connection from HTTP
> server to REST interface.
>
> So there is less load in broker(less number of connection in broker)
> and connection management comes to REST layer.
>
>
>
>
> Our requirement is,
> sensors (client which connect to Artemis server) in concurrent way we
> are using AMQP acceptor in broker We would like to know:
> 1. What would be maximum concurrent connection could be handled by
> Artemis broker (includes both publisher and subscriber) 2. Does broker
> connection has any dependency with acceptor (STOMP, AMQP, HTTP etc...)
>
> Thanks,
> Mohan
>
>
Re: ActiveMQ Artemis Client interface (AMQP vs REST)
Posted by Justin Bertram <jb...@apache.org>.
You don't really provide any meaningful info about what exactly is
challenging to manage and troubleshoot with 10,000 connections. Therefore,
your idea of moving to REST in the first place could be fundamentally
misguided. The trouble (whatever it was) could be tied to something other
than connection management which may not be solved by offloading that work
to an HTTP server.
Also, you flatly assert that Artemis wasn't built with the intention of
serving very large numbers of endpoints but simply with the intention of
moving messages quickly between endpoints. However, Artemis was designed
with high concurrency and vertical scalability in mind so I would push back
on your assertion here.
There is no arbitrary limit for the number of connections which a broker
can support. The functional limit will depend on your hardware and your
use-case.
I don't really understand your question about if the "broker connection has
any dependency with acceptor." Can you clarify this?
Ultimately I would recommend against using the REST interface as a way to
offload connection management. It wasn't designed for this purpose so I
wouldn't expect it to scale well. In other words, I would expect
performance to be poor. Keep in mind that messaging connections (unlike
HTTP connections) are *stateful* and that state must be tracked somewhere.
You can't just switch to REST and expect that to go away.
If you really need to scale up the number of AMQP connections beyond what
the broker is able to support I would recommend using Qpid Dispatch Router
[1] as a connection concentrator.
Justin
[1] https://qpid.apache.org/components/dispatch-router/index.html
On Mon, Dec 14, 2020 at 1:57 AM Mohan Kumar <mk...@netsurion.com.invalid>
wrote:
> Hi,
>
> Based on the suggestion from Justin Bertram I am posting my query here.
>
> We are new to ActiveMQ Artemis world.
> When we started in ActiveMQ Artemis Broker we choose AMQP protocol to
> produce and consume data from broker.
> But as per the suggestion we received from ActiveMQ Artemis consultant we
> switched from AMQP to REST interface.
>
> Reason to switch from AMQP to REST
>
> * As per the suggestion
>
> Using AMQP, one of the problem we run into managing the connections come
> and go it is hard to get lot of insight into what's going on the broker.
>
> So it is challenging to manage and troubleshoot.
>
> broker aren't built with the intension of serving very large numbers of
> endpoints they built with the intension of moving messages quickly between
> endpoints.
>
> Rest: The tooling which are available in HTTP, and for scaling and for
> frontend and it is really for superior to broker itself
>
>
>
>
>
> * When we create 10000 connection using AMQP, It creates 10000
> connections in Artemis broker(i.e. 10000 clients : 10000 connection in
> Artemis broker)
>
> But using rest, 10000 clients connecting to HTTP server, creates 10000
> connection at HTTP server and there is only one connection from HTTP server
> to REST interface.
>
> So there is less load in broker(less number of connection in broker) and
> connection management comes to REST layer.
>
>
>
>
> Our requirement is,
> sensors (client which connect to Artemis server) in concurrent way we are
> using AMQP acceptor in broker We would like to know:
> 1. What would be maximum concurrent connection could be handled by Artemis
> broker (includes both publisher and subscriber)
> 2. Does broker connection has any dependency with acceptor (STOMP, AMQP,
> HTTP etc...)
>
> Thanks,
> Mohan
>
>