You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@qpid.apache.org by Morgan Lindqvist <mo...@ericsson.com> on 2016/01/08 16:34:16 UTC
[AMQP 1.0] Basic question on loadsharing due to high volume
Hi All,
I have a little problem understanding how load sharing is supported with
AMQP. Up till now that has not been an issue for me but now I will start
hitting the capacity on the HW where the server is running.
I have a number of producer and a number of consumers. The AMQP server in
the middle now have such a high load that it does not cope processing more
requests. What I would like to do is to solve this on the AMQP server side
without affecting the clients. I do however not understand how this can be
done.
The description on
http://qpid.apache.org/releases/qpid-cpp-0.34/cpp-broker/book/chap-Messaging_User_Guide-Broker_Federation.html
talk about using Federation for load balancing. My understanding is that all
the producers still talk to one AMQP instance that then forward the requests
to "slave" servers. When you hit the capacity of this one single instance
how does one scale the system even more?
When the load is shared between multiple slave AMQP servers, are then all
the consumers required to connect to each of these slave servers to fetch
the messages? So if the load is increased and I need to add another AMQP
slave server all the consumers need to be re-configured to also fetch
messages from this one?
Is there an other way that is better for scaling a system when one have a
lot of data?
Best Regards,
Morgan
--
View this message in context: http://qpid.2158936.n2.nabble.com/AMQP-1-0-Basic-question-on-loadsharing-due-to-high-volume-tp7636285.html
Sent from the Apache Qpid users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: [AMQP 1.0] Basic question on loadsharing due to high volume
Posted by Ted Ross <tr...@redhat.com>.
Hi Morgan,
Your issue is one of the major drivers behind the Qpid Dispatch Router
project (http://qpid.apache.org/components/dispatch-router/index.html).
The architectural premise behind Dispatch Router is that message
routing and message brokering/queuing are separate activities that
should be handled by separate components.
One of the key use-cases for the router is to place one or more routers
between the messaging clients and the brokers. This provides several
scaling benefits:
- Messages that do not require queuing may be transferred directly
from endpoint to endpoint without passing through a broker. This
transfer may be multicast (all subscribers for an address receive
one copy of every produced message) or anycast (every produced
message is delivered to one subscriber).
- Multiple brokers can provide queues and/or topics on the same
address. The router network can then distribute message traffic
across the brokers allowing for horizontal scaling without a single
choke point for message flow.
I'd be happy to provide more detail if you are interested.
Regards,
-Ted
On 01/08/2016 10:34 AM, Morgan Lindqvist wrote:
> Hi All,
>
> I have a little problem understanding how load sharing is supported with
> AMQP. Up till now that has not been an issue for me but now I will start
> hitting the capacity on the HW where the server is running.
>
> I have a number of producer and a number of consumers. The AMQP server in
> the middle now have such a high load that it does not cope processing more
> requests. What I would like to do is to solve this on the AMQP server side
> without affecting the clients. I do however not understand how this can be
> done.
>
> The description on
> http://qpid.apache.org/releases/qpid-cpp-0.34/cpp-broker/book/chap-Messaging_User_Guide-Broker_Federation.html
> talk about using Federation for load balancing. My understanding is that all
> the producers still talk to one AMQP instance that then forward the requests
> to "slave" servers. When you hit the capacity of this one single instance
> how does one scale the system even more?
>
> When the load is shared between multiple slave AMQP servers, are then all
> the consumers required to connect to each of these slave servers to fetch
> the messages? So if the load is increased and I need to add another AMQP
> slave server all the consumers need to be re-configured to also fetch
> messages from this one?
>
> Is there an other way that is better for scaling a system when one have a
> lot of data?
>
> Best Regards,
> Morgan
>
>
>
>
> --
> View this message in context: http://qpid.2158936.n2.nabble.com/AMQP-1-0-Basic-question-on-loadsharing-due-to-high-volume-tp7636285.html
> Sent from the Apache Qpid users mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org