You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by John Palmisano <Jo...@tidalsoftware.com> on 2023/02/06 14:26:01 UTC
ActiveMQ in the cloud
Hello,
How do I properly configure ActiveMQ, with respect to ports, etc., so that queues will work properly within a cloud environment?
Specifically, I want to ensure that the acknowledgements sent from consumers are being sent/received successfully so that queued messages do not build up.
Re: ActiveMQ in the cloud
Posted by Łukasz Dywicki <lu...@code-house.org>.
Speaking about ActiveMQ 5.x, try enabling additional advisory for
messages which are disabled by default for performance reasons. [1]
Especially advisoryForDelivery, advisoryForConsumed may help with
indicating if any messages at all are being delivered by broker to its
consumers. Watching producers and consumers may also help you indicating
if they are registered properly.
Do you have any settings applied to client or suspicious log entries on
broker side?
Best,
Łukasz
[1] https://activemq.apache.org/advisory-message.html
On 6.02.2023 17:29, John Palmisano wrote:
> The configuration is the same, except that the master and each of the consumers is running within their own respective containers. As a side note, we override the default 61616 port with another port in both environments.
>
> Messages are getting consumed and processed, but I have no way of determining if the acks themselves are sent/received. How should I configure the logging to determine this?
>
> -----Original Message-----
> From: Justin Bertram <jb...@apache.org>
> Sent: Monday, February 6, 2023 10:11 AM
> To: users@activemq.apache.org
> Subject: Re: ActiveMQ in the cloud
>
> I wouldn't expect any difference with regard to how messages are acknowledged between cloud and non-cloud environments. Are the master and consumer processes configured the same way in both environments? Are they messages being consumed and not acknowledged or not consumed at all?
>
>
> Justin
>
> On Mon, Feb 6, 2023 at 10:02 AM John Palmisano < John.Palmisano@tidalsoftware.com> wrote:
>
>> Let me clarify the issue we are running into:
>>
>> We have a master process which contains an embedded broker. This
>> process communicates with one or more consumer processes via ActiveMQ.
>> Messages are queued on the master side (per consumer) and processed by
>> each of the consumer processes. For some reason, within a cloud
>> environment, the queued messages are building up in the master process
>> (i.e. never cleaned up), up to the point where the master process runs
>> out of memory. This scenario does not happen within a non-cloud environment.
>>
>> Any insights on why this would be happening?
>>
>> -----Original Message-----
>> From: Justin Bertram <jb...@apache.org>
>> Sent: Monday, February 6, 2023 9:11 AM
>> To: users@activemq.apache.org
>> Subject: Re: ActiveMQ in the cloud
>>
>> The exact port depends on the protocol you're using, but all messaging
>> protocols supported by the broker use the same connection for
>> everything (i.e sending messages, receiving messages, acknowledging
>> messages, etc.) so the port won't change.
>>
>> Here's the default port used by each supported protocol:
>> - OpenWire (JMS) - 61616
>> - AMQP - 5672
>> - STOMP - 61613
>> - MQTT - 1883
>>
>>
>> Justin
>>
>> On Mon, Feb 6, 2023 at 9:01 AM John Palmisano <
>> John.Palmisano@tidalsoftware.com> wrote:
>>
>>> On what port(s) are acknowledgements sent back?
>>>
>>> -----Original Message-----
>>> From: Justin Bertram <jb...@apache.org>
>>> Sent: Monday, February 6, 2023 8:48 AM
>>> To: users@activemq.apache.org
>>> Subject: Re: ActiveMQ in the cloud
>>>
>>> Your question is pretty vague, but the main thing to remember is
>>> that the cloud is really no different from any other environment.
>>> The host and port must be reachable across the network. As long as
>>> that is true then clients can connect, consume, and acknowledge messages.
>>>
>>>
>>> Justin
>>>
>>> On Mon, Feb 6, 2023 at 8:26 AM John Palmisano <
>>> John.Palmisano@tidalsoftware.com> wrote:
>>>
>>>> Hello,
>>>>
>>>> How do I properly configure ActiveMQ, with respect to ports, etc.,
>>>> so that queues will work properly within a cloud environment?
>>>>
>>>> Specifically, I want to ensure that the acknowledgements sent from
>>>> consumers are being sent/received successfully so that queued
>>>> messages do not build up.
>>>>
>>>
>>
RE: ActiveMQ in the cloud
Posted by John Palmisano <Jo...@tidalsoftware.com>.
The configuration is the same, except that the master and each of the consumers is running within their own respective containers. As a side note, we override the default 61616 port with another port in both environments.
Messages are getting consumed and processed, but I have no way of determining if the acks themselves are sent/received. How should I configure the logging to determine this?
-----Original Message-----
From: Justin Bertram <jb...@apache.org>
Sent: Monday, February 6, 2023 10:11 AM
To: users@activemq.apache.org
Subject: Re: ActiveMQ in the cloud
I wouldn't expect any difference with regard to how messages are acknowledged between cloud and non-cloud environments. Are the master and consumer processes configured the same way in both environments? Are they messages being consumed and not acknowledged or not consumed at all?
Justin
On Mon, Feb 6, 2023 at 10:02 AM John Palmisano < John.Palmisano@tidalsoftware.com> wrote:
> Let me clarify the issue we are running into:
>
> We have a master process which contains an embedded broker. This
> process communicates with one or more consumer processes via ActiveMQ.
> Messages are queued on the master side (per consumer) and processed by
> each of the consumer processes. For some reason, within a cloud
> environment, the queued messages are building up in the master process
> (i.e. never cleaned up), up to the point where the master process runs
> out of memory. This scenario does not happen within a non-cloud environment.
>
> Any insights on why this would be happening?
>
> -----Original Message-----
> From: Justin Bertram <jb...@apache.org>
> Sent: Monday, February 6, 2023 9:11 AM
> To: users@activemq.apache.org
> Subject: Re: ActiveMQ in the cloud
>
> The exact port depends on the protocol you're using, but all messaging
> protocols supported by the broker use the same connection for
> everything (i.e sending messages, receiving messages, acknowledging
> messages, etc.) so the port won't change.
>
> Here's the default port used by each supported protocol:
> - OpenWire (JMS) - 61616
> - AMQP - 5672
> - STOMP - 61613
> - MQTT - 1883
>
>
> Justin
>
> On Mon, Feb 6, 2023 at 9:01 AM John Palmisano <
> John.Palmisano@tidalsoftware.com> wrote:
>
> > On what port(s) are acknowledgements sent back?
> >
> > -----Original Message-----
> > From: Justin Bertram <jb...@apache.org>
> > Sent: Monday, February 6, 2023 8:48 AM
> > To: users@activemq.apache.org
> > Subject: Re: ActiveMQ in the cloud
> >
> > Your question is pretty vague, but the main thing to remember is
> > that the cloud is really no different from any other environment.
> > The host and port must be reachable across the network. As long as
> > that is true then clients can connect, consume, and acknowledge messages.
> >
> >
> > Justin
> >
> > On Mon, Feb 6, 2023 at 8:26 AM John Palmisano <
> > John.Palmisano@tidalsoftware.com> wrote:
> >
> > > Hello,
> > >
> > > How do I properly configure ActiveMQ, with respect to ports, etc.,
> > > so that queues will work properly within a cloud environment?
> > >
> > > Specifically, I want to ensure that the acknowledgements sent from
> > > consumers are being sent/received successfully so that queued
> > > messages do not build up.
> > >
> >
>
Re: ActiveMQ in the cloud
Posted by Justin Bertram <jb...@apache.org>.
I wouldn't expect any difference with regard to how messages are
acknowledged between cloud and non-cloud environments. Are the master and
consumer processes configured the same way in both environments? Are they
messages being consumed and not acknowledged or not consumed at all?
Justin
On Mon, Feb 6, 2023 at 10:02 AM John Palmisano <
John.Palmisano@tidalsoftware.com> wrote:
> Let me clarify the issue we are running into:
>
> We have a master process which contains an embedded broker. This process
> communicates with one or more consumer processes via ActiveMQ. Messages are
> queued on the master side (per consumer) and processed by each of the
> consumer processes. For some reason, within a cloud environment, the queued
> messages are building up in the master process (i.e. never cleaned up), up
> to the point where the master process runs out of memory. This scenario
> does not happen within a non-cloud environment.
>
> Any insights on why this would be happening?
>
> -----Original Message-----
> From: Justin Bertram <jb...@apache.org>
> Sent: Monday, February 6, 2023 9:11 AM
> To: users@activemq.apache.org
> Subject: Re: ActiveMQ in the cloud
>
> The exact port depends on the protocol you're using, but all messaging
> protocols supported by the broker use the same connection for everything
> (i.e sending messages, receiving messages, acknowledging messages, etc.) so
> the port won't change.
>
> Here's the default port used by each supported protocol:
> - OpenWire (JMS) - 61616
> - AMQP - 5672
> - STOMP - 61613
> - MQTT - 1883
>
>
> Justin
>
> On Mon, Feb 6, 2023 at 9:01 AM John Palmisano <
> John.Palmisano@tidalsoftware.com> wrote:
>
> > On what port(s) are acknowledgements sent back?
> >
> > -----Original Message-----
> > From: Justin Bertram <jb...@apache.org>
> > Sent: Monday, February 6, 2023 8:48 AM
> > To: users@activemq.apache.org
> > Subject: Re: ActiveMQ in the cloud
> >
> > Your question is pretty vague, but the main thing to remember is that
> > the cloud is really no different from any other environment. The host
> > and port must be reachable across the network. As long as that is true
> > then clients can connect, consume, and acknowledge messages.
> >
> >
> > Justin
> >
> > On Mon, Feb 6, 2023 at 8:26 AM John Palmisano <
> > John.Palmisano@tidalsoftware.com> wrote:
> >
> > > Hello,
> > >
> > > How do I properly configure ActiveMQ, with respect to ports, etc.,
> > > so that queues will work properly within a cloud environment?
> > >
> > > Specifically, I want to ensure that the acknowledgements sent from
> > > consumers are being sent/received successfully so that queued
> > > messages do not build up.
> > >
> >
>
RE: ActiveMQ in the cloud
Posted by John Palmisano <Jo...@tidalsoftware.com>.
Let me clarify the issue we are running into:
We have a master process which contains an embedded broker. This process communicates with one or more consumer processes via ActiveMQ. Messages are queued on the master side (per consumer) and processed by each of the consumer processes. For some reason, within a cloud environment, the queued messages are building up in the master process (i.e. never cleaned up), up to the point where the master process runs out of memory. This scenario does not happen within a non-cloud environment.
Any insights on why this would be happening?
-----Original Message-----
From: Justin Bertram <jb...@apache.org>
Sent: Monday, February 6, 2023 9:11 AM
To: users@activemq.apache.org
Subject: Re: ActiveMQ in the cloud
The exact port depends on the protocol you're using, but all messaging protocols supported by the broker use the same connection for everything (i.e sending messages, receiving messages, acknowledging messages, etc.) so the port won't change.
Here's the default port used by each supported protocol:
- OpenWire (JMS) - 61616
- AMQP - 5672
- STOMP - 61613
- MQTT - 1883
Justin
On Mon, Feb 6, 2023 at 9:01 AM John Palmisano < John.Palmisano@tidalsoftware.com> wrote:
> On what port(s) are acknowledgements sent back?
>
> -----Original Message-----
> From: Justin Bertram <jb...@apache.org>
> Sent: Monday, February 6, 2023 8:48 AM
> To: users@activemq.apache.org
> Subject: Re: ActiveMQ in the cloud
>
> Your question is pretty vague, but the main thing to remember is that
> the cloud is really no different from any other environment. The host
> and port must be reachable across the network. As long as that is true
> then clients can connect, consume, and acknowledge messages.
>
>
> Justin
>
> On Mon, Feb 6, 2023 at 8:26 AM John Palmisano <
> John.Palmisano@tidalsoftware.com> wrote:
>
> > Hello,
> >
> > How do I properly configure ActiveMQ, with respect to ports, etc.,
> > so that queues will work properly within a cloud environment?
> >
> > Specifically, I want to ensure that the acknowledgements sent from
> > consumers are being sent/received successfully so that queued
> > messages do not build up.
> >
>
Re: ActiveMQ in the cloud
Posted by Justin Bertram <jb...@apache.org>.
The exact port depends on the protocol you're using, but all messaging
protocols supported by the broker use the same connection for everything
(i.e sending messages, receiving messages, acknowledging messages, etc.) so
the port won't change.
Here's the default port used by each supported protocol:
- OpenWire (JMS) - 61616
- AMQP - 5672
- STOMP - 61613
- MQTT - 1883
Justin
On Mon, Feb 6, 2023 at 9:01 AM John Palmisano <
John.Palmisano@tidalsoftware.com> wrote:
> On what port(s) are acknowledgements sent back?
>
> -----Original Message-----
> From: Justin Bertram <jb...@apache.org>
> Sent: Monday, February 6, 2023 8:48 AM
> To: users@activemq.apache.org
> Subject: Re: ActiveMQ in the cloud
>
> Your question is pretty vague, but the main thing to remember is that the
> cloud is really no different from any other environment. The host and port
> must be reachable across the network. As long as that is true then clients
> can connect, consume, and acknowledge messages.
>
>
> Justin
>
> On Mon, Feb 6, 2023 at 8:26 AM John Palmisano <
> John.Palmisano@tidalsoftware.com> wrote:
>
> > Hello,
> >
> > How do I properly configure ActiveMQ, with respect to ports, etc., so
> > that queues will work properly within a cloud environment?
> >
> > Specifically, I want to ensure that the acknowledgements sent from
> > consumers are being sent/received successfully so that queued messages
> > do not build up.
> >
>
RE: ActiveMQ in the cloud
Posted by John Palmisano <Jo...@tidalsoftware.com>.
On what port(s) are acknowledgements sent back?
-----Original Message-----
From: Justin Bertram <jb...@apache.org>
Sent: Monday, February 6, 2023 8:48 AM
To: users@activemq.apache.org
Subject: Re: ActiveMQ in the cloud
Your question is pretty vague, but the main thing to remember is that the cloud is really no different from any other environment. The host and port must be reachable across the network. As long as that is true then clients can connect, consume, and acknowledge messages.
Justin
On Mon, Feb 6, 2023 at 8:26 AM John Palmisano < John.Palmisano@tidalsoftware.com> wrote:
> Hello,
>
> How do I properly configure ActiveMQ, with respect to ports, etc., so
> that queues will work properly within a cloud environment?
>
> Specifically, I want to ensure that the acknowledgements sent from
> consumers are being sent/received successfully so that queued messages
> do not build up.
>
Re: ActiveMQ in the cloud
Posted by Justin Bertram <jb...@apache.org>.
Your question is pretty vague, but the main thing to remember is that the
cloud is really no different from any other environment. The host and port
must be reachable across the network. As long as that is true then clients
can connect, consume, and acknowledge messages.
Justin
On Mon, Feb 6, 2023 at 8:26 AM John Palmisano <
John.Palmisano@tidalsoftware.com> wrote:
> Hello,
>
> How do I properly configure ActiveMQ, with respect to ports, etc., so that
> queues will work properly within a cloud environment?
>
> Specifically, I want to ensure that the acknowledgements sent from
> consumers are being sent/received successfully so that queued messages do
> not build up.
>