You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by Kevin Burton <bu...@spinn3r.com> on 2015/02/01 18:38:59 UTC
ActiveMQ should GC inactive when producers is 0 , even if consumers
is > 0
I think this is another RFE but I might be wrong.. it would be nice to see
this fixed in AMQ.
I have code that uses LOTS of ephemeral queues. It’s probably less than 1k
total active queues at any one time, but if I don’t allow them to be GCd
due to inactivity, it would EASILY grow and grow over months, overwhelming
memory.
I’m trying to implement code that closes the consumers on the backend , but
there are races.
If I close a consumer, a message could come in JUST after I close it, which
would mean it NEVER gets processed.
This would yield to data loss.
However, if ActiveMQ GCd the queue when producers = 0 , and messages = 0,
*regardless* of the number of consumers, then I thinks we’re golden.
I can listen to advisory queue message telling me that the queue was
closed, and then release my resources/consumers on the client.
But in the current situation I have to write lots of ugly code to close the
consumer , and some JMX code to verify that the count stays zero …
… of course one strategy that I just thought of is that I could wait until
I get an advisory queue message that the queue is GCd , but then I have
latency where 1-2 message could be idle for a few minutes before they’re
eventually serviced.
--
Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
<https://plus.google.com/102718274791889610666/posts>
<http://spinn3r.com>
Re: ActiveMQ should GC inactive when producers is 0 , even if
consumers is > 0
Posted by Kevin Burton <bu...@spinn3r.com>.
I created a JIRA for this.
https://issues.apache.org/jira/browse/AMQ-5557
I’m convinced this is the wrong behavior because there are legitimate race
conditions present here that can’t be solved without allowing ActiveMQ to
make the decision on GC.
On Sun, Feb 1, 2015 at 9:38 AM, Kevin Burton <bu...@spinn3r.com> wrote:
> I think this is another RFE but I might be wrong.. it would be nice to see
> this fixed in AMQ.
>
> I have code that uses LOTS of ephemeral queues. It’s probably less than 1k
> total active queues at any one time, but if I don’t allow them to be GCd
> due to inactivity, it would EASILY grow and grow over months, overwhelming
> memory.
>
> I’m trying to implement code that closes the consumers on the backend ,
> but there are races.
>
> If I close a consumer, a message could come in JUST after I close it,
> which would mean it NEVER gets processed.
>
> This would yield to data loss.
>
> However, if ActiveMQ GCd the queue when producers = 0 , and messages = 0,
> *regardless* of the number of consumers, then I thinks we’re golden.
>
> I can listen to advisory queue message telling me that the queue was
> closed, and then release my resources/consumers on the client.
>
> But in the current situation I have to write lots of ugly code to close
> the consumer , and some JMX code to verify that the count stays zero …
>
> … of course one strategy that I just thought of is that I could wait until
> I get an advisory queue message that the queue is GCd , but then I have
> latency where 1-2 message could be idle for a few minutes before they’re
> eventually serviced.
>
> --
>
> Founder/CEO Spinn3r.com
> Location: *San Francisco, CA*
> blog: http://burtonator.wordpress.com
> … or check out my Google+ profile
> <https://plus.google.com/102718274791889610666/posts>
> <http://spinn3r.com>
>
>
--
Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
<https://plus.google.com/102718274791889610666/posts>
<http://spinn3r.com>
Re: ActiveMQ should GC inactive when producers is 0 , even if
consumers is > 0
Posted by Kevin Burton <bu...@spinn3r.com>.
Agreed … I’m usually pretty good at these though :)
are there any obvious ActiveMQ ones you’re thinking of or is this just a
general warning?
On Mon, Feb 2, 2015 at 1:57 PM, artnaseef <ar...@artnaseef.com> wrote:
> Check carefully. I suspect you'll find race conditions still exist there.
>
>
>
> --
> View this message in context:
> http://activemq.2283324.n4.nabble.com/ActiveMQ-should-GC-inactive-when-producers-is-0-even-if-consumers-is-0-tp4690776p4690885.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
--
Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
<https://plus.google.com/102718274791889610666/posts>
<http://spinn3r.com>
Re: ActiveMQ should GC inactive when producers is 0 , even if
consumers is > 0
Posted by artnaseef <ar...@artnaseef.com>.
Check carefully. I suspect you'll find race conditions still exist there.
--
View this message in context: http://activemq.2283324.n4.nabble.com/ActiveMQ-should-GC-inactive-when-producers-is-0-even-if-consumers-is-0-tp4690776p4690885.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: ActiveMQ should GC inactive when producers is 0 , even if
consumers is > 0
Posted by Kevin Burton <bu...@spinn3r.com>.
FYI.. I closed this bug as WONTFIX as I think this is the best strategy
post feedback.
Thanks guys.
On Mon, Feb 2, 2015 at 11:52 AM, Kevin Burton <bu...@spinn3r.com> wrote:
> the problem with this strategy is that queues can come and go.. it’s a
> scheduler system for a web crawler. so if a URL request comes in for an IP
> we place it in that queue. And if, a month from now, we have to request
> another resources from that IP, we place it in a queue for that IP.
>
> MOST of the time, we just have a few IPs because most web properties are
> large. However, sometimes, we have a request once per hour to an IP. In
> that situation we want to GC the queue 60 seconds later.
>
> Kevin
>
> On Mon, Feb 2, 2015 at 11:17 AM, Tim Bain <tb...@alumni.duke.edu> wrote:
>
>> It sounds like the advisory message you mentioned should let you work
>> around the problem.
>>
>> But what it sounds like you really need is a way to know, for sure, that
>> the producer is finished using a queue. If your producers write an EOF
>> message when they're through, your consumer can then safely go away
>> without
>> having to worry about stragglers. Then your only remaining problem is how
>> to know about producers that die unexpectedly without writing the EOF or
>> that take a really long time to producer their next message. A heartbeat
>> mechanism would solve those issues: every X amount of time, the producer
>> produces a small no-op heartbeat message that tells the consumer it's
>> still
>> there; failure to receive one of those messages after N heartbeat
>> intervals
>> indicates that the producer is no longer alive. The only thing it doesn't
>> protect you against is temporary network perturbations (since they would
>> make it appear that the producer is gone when it's not really), but you
>> might be able to live with that.
>>
>> I'm not sure that heartbeats are a better solution than processing the
>> advisory messages as you've suggested, but I think at a minimum you should
>> have an EOF that's always sent (during normal operations) when the
>> producer
>> closes, and to which your consumer reacts by disconnecting as well. Then
>> the advisory messages can be used only for handling truly exceptional
>> conditions.
>>
>> On Mon, Feb 2, 2015 at 11:29 AM, Kevin Burton <bu...@spinn3r.com> wrote:
>>
>> > > ActiveMQ provides no means to guarantee that a producer will fail to
>> > > deliver
>> > > a message when there are no consumers interested in the message. It
>> is
>> > > actually designed more for cases of consumers being away and returning
>> > at a
>> > > later time.
>> > >
>> > >
>> > Perhaps. But I think the advisory queue message
>> >
>> > ActiveMQ.Advisory.NoConsumer.Queue
>> >
>> > … actually solves my problem.
>> >
>> > If a producer creates a queue, and no one is consuming, I’ll just
>> create a
>> > new consumer once I get the advisory message about the queue.
>> >
>> > This is actually how I do it *now* except I was listening to NEW queues…
>> >
>> > I’ve actually gotten a lot of use out of the advisory queue system.
>> >
>> >
>> > > As you noted correctly, ActiveMQ automatically recreates destinations
>> > every
>> > > time they are needed, so even if they were GC'ed, they would just be
>> > > automatically recreated when a producer posted a message.
>> > >
>> >
>> > Yes. And in my case, I think that’s ok.
>> >
>> > I might actually write up a blog post about this design. It’s novel but
>> AMQ
>> > actually does seem to be working out well for us without the limitations
>> > you describe due to advisory messages.
>> >
>> > Kevin
>> >
>> > --
>> >
>> > Founder/CEO Spinn3r.com
>> > Location: *San Francisco, CA*
>> > blog: http://burtonator.wordpress.com
>> > … or check out my Google+ profile
>> > <https://plus.google.com/102718274791889610666/posts>
>> > <http://spinn3r.com>
>> >
>>
>
>
>
> --
>
> Founder/CEO Spinn3r.com
> Location: *San Francisco, CA*
> blog: http://burtonator.wordpress.com
> … or check out my Google+ profile
> <https://plus.google.com/102718274791889610666/posts>
> <http://spinn3r.com>
>
>
--
Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
<https://plus.google.com/102718274791889610666/posts>
<http://spinn3r.com>
Re: ActiveMQ should GC inactive when producers is 0 , even if
consumers is > 0
Posted by Kevin Burton <bu...@spinn3r.com>.
the problem with this strategy is that queues can come and go.. it’s a
scheduler system for a web crawler. so if a URL request comes in for an IP
we place it in that queue. And if, a month from now, we have to request
another resources from that IP, we place it in a queue for that IP.
MOST of the time, we just have a few IPs because most web properties are
large. However, sometimes, we have a request once per hour to an IP. In
that situation we want to GC the queue 60 seconds later.
Kevin
On Mon, Feb 2, 2015 at 11:17 AM, Tim Bain <tb...@alumni.duke.edu> wrote:
> It sounds like the advisory message you mentioned should let you work
> around the problem.
>
> But what it sounds like you really need is a way to know, for sure, that
> the producer is finished using a queue. If your producers write an EOF
> message when they're through, your consumer can then safely go away without
> having to worry about stragglers. Then your only remaining problem is how
> to know about producers that die unexpectedly without writing the EOF or
> that take a really long time to producer their next message. A heartbeat
> mechanism would solve those issues: every X amount of time, the producer
> produces a small no-op heartbeat message that tells the consumer it's still
> there; failure to receive one of those messages after N heartbeat intervals
> indicates that the producer is no longer alive. The only thing it doesn't
> protect you against is temporary network perturbations (since they would
> make it appear that the producer is gone when it's not really), but you
> might be able to live with that.
>
> I'm not sure that heartbeats are a better solution than processing the
> advisory messages as you've suggested, but I think at a minimum you should
> have an EOF that's always sent (during normal operations) when the producer
> closes, and to which your consumer reacts by disconnecting as well. Then
> the advisory messages can be used only for handling truly exceptional
> conditions.
>
> On Mon, Feb 2, 2015 at 11:29 AM, Kevin Burton <bu...@spinn3r.com> wrote:
>
> > > ActiveMQ provides no means to guarantee that a producer will fail to
> > > deliver
> > > a message when there are no consumers interested in the message. It is
> > > actually designed more for cases of consumers being away and returning
> > at a
> > > later time.
> > >
> > >
> > Perhaps. But I think the advisory queue message
> >
> > ActiveMQ.Advisory.NoConsumer.Queue
> >
> > … actually solves my problem.
> >
> > If a producer creates a queue, and no one is consuming, I’ll just create
> a
> > new consumer once I get the advisory message about the queue.
> >
> > This is actually how I do it *now* except I was listening to NEW queues…
> >
> > I’ve actually gotten a lot of use out of the advisory queue system.
> >
> >
> > > As you noted correctly, ActiveMQ automatically recreates destinations
> > every
> > > time they are needed, so even if they were GC'ed, they would just be
> > > automatically recreated when a producer posted a message.
> > >
> >
> > Yes. And in my case, I think that’s ok.
> >
> > I might actually write up a blog post about this design. It’s novel but
> AMQ
> > actually does seem to be working out well for us without the limitations
> > you describe due to advisory messages.
> >
> > Kevin
> >
> > --
> >
> > Founder/CEO Spinn3r.com
> > Location: *San Francisco, CA*
> > blog: http://burtonator.wordpress.com
> > … or check out my Google+ profile
> > <https://plus.google.com/102718274791889610666/posts>
> > <http://spinn3r.com>
> >
>
--
Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
<https://plus.google.com/102718274791889610666/posts>
<http://spinn3r.com>
Re: ActiveMQ should GC inactive when producers is 0 , even if
consumers is > 0
Posted by Tim Bain <tb...@alumni.duke.edu>.
It sounds like the advisory message you mentioned should let you work
around the problem.
But what it sounds like you really need is a way to know, for sure, that
the producer is finished using a queue. If your producers write an EOF
message when they're through, your consumer can then safely go away without
having to worry about stragglers. Then your only remaining problem is how
to know about producers that die unexpectedly without writing the EOF or
that take a really long time to producer their next message. A heartbeat
mechanism would solve those issues: every X amount of time, the producer
produces a small no-op heartbeat message that tells the consumer it's still
there; failure to receive one of those messages after N heartbeat intervals
indicates that the producer is no longer alive. The only thing it doesn't
protect you against is temporary network perturbations (since they would
make it appear that the producer is gone when it's not really), but you
might be able to live with that.
I'm not sure that heartbeats are a better solution than processing the
advisory messages as you've suggested, but I think at a minimum you should
have an EOF that's always sent (during normal operations) when the producer
closes, and to which your consumer reacts by disconnecting as well. Then
the advisory messages can be used only for handling truly exceptional
conditions.
On Mon, Feb 2, 2015 at 11:29 AM, Kevin Burton <bu...@spinn3r.com> wrote:
> > ActiveMQ provides no means to guarantee that a producer will fail to
> > deliver
> > a message when there are no consumers interested in the message. It is
> > actually designed more for cases of consumers being away and returning
> at a
> > later time.
> >
> >
> Perhaps. But I think the advisory queue message
>
> ActiveMQ.Advisory.NoConsumer.Queue
>
> … actually solves my problem.
>
> If a producer creates a queue, and no one is consuming, I’ll just create a
> new consumer once I get the advisory message about the queue.
>
> This is actually how I do it *now* except I was listening to NEW queues…
>
> I’ve actually gotten a lot of use out of the advisory queue system.
>
>
> > As you noted correctly, ActiveMQ automatically recreates destinations
> every
> > time they are needed, so even if they were GC'ed, they would just be
> > automatically recreated when a producer posted a message.
> >
>
> Yes. And in my case, I think that’s ok.
>
> I might actually write up a blog post about this design. It’s novel but AMQ
> actually does seem to be working out well for us without the limitations
> you describe due to advisory messages.
>
> Kevin
>
> --
>
> Founder/CEO Spinn3r.com
> Location: *San Francisco, CA*
> blog: http://burtonator.wordpress.com
> … or check out my Google+ profile
> <https://plus.google.com/102718274791889610666/posts>
> <http://spinn3r.com>
>
Re: ActiveMQ should GC inactive when producers is 0 , even if
consumers is > 0
Posted by Kevin Burton <bu...@spinn3r.com>.
> ActiveMQ provides no means to guarantee that a producer will fail to
> deliver
> a message when there are no consumers interested in the message. It is
> actually designed more for cases of consumers being away and returning at a
> later time.
>
>
Perhaps. But I think the advisory queue message
ActiveMQ.Advisory.NoConsumer.Queue
… actually solves my problem.
If a producer creates a queue, and no one is consuming, I’ll just create a
new consumer once I get the advisory message about the queue.
This is actually how I do it *now* except I was listening to NEW queues…
I’ve actually gotten a lot of use out of the advisory queue system.
> As you noted correctly, ActiveMQ automatically recreates destinations every
> time they are needed, so even if they were GC'ed, they would just be
> automatically recreated when a producer posted a message.
>
Yes. And in my case, I think that’s ok.
I might actually write up a blog post about this design. It’s novel but AMQ
actually does seem to be working out well for us without the limitations
you describe due to advisory messages.
Kevin
--
Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
<https://plus.google.com/102718274791889610666/posts>
<http://spinn3r.com>
Re: ActiveMQ should GC inactive when producers is 0 , even if
consumers is > 0
Posted by artnaseef <ar...@artnaseef.com>.
So, reading carefully, it sounds the solution has orders-of-operation
concerns with messages that cannot be lost, consumers that go away,
producers that must not produce when the consumers are missing, the
application is using an asynchronous messaging solution (ActiveMQ), and AMQ
is being used to coordinate between the producers and consumers.
If so, that's problematic.
ActiveMQ provides no means to guarantee that a producer will fail to deliver
a message when there are no consumers interested in the message. It is
actually designed more for cases of consumers being away and returning at a
later time.
As you noted correctly, ActiveMQ automatically recreates destinations every
time they are needed, so even if they were GC'ed, they would just be
automatically recreated when a producer posted a message.
Hope this helps.
--
View this message in context: http://activemq.2283324.n4.nabble.com/ActiveMQ-should-GC-inactive-when-producers-is-0-even-if-consumers-is-0-tp4690776p4690850.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: ActiveMQ should GC inactive when producers is 0 , even if
consumers is > 0
Posted by Kevin Burton <bu...@spinn3r.com>.
On Sun, Feb 1, 2015 at 8:22 PM, artnaseef <ar...@artnaseef.com> wrote:
> Something has me confused here. If there is a consumer, why would messages
> sit around unconsumed?
>
>
If you want a queue to GC, you have to close any consumers on it.
So lets say the queue has been idle , with no messages, for 60 seconds.
at 61 seconds you close it.
at 62 seconds a new message comes in from a straggler.
Now that message sits around forever because there’s no way to no its there
and no way to have the queue GC itself.
> Also, I have trouble understanding why one would want the broker to remove
> a
> destination while it has a consumer --
Because of the above race … there’s no way to GC a queue and risk a message
coming in due to this race.
By allowing broker to GC the queue with a consumer attached you avoid this
problem.
> what is the consumer doing with the
> destination then, and what is expected to happen to the consumer?
>
The consumer should just allow itself to stay connected.
One problem is that I had THOUGHT that you could create a consumer to a
nonexistent queue, but I was wrong. The queue is automatically created. I
had thought you had to write a message to it first but I was wrong.
>
> Also, if messages are sitting in a queue, the queue cannot be GC'ed -
> otherwise the broker is losing messages. A good solution there may be the
> use of message TTLs.
>
Well this is a problem too because then messages get lost.
> BTW, it sounds like the application may have a consumer leak. Consumers
> can
> be thought of as the center of purpose for a destination.
So my issue is the situation where you have a wide number of queues. My
application is generally working FINE but I have too problems:
1. AMQ memory grows VERY high with large numbers of queues.
2. BULK Queue GC when I restart my daemons causes AMQ to come to a
complete halt for 30 minutes or more.
3. I can’t release these queues to become GCd because to the race above.
So right now if AMQ could close inactive queues I would be set.
ANOTHER idea is some sort of Advisory queue message like “this queue is now
active again” … so if a straggler message comes in you can open another
consumer and consume the queue again.
--
Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
<https://plus.google.com/102718274791889610666/posts>
<http://spinn3r.com>
Re: ActiveMQ should GC inactive when producers is 0 , even if
consumers is > 0
Posted by Kevin Burton <bu...@spinn3r.com>.
On Mon, Feb 2, 2015 at 5:39 AM, Tim Bain <tb...@alumni.duke.edu> wrote:
> Also, the most common (I think) broker configuration will auto-create a
> destination if a client tries to use it and it doesn't exist. So if you
> delete the destination while a consumer is still connected, won't the
> broker just immediately recreate it? If implemented, would this
> enhancement actually do what you want?
>
>
True. I mean part of this depends on the complexity. If it’s impossible
to implement this feature or take months then clearly it’s a bad idea.
> It seems like you need some means for the consumer to figure out that the
> producer is done and disconnect itself, which would then let the existing
> destination GC logic work without this enhancement.
Correct. And to fix the face where a straggler could come in a moment
after the consumer disconnects and writes a message. This is the main
problem I need a work around for.
Kevin
--
Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
<https://plus.google.com/102718274791889610666/posts>
<http://spinn3r.com>
Re: ActiveMQ should GC inactive when producers is 0 , even if
consumers is > 0
Posted by Kevin Burton <bu...@spinn3r.com>.
After reading this thread I think I might have a workaround for this
problem.
The main problem I have is stragglers.
If I close a queue, and a moment later one messages is published, I’ll
never be able to consume it.
BUT… I could use this Advisory message:
> ActiveMQ.Advisory.NoConsumer.Queue
> No consumer is available to process messages being sent on a Queue
… so basically I would do something along the lines of:
- register listeners for ActiveMQ.Advisory.NoConsumer.Queue
- wait 30 seconds until a queue is inactive.
- mark that queue in my app as inactive
- close the queue()
- if any messages come in for ActiveMQ.Advisory.NoConsumer.Queue on that
queue I immediately re-open a consumer.
This should solve the straggler problem as I’ll immediately see if after it
publishes. Further, normal GC would just work just the way it does now.
We definitely use AMQ in a somewhat non-standard way. We have lots of
small ephemeral queues which get flurries of messages, then cool down and
go away.
ActiveMQ has been GREAT so far and works well and even in this non-standard
use case works great.
This has been the only main issue we’re dealing with so hopefully I can
resolve it easily.
On Mon, Feb 2, 2015 at 6:10 AM, Tim Bain <tb...@alumni.duke.edu> wrote:
> Kevin probably doesn't want a separate control channel given the volumes
> he's been talking about; that would be twice as many destinations to GC.
> Just send the "work is done" message via the real queue.
>
> But Tim's absolutely right about setting a message expiration time; either
> make 100% sure every message will be consumed (which you give up on if
> you're allowing the broker to infer completeness based on inactivity) or
> make sure that when they get missed they go quickly into the bit bucket.
> To that end, you probably want to use the DiscardingDLQPlugin to keep them
> from sitting in the DLQ.
>
> Tim
> On Feb 2, 2015 6:56 AM, "Timothy Bish" <ta...@gmail.com> wrote:
>
> > On 02/02/2015 08:39 AM, Tim Bain wrote:
> >
> >> Also, the most common (I think) broker configuration will auto-create a
> >> destination if a client tries to use it and it doesn't exist. So if you
> >> delete the destination while a consumer is still connected, won't the
> >> broker just immediately recreate it? If implemented, would this
> >> enhancement actually do what you want?
> >>
> >> It seems like you need some means for the consumer to figure out that
> the
> >> producer is done and disconnect itself, which would then let the
> existing
> >> destination GC logic work without this enhancement.
> >> On Feb 1, 2015 9:29 PM, "artnaseef" <ar...@artnaseef.com> wrote:
> >>
> > In this case I'd guess it would be implemented such that the consumer
> > would get closed via a ConsumerControl command or more harshly just
> killing
> > the consumer connection but if there were other consumers on that
> > connection that could cause all sorts of problems in the users app. If
> the
> > connection was dropped and the consumer was using failover then it would
> > attempt to reconnect and would end up recreating the destination.
> >
> > Doing this wouldn't really solve the scenario that the user is attempting
> > to solve, that is that a message could arrive after the consumer shuts
> down
> > because as you've guessed the auto create feature would recreate the
> > destination for a producer that suddenly appeared and published this
> > presumed last second message. So adding the option to remove the Queue
> > even if there are consumers doesn't really solve anything and more likely
> > just creates a new set of issues to contend with.
> >
> > It'd be better to use some expiry on the messages and probably some
> > control channel to tell the consumer that it's work is done and that it
> is
> > safe for it to close.
> >
> > Something has me confused here. If there is a consumer, why would
> >>> messages
> >>> sit around unconsumed?
> >>>
> >>> Also, I have trouble understanding why one would want the broker to
> >>> remove
> >>> a
> >>> destination while it has a consumer -- what is the consumer doing with
> >>> the
> >>> destination then, and what is expected to happen to the consumer?
> >>>
> >>> Also, if messages are sitting in a queue, the queue cannot be GC'ed -
> >>> otherwise the broker is losing messages. A good solution there may be
> >>> the
> >>> use of message TTLs.
> >>>
> >>> Perhaps a review of the architecture would help to determine the best
> way
> >>> to
> >>> fit ActiveMQ and the application to the problem.
> >>>
> >>> BTW, it sounds like the application may have a consumer leak.
> Consumers
> >>> can
> >>> be thought of as the center of purpose for a destination. A
> destination
> >>> that never has consumers would be a bit-bucket, or worse - a
> >>> continually-growing message store. Consumers that don't expect to
> >>> receive
> >>> messages cannot be distinguished by ActiveMQ from those that do expect
> to
> >>> receive messages.
> >>>
> >>> Hope this helps.
> >>>
> >>>
> >>>
> >>> --
> >>> View this message in context:
> >>> http://activemq.2283324.n4.nabble.com/ActiveMQ-should-GC-
> >>> inactive-when-producers-is-0-even-if-consumers-is-0-
> >>> tp4690776p4690797.html
> >>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
> >>>
> >>>
> >
> > --
> > Tim Bish
> > Sr Software Engineer | RedHat Inc.
> > tim.bish@redhat.com | www.redhat.com
> > skype: tabish121 | twitter: @tabish121
> > blog: http://timbish.blogspot.com/
> >
> >
>
--
Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
<https://plus.google.com/102718274791889610666/posts>
<http://spinn3r.com>
Re: ActiveMQ should GC inactive when producers is 0 , even if
consumers is > 0
Posted by Tim Bain <tb...@alumni.duke.edu>.
Kevin probably doesn't want a separate control channel given the volumes
he's been talking about; that would be twice as many destinations to GC.
Just send the "work is done" message via the real queue.
But Tim's absolutely right about setting a message expiration time; either
make 100% sure every message will be consumed (which you give up on if
you're allowing the broker to infer completeness based on inactivity) or
make sure that when they get missed they go quickly into the bit bucket.
To that end, you probably want to use the DiscardingDLQPlugin to keep them
from sitting in the DLQ.
Tim
On Feb 2, 2015 6:56 AM, "Timothy Bish" <ta...@gmail.com> wrote:
> On 02/02/2015 08:39 AM, Tim Bain wrote:
>
>> Also, the most common (I think) broker configuration will auto-create a
>> destination if a client tries to use it and it doesn't exist. So if you
>> delete the destination while a consumer is still connected, won't the
>> broker just immediately recreate it? If implemented, would this
>> enhancement actually do what you want?
>>
>> It seems like you need some means for the consumer to figure out that the
>> producer is done and disconnect itself, which would then let the existing
>> destination GC logic work without this enhancement.
>> On Feb 1, 2015 9:29 PM, "artnaseef" <ar...@artnaseef.com> wrote:
>>
> In this case I'd guess it would be implemented such that the consumer
> would get closed via a ConsumerControl command or more harshly just killing
> the consumer connection but if there were other consumers on that
> connection that could cause all sorts of problems in the users app. If the
> connection was dropped and the consumer was using failover then it would
> attempt to reconnect and would end up recreating the destination.
>
> Doing this wouldn't really solve the scenario that the user is attempting
> to solve, that is that a message could arrive after the consumer shuts down
> because as you've guessed the auto create feature would recreate the
> destination for a producer that suddenly appeared and published this
> presumed last second message. So adding the option to remove the Queue
> even if there are consumers doesn't really solve anything and more likely
> just creates a new set of issues to contend with.
>
> It'd be better to use some expiry on the messages and probably some
> control channel to tell the consumer that it's work is done and that it is
> safe for it to close.
>
> Something has me confused here. If there is a consumer, why would
>>> messages
>>> sit around unconsumed?
>>>
>>> Also, I have trouble understanding why one would want the broker to
>>> remove
>>> a
>>> destination while it has a consumer -- what is the consumer doing with
>>> the
>>> destination then, and what is expected to happen to the consumer?
>>>
>>> Also, if messages are sitting in a queue, the queue cannot be GC'ed -
>>> otherwise the broker is losing messages. A good solution there may be
>>> the
>>> use of message TTLs.
>>>
>>> Perhaps a review of the architecture would help to determine the best way
>>> to
>>> fit ActiveMQ and the application to the problem.
>>>
>>> BTW, it sounds like the application may have a consumer leak. Consumers
>>> can
>>> be thought of as the center of purpose for a destination. A destination
>>> that never has consumers would be a bit-bucket, or worse - a
>>> continually-growing message store. Consumers that don't expect to
>>> receive
>>> messages cannot be distinguished by ActiveMQ from those that do expect to
>>> receive messages.
>>>
>>> Hope this helps.
>>>
>>>
>>>
>>> --
>>> View this message in context:
>>> http://activemq.2283324.n4.nabble.com/ActiveMQ-should-GC-
>>> inactive-when-producers-is-0-even-if-consumers-is-0-
>>> tp4690776p4690797.html
>>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>>
>>>
>
> --
> Tim Bish
> Sr Software Engineer | RedHat Inc.
> tim.bish@redhat.com | www.redhat.com
> skype: tabish121 | twitter: @tabish121
> blog: http://timbish.blogspot.com/
>
>
Re: ActiveMQ should GC inactive when producers is 0 , even if consumers
is > 0
Posted by Timothy Bish <ta...@gmail.com>.
On 02/02/2015 08:39 AM, Tim Bain wrote:
> Also, the most common (I think) broker configuration will auto-create a
> destination if a client tries to use it and it doesn't exist. So if you
> delete the destination while a consumer is still connected, won't the
> broker just immediately recreate it? If implemented, would this
> enhancement actually do what you want?
>
> It seems like you need some means for the consumer to figure out that the
> producer is done and disconnect itself, which would then let the existing
> destination GC logic work without this enhancement.
> On Feb 1, 2015 9:29 PM, "artnaseef" <ar...@artnaseef.com> wrote:
In this case I'd guess it would be implemented such that the consumer
would get closed via a ConsumerControl command or more harshly just
killing the consumer connection but if there were other consumers on
that connection that could cause all sorts of problems in the users
app. If the connection was dropped and the consumer was using failover
then it would attempt to reconnect and would end up recreating the
destination.
Doing this wouldn't really solve the scenario that the user is
attempting to solve, that is that a message could arrive after the
consumer shuts down because as you've guessed the auto create feature
would recreate the destination for a producer that suddenly appeared and
published this presumed last second message. So adding the option to
remove the Queue even if there are consumers doesn't really solve
anything and more likely just creates a new set of issues to contend with.
It'd be better to use some expiry on the messages and probably some
control channel to tell the consumer that it's work is done and that it
is safe for it to close.
>> Something has me confused here. If there is a consumer, why would messages
>> sit around unconsumed?
>>
>> Also, I have trouble understanding why one would want the broker to remove
>> a
>> destination while it has a consumer -- what is the consumer doing with the
>> destination then, and what is expected to happen to the consumer?
>>
>> Also, if messages are sitting in a queue, the queue cannot be GC'ed -
>> otherwise the broker is losing messages. A good solution there may be the
>> use of message TTLs.
>>
>> Perhaps a review of the architecture would help to determine the best way
>> to
>> fit ActiveMQ and the application to the problem.
>>
>> BTW, it sounds like the application may have a consumer leak. Consumers
>> can
>> be thought of as the center of purpose for a destination. A destination
>> that never has consumers would be a bit-bucket, or worse - a
>> continually-growing message store. Consumers that don't expect to receive
>> messages cannot be distinguished by ActiveMQ from those that do expect to
>> receive messages.
>>
>> Hope this helps.
>>
>>
>>
>> --
>> View this message in context:
>> http://activemq.2283324.n4.nabble.com/ActiveMQ-should-GC-inactive-when-producers-is-0-even-if-consumers-is-0-tp4690776p4690797.html
>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>
--
Tim Bish
Sr Software Engineer | RedHat Inc.
tim.bish@redhat.com | www.redhat.com
skype: tabish121 | twitter: @tabish121
blog: http://timbish.blogspot.com/
Re: ActiveMQ should GC inactive when producers is 0 , even if
consumers is > 0
Posted by Tim Bain <tb...@alumni.duke.edu>.
Also, the most common (I think) broker configuration will auto-create a
destination if a client tries to use it and it doesn't exist. So if you
delete the destination while a consumer is still connected, won't the
broker just immediately recreate it? If implemented, would this
enhancement actually do what you want?
It seems like you need some means for the consumer to figure out that the
producer is done and disconnect itself, which would then let the existing
destination GC logic work without this enhancement.
On Feb 1, 2015 9:29 PM, "artnaseef" <ar...@artnaseef.com> wrote:
> Something has me confused here. If there is a consumer, why would messages
> sit around unconsumed?
>
> Also, I have trouble understanding why one would want the broker to remove
> a
> destination while it has a consumer -- what is the consumer doing with the
> destination then, and what is expected to happen to the consumer?
>
> Also, if messages are sitting in a queue, the queue cannot be GC'ed -
> otherwise the broker is losing messages. A good solution there may be the
> use of message TTLs.
>
> Perhaps a review of the architecture would help to determine the best way
> to
> fit ActiveMQ and the application to the problem.
>
> BTW, it sounds like the application may have a consumer leak. Consumers
> can
> be thought of as the center of purpose for a destination. A destination
> that never has consumers would be a bit-bucket, or worse - a
> continually-growing message store. Consumers that don't expect to receive
> messages cannot be distinguished by ActiveMQ from those that do expect to
> receive messages.
>
> Hope this helps.
>
>
>
> --
> View this message in context:
> http://activemq.2283324.n4.nabble.com/ActiveMQ-should-GC-inactive-when-producers-is-0-even-if-consumers-is-0-tp4690776p4690797.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
Re: ActiveMQ should GC inactive when producers is 0 , even if
consumers is > 0
Posted by artnaseef <ar...@artnaseef.com>.
Something has me confused here. If there is a consumer, why would messages
sit around unconsumed?
Also, I have trouble understanding why one would want the broker to remove a
destination while it has a consumer -- what is the consumer doing with the
destination then, and what is expected to happen to the consumer?
Also, if messages are sitting in a queue, the queue cannot be GC'ed -
otherwise the broker is losing messages. A good solution there may be the
use of message TTLs.
Perhaps a review of the architecture would help to determine the best way to
fit ActiveMQ and the application to the problem.
BTW, it sounds like the application may have a consumer leak. Consumers can
be thought of as the center of purpose for a destination. A destination
that never has consumers would be a bit-bucket, or worse - a
continually-growing message store. Consumers that don't expect to receive
messages cannot be distinguished by ActiveMQ from those that do expect to
receive messages.
Hope this helps.
--
View this message in context: http://activemq.2283324.n4.nabble.com/ActiveMQ-should-GC-inactive-when-producers-is-0-even-if-consumers-is-0-tp4690776p4690797.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.