Publish/Subscribe with ActiveMQ

["JK (Programmer)" <jk...@penguinsfan.com>]

I am in the process of evaluating the use of ActiveMQ in a large pub/sub
architecture.

I was very impressed with the large list of features supported by ActiveMQ,
and am trying to evaluate if it is suitable for the system.

There will be 1.5 million messages per day, MOST of which each fan out to
all 10 subscribers. In the long term, there may be several times this many
subscribers but not in the immediate future. Average message size will be
10KB, but a few thousand messages per day will be several megabytes. The
platform would be Windows/.NET using NMS (or maybe STOMP). Hope to use the
embedded database with master/slave failover a possibility.

These subscribers must be DURABLE (probably done using the virtual topic
feature), and (for worst case planning purposes) several of the subscribers
may be down for a day or more.

If anyone has experience with a system similar to these specifications, I
would much appreciate your comments on how well it worked out, what the
major issues were, etc. You may post your comments here or email them
directly to me if for some reason you do not want to post them.

I am concerned with some of the issues in the issue tracker, especially a
few that are marked as "Blocker".
AMQ-845: Sending messages to a topic with an inactive durable subscription
will hang producers
AMQ-721: Openwire client hangs after receiving 999 messages

Is AMQ-845 a blocker for the scenario I describe (millions of messages to
several failed subscribers)?

By "Blocker", does this mean that no new release of ActiveMQ until these
issues are resolved?

I see that a "Fix Version" is unassigned on these bugs. When is a release
anticipated that will fix these issues (if you think they are problems for
my scenario)?

Obviously I will need to do some experimentation with ActiveMQ (I've already
done a little), but would appreciate the heads up.

Thank you in advance.

-- 
View this message in context: http://www.nabble.com/Publish-Subscribe-with-ActiveMQ-tf2626227.html#a7328565
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: Publish/Subscribe with ActiveMQ

["jk@penguinsfan.com" <jk...@penguinsfan.com>]

Just to let you know that after further experimentation it does not seem
likely that we will be able to use ActiveMQ.

Testing a scenario where ActiveMQ is shutdown and restarted, the
subscription recovery time was humongous for 50000 messages x 10 virtual
subscribers.  I was looking for something that would work when millions of
messages were on disk, possible to subscribers that are non-operational for
a day or two.

This was disappointing, because the performance seemed excellent until I saw
this could happen.

I believe this may be related to AMQ-493:  Improve queue and durable
subscription recovery performance.

It would be great if ActiveMQ could someday handle this kind of large
durable volume (does anyone know if the intended fix might handle volumes
such as what I describe?).

If I missed something, and anyone has any comments as to how I could "get
this to work", please let me know.  


jk@penguinsfan.com wrote:
> 
> I am in the process of evaluating the use of ActiveMQ in a large pub/sub
> architecture.
> 
> ...
> 
> There will be 1.5 million messages per day, MOST of which each fan out to
> all 10 subscribers. In the long term, there may be several times this many
> subscribers but not in the immediate future. Average message size will be
> 10KB, but a few thousand messages per day will be several megabytes. The
> platform would be Windows/.NET using NMS (or maybe STOMP). Hope to use the
> embedded database with master/slave failover a possibility.
> 
> These subscribers must be DURABLE (probably done using the virtual topic
> feature), and (for worst case planning purposes) several of the
> subscribers may be down for a day or more.
> 
> ...
> 
> 

-- 
View this message in context: http://www.nabble.com/Publish-Subscribe-with-ActiveMQ-tf2626227.html#a7351789
Sent from the ActiveMQ - User mailing list archive at Nabble.com.