You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by breezee <am...@hotmail.com> on 2008/01/06 12:31:58 UTC
more topics against more filters - overhead estmation
Hi,
I am planning a system which will involve one broker and many clients (about
40) that will use it.
calculating the usage, i will have to hold about 700 topics per client,
total of 28000.
That seems pretty large . I am considering making some topics shared for
different usages, using message filters.
My question is simple:
how can I best define the overhead of lots of topics/lots of filters? does
every new topic keep a live thread, for example? does large amount of
filters on the same topic wil make te filter non-usable?
Thanks in advance,
Amit.
--
View this message in context: http://www.nabble.com/more-topics-against-more-filters---overhead-estmation-tp14647100s2354p14647100.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: more topics against more filters - overhead estmation
Posted by breezee <am...@hotmail.com>.
Hi Rob,
Thanks for the quick response.
I am trying to evaluate the trade off with actual numbers, so I can break it
into pieces. For example: what the actual different between a topic (we use
topics, not queues) which is heavy loaded, against topics with light load.
Should I consider combine different light load topics into one with
selectors? this kind of questions.
To get into the hardcore of things: thread usage (does every topic hold its
own thread, or there is a sophisticated thread pool that can serve a large
amount of topics, and what is the time loss on a thread pool of size X when
the topics list size is much larger then X?), memory consumption (does every
topics holds internally its own data structure? how does it effect the
search time on less but big topics with selectors against lot of topics
without selectors?), time (how fast is the search for suitable subscribers
in a topic with couple of hundreds of selectors, against search of a topic,
with no selector?)
Thanks in advance,
Amit.
rajdavies wrote:
>
> I'll assume all the clients are active at the same time - and that you
> don't require durable subscribers. In which case there will be the
> destination itself - which is a relatively small object and queued
> messages for each subscriber to a destination.
> The trade of is one of performance against memory usage. Using filters
> (via selectors - http://activemq.apache.org/selectors.html) - will
> result in every message being evaluated - which will be moderately
> slower than the look-up for the destination itself. This can be
> traded against message references held for each subscriber (one per
> destination per client) - as well as the destination object overhead.
>
> Alternatively - you can use wild cards -
> http://activemq.apache.org/wildcards.html
> on the destinations - which is faster than selectors - providing you
> can break your destinations into a suitable hierarchy (this would be
> my choice btw).
>
> If you want your application to remain 100% JMS compliant, use
> selectors - as not every vendor can handle a large number of
> destinations - and wildcards are not always supported - and the syntax
> can change from vendor to vendor.
>
> Hope that helps,
>
> cheers,
>
> Rob
>
> On Jan 6, 2008, at 11:31 AM, breezee wrote:
>
>>
>> Hi,
>> I am planning a system which will involve one broker and many
>> clients (about
>> 40) that will use it.
>> calculating the usage, i will have to hold about 700 topics per
>> client,
>> total of 28000.
>> That seems pretty large . I am considering making some topics shared
>> for
>> different usages, using message filters.
>> My question is simple:
>> how can I best define the overhead of lots of topics/lots of
>> filters? does
>> every new topic keep a live thread, for example? does large amount of
>> filters on the same topic wil make te filter non-usable?
>>
>> Thanks in advance,
>> Amit.
>>
>> --
>> View this message in context:
>> http://www.nabble.com/more-topics-against-more-filters---overhead-estmation-tp14647100s2354p14647100.html
>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>
>
>
>
--
View this message in context: http://www.nabble.com/more-topics-against-more-filters---overhead-estmation-tp14647100s2354p14664571.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: more topics against more filters - overhead estmation
Posted by Rob Davies <ra...@gmail.com>.
I'll assume all the clients are active at the same time - and that you
don't require durable subscribers. In which case there will be the
destination itself - which is a relatively small object and queued
messages for each subscriber to a destination.
The trade of is one of performance against memory usage. Using filters
(via selectors - http://activemq.apache.org/selectors.html) - will
result in every message being evaluated - which will be moderately
slower than the look-up for the destination itself. This can be
traded against message references held for each subscriber (one per
destination per client) - as well as the destination object overhead.
Alternatively - you can use wild cards - http://activemq.apache.org/wildcards.html
on the destinations - which is faster than selectors - providing you
can break your destinations into a suitable hierarchy (this would be
my choice btw).
If you want your application to remain 100% JMS compliant, use
selectors - as not every vendor can handle a large number of
destinations - and wildcards are not always supported - and the syntax
can change from vendor to vendor.
Hope that helps,
cheers,
Rob
On Jan 6, 2008, at 11:31 AM, breezee wrote:
>
> Hi,
> I am planning a system which will involve one broker and many
> clients (about
> 40) that will use it.
> calculating the usage, i will have to hold about 700 topics per
> client,
> total of 28000.
> That seems pretty large . I am considering making some topics shared
> for
> different usages, using message filters.
> My question is simple:
> how can I best define the overhead of lots of topics/lots of
> filters? does
> every new topic keep a live thread, for example? does large amount of
> filters on the same topic wil make te filter non-usable?
>
> Thanks in advance,
> Amit.
>
> --
> View this message in context: http://www.nabble.com/more-topics-against-more-filters---overhead-estmation-tp14647100s2354p14647100.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>