Re: Scale question

[Jon Watte <jw...@gmail.com>]

>
> Yow... I've never tried something like that, but given enough switching

and horsepower, it's worth a try.



Thanks for your answer! Yeah, let's just slap it into production and see how
it goes, huh? :-)

I may set up some Amazon ECC experiment at some percentage of scale to see
how it goes, though.

The reason I'm asking is that I *know* I can write something that works like
this if I write it custom (and use some ad-hoc wire protocol), but why write
something custom with no standards or community, if something that exists
could do it already?
On the other hand, if nothing that exists can do it, well, I still need to
satisfy that kind of requirement somehow...

Also, if anyone else on the list is back from the holidays and have some
experience or theories, I'd appreciate any feedback!

Sincerely,

jw


--
Americans might object: there is no way we would sacrifice our living
standards for the benefit of people in the rest of the world. Nevertheless,
whether we get there willingly or not, we shall soon have lower consumption
rates, because our present rates are unsustainable.



On Thu, Dec 31, 2009 at 9:15 AM, Steve Huston <sh...@riverace.com> wrote:

> Hi Jon,
>
> > I notice that the default connection count for qpid is 300
> > connections. That seems very low for the use case I'm interested in.
>
> Ok, it's tunable.
>
> > Is it reasonable to expect that each broker can serve 10,000
> > to 20,000 connections, if each connection just sees a few
> > messages a second, totaling perhaps a kilobyte each of
> > payload per second?
>
> Those numbers will pretty well max out a Gbit ethernet interface, but
> there's no reason it can't work with sufficient hardware. 60K msgs/sec
> should be doable without much problem.
>
> > Does this expectation hold up if I use queue forward
> > clustering (all exchanges are fan-out) across a hundred
> > brokers, each with 10,000 to 20,000 users?
>
> Yow... I've never tried something like that, but given enough switching
> and horsepower, it's worth a try.
>
> Keep us posted... Sounds like an interesting use.
>
> -Steve
>
> --
> Steve Huston, Riverace Corporation
> Total Lifecycle Support for Your Networked Applications
> http://www.riverace.com
>
>
> ---------------------------------------------------------------------
> Apache Qpid - AMQP Messaging Implementation
> Project:      http://qpid.apache.org
> Use/Interact: mailto:users-subscribe@qpid.apache.org
>
>

Re: Scale question

[Carl Trieloff <cc...@redhat.com>]

make sure to also set ulimit appropriately.

Carl.


On 01/02/2010 02:14 PM, Jon Watte wrote:
>> Yow... I've never tried something like that, but given enough switching
>>      
> and horsepower, it's worth a try.
>
>
>
> Thanks for your answer! Yeah, let's just slap it into production and see how
> it goes, huh? :-)
>
> I may set up some Amazon ECC experiment at some percentage of scale to see
> how it goes, though.
>
> The reason I'm asking is that I *know* I can write something that works like
> this if I write it custom (and use some ad-hoc wire protocol), but why write
> something custom with no standards or community, if something that exists
> could do it already?
> On the other hand, if nothing that exists can do it, well, I still need to
> satisfy that kind of requirement somehow...
>
> Also, if anyone else on the list is back from the holidays and have some
> experience or theories, I'd appreciate any feedback!
>
> Sincerely,
>
> jw
>
>
> --
> Americans might object: there is no way we would sacrifice our living
> standards for the benefit of people in the rest of the world. Nevertheless,
> whether we get there willingly or not, we shall soon have lower consumption
> rates, because our present rates are unsustainable.
>
>
>
> On Thu, Dec 31, 2009 at 9:15 AM, Steve Huston<sh...@riverace.com>  wrote:
>
>    
>> Hi Jon,
>>
>>      
>>> I notice that the default connection count for qpid is 300
>>> connections. That seems very low for the use case I'm interested in.
>>>        
>> Ok, it's tunable.
>>
>>      
>>> Is it reasonable to expect that each broker can serve 10,000
>>> to 20,000 connections, if each connection just sees a few
>>> messages a second, totaling perhaps a kilobyte each of
>>> payload per second?
>>>        
>> Those numbers will pretty well max out a Gbit ethernet interface, but
>> there's no reason it can't work with sufficient hardware. 60K msgs/sec
>> should be doable without much problem.
>>
>>      
>>> Does this expectation hold up if I use queue forward
>>> clustering (all exchanges are fan-out) across a hundred
>>> brokers, each with 10,000 to 20,000 users?
>>>        
>> Yow... I've never tried something like that, but given enough switching
>> and horsepower, it's worth a try.
>>
>> Keep us posted... Sounds like an interesting use.
>>
>> -Steve
>>
>> --
>> Steve Huston, Riverace Corporation
>> Total Lifecycle Support for Your Networked Applications
>> http://www.riverace.com
>>
>>
>> ---------------------------------------------------------------------
>> Apache Qpid - AMQP Messaging Implementation
>> Project:      http://qpid.apache.org
>> Use/Interact: mailto:users-subscribe@qpid.apache.org
>>
>>
>>      
>    


---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project:      http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org