You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by RickBadertscher <rb...@packeteer.com> on 2006/04/10 17:37:32 UTC
Massive parallel message distribution (Thousands of clients)
I need to asynchronously transmit different messages to possibly thousands of
clients. I don't know enough about activemq implementation to know whether
each client maintains an open socket to the provider, if so, it would seem
jms point-to-point my not work for this type of problem as the provider
would be overloaded with open sockets. Does anyone have an insight on
this? I was thinking of just writing a simple multi-threaded distributor
to do what I need, but it seems that this problem should already be solved.
Thanks in advance.
- Rick
--
View this message in context: http://www.nabble.com/Massive-parallel-message-distribution-%28Thousands-of-clients%29-t1425732.html#a3844340
Sent from the ActiveMQ - User forum at Nabble.com.
Re: Massive parallel message distribution (Thousands of clients)
Posted by James Strachan <ja...@gmail.com>.
On 4/10/06, RickBadertscher <rb...@packeteer.com> wrote:
> James.Strachan wrote:
> >
> > On 4/10/06, RickBadertscher <rb...@packeteer.com> wrote:
> >> I need to asynchronously transmit different messages to possibly
> >> thousands of
> >> clients. I don't know enough about activemq implementation to know
> >> whether
> >> each client maintains an open socket to the provider, if so, it would
> >> seem
> >> jms point-to-point my not work for this type of problem as the provider
> >> would be overloaded with open sockets. Does anyone have an insight on
> >> this? I was thinking of just writing a simple multi-threaded
> >> distributor
> >> to do what I need, but it seems that this problem should already be
> >> solved.
> >> Thanks in advance.
> >
> > When you say "client" what do you mean? FWIW if its a proces (JVM) of
> > some kind then it needs to use some kind of networking (TCP /
> > multicast / UDP) to communicate with a JMS broker.
> >
> > The publish/subscribe feature of ActiveMQ can handle many thousands of
> > clients across a network of brokers; so don't worry about the number
> > of sockets; its an issue for the broker but not for the
> > producer/consumer side. On modern operating systems you can often have
> > 1000s of sockets open per JVM; on Solaris and modern linuxes this
> > number can be > 10K sockets.
>
> Hi James,
>
> a client is a JVM somewhere on a WAN (a java agent running on a device on
> the internet). The messages being distributed are different per client, so
> pub/sub is a no go as far as I understand it, it's point to point. I have
> to admit some ignorance here, but I get confused when you say the broker vs.
> producer/consumer,
So a JMS client connects to the broker over (say) a socket to a
broker. You can have 1 or many brokers in a network.
A single JMS connetion can have 0..many producers & consumers. Only 1
socket is used by the JMS connection to communicate with the broker.
> I guess I'm used to the jms provider just being an
> integral part of the appserver.
A JMS client is used by any JVM connecting on your messaging fabric.
However where the JMS brokers run is up to you; they can be in-process
in your producer/consumers or they can be separate processes.
In cases where you have lots of LAN traffic and some WAN traffic you
can use brokers as network-concentrators to minimse similar messages
being repeatedly sent over the WAN
> But, I think I get your point, I was
> worried that maintaining such a huge # of sockets was too heavyweight, but
> it sounds like its not. In this scenerio, would you recommend using
> ActiveMQ as a standalone process, or would it work just as well embedded in
> a jboss process. ?
Whether it should be embedded or separate depends on various things
like how many JVMS do you need, what is the network topology of the
JMS network and what message persistence strategy do you need.
But a real quick answer; I'd start your experiments with a single
remote JMS broker; its the easiest way to get started - that should be
able to handle 1000s of clients.
--
James
-------
http://radio.weblogs.com/0112098/
Re: Massive parallel message distribution (Thousands of clients)
Posted by RickBadertscher <rb...@packeteer.com>.
James.Strachan wrote:
>
> On 4/10/06, RickBadertscher <rb...@packeteer.com> wrote:
>> I need to asynchronously transmit different messages to possibly
>> thousands of
>> clients. I don't know enough about activemq implementation to know
>> whether
>> each client maintains an open socket to the provider, if so, it would
>> seem
>> jms point-to-point my not work for this type of problem as the provider
>> would be overloaded with open sockets. Does anyone have an insight on
>> this? I was thinking of just writing a simple multi-threaded
>> distributor
>> to do what I need, but it seems that this problem should already be
>> solved.
>> Thanks in advance.
>
> When you say "client" what do you mean? FWIW if its a proces (JVM) of
> some kind then it needs to use some kind of networking (TCP /
> multicast / UDP) to communicate with a JMS broker.
>
> The publish/subscribe feature of ActiveMQ can handle many thousands of
> clients across a network of brokers; so don't worry about the number
> of sockets; its an issue for the broker but not for the
> producer/consumer side. On modern operating systems you can often have
> 1000s of sockets open per JVM; on Solaris and modern linuxes this
> number can be > 10K sockets.
>
> James
>
>> - Rick
>> --
>> View this message in context:
>> http://www.nabble.com/Massive-parallel-message-distribution-%28Thousands-of-clients%29-t1425732.html#a3844340
>> Sent from the ActiveMQ - User forum at Nabble.com.
>>
>>
>
>
> --
>
> James
> -------
> http://radio.weblogs.com/0112098/
>
>
Hi James,
a client is a JVM somewhere on a WAN (a java agent running on a device on
the internet). The messages being distributed are different per client, so
pub/sub is a no go as far as I understand it, it's point to point. I have
to admit some ignorance here, but I get confused when you say the broker vs.
producer/consumer, in this scenerio I would look at the broker as the
producer (or at least on the same box as the producer), while the consumers
are remote jvm's. I guess I'm used to the jms provider just being an
integral part of the appserver. But, I think I get your point, I was
worried that maintaining such a huge # of sockets was too heavyweight, but
it sounds like its not. In this scenerio, would you recommend using
ActiveMQ as a standalone process, or would it work just as well embedded in
a jboss process. ?
thanks,
Rick
--
View this message in context: http://www.nabble.com/Massive-parallel-message-distribution-%28Thousands-of-clients%29-t1425732.html#a3845762
Sent from the ActiveMQ - User forum at Nabble.com.
Re: Massive parallel message distribution (Thousands of clients)
Posted by James Strachan <ja...@gmail.com>.
On 4/10/06, RickBadertscher <rb...@packeteer.com> wrote:
> I need to asynchronously transmit different messages to possibly thousands of
> clients. I don't know enough about activemq implementation to know whether
> each client maintains an open socket to the provider, if so, it would seem
> jms point-to-point my not work for this type of problem as the provider
> would be overloaded with open sockets. Does anyone have an insight on
> this? I was thinking of just writing a simple multi-threaded distributor
> to do what I need, but it seems that this problem should already be solved.
> Thanks in advance.
When you say "client" what do you mean? FWIW if its a proces (JVM) of
some kind then it needs to use some kind of networking (TCP /
multicast / UDP) to communicate with a JMS broker.
The publish/subscribe feature of ActiveMQ can handle many thousands of
clients across a network of brokers; so don't worry about the number
of sockets; its an issue for the broker but not for the
producer/consumer side. On modern operating systems you can often have
1000s of sockets open per JVM; on Solaris and modern linuxes this
number can be > 10K sockets.
James
> - Rick
> --
> View this message in context: http://www.nabble.com/Massive-parallel-message-distribution-%28Thousands-of-clients%29-t1425732.html#a3844340
> Sent from the ActiveMQ - User forum at Nabble.com.
>
>
--
James
-------
http://radio.weblogs.com/0112098/