facility for temporary connections?

[joe <fi...@yahoo.de>]

Hi,

I am searching for a kind of MOM technology and played around with 
ActiveMQ in the last weeks.
The beginning was/is quite hard for me, but with the MEAP of "ActiveMQ 
in Action" and the docs on your pages i made some progress. ;)

For our requirements ActiveMQ looks promising, but there is one thing i 
am not sure about.

We have one central server where a scheduler is running. This scheduler 
produces jobs for clients(machines in the network). Most of the jobs are 
scheduled between 02.00-05.00 but some are scheduled at other times. 
This can be defined by the users.

Cause we need to support many clients (30-5000) i am not sure the broker 
is able to handle so many client connections regarding the needed threads.

Searching the web, it seems possible to tune ActiveMQ to handle so many 
connections, but it seems to be quite difficult/tricky.

Consindering our requirenments, its not needed that the MOM holds a 
connection to every client at every time. For us it would be ok if the 
broker establishes a connection when the connection is needed, then 
sending the message and then close the connection.

Is there a facility to use ActiveMQ in this way?

Re: facility for temporary connections?

[Dejan Bosanac <de...@nighttale.net>]

Hi Joe,

one queue sounds better thread-wise. As the number of clients broker can
handle, it's best to do some testing and find that figure for your
environment/platform.

Cheers
--
Dejan Bosanac - http://twitter.com/dejanb

Open Source Integration - http://fusesource.com/
ActiveMQ in Action - http://www.manning.com/snyder/
Blog - http://www.nighttale.net


On Mon, Mar 29, 2010 at 12:11 PM, joe <fi...@yahoo.de> wrote:

> Hi Dejan,
>
>
> > the main question is do you really need that many clients and can you
> > redesign your system to use smaller number of clients and selectors
> > for example?
>
> Every Client has some data which should be backuped. So if a customer has
> so many clients he wishes to backup it must be supported. The backup is done
> over direct socket communication by a legacy application, the broker
> wouldn't be involved for that task, he just has to inform the client about
> executing a job.
> In general the messages handled by the broker are quite small and the
> performance is not so important.
>
> Currently this is done by a kind of proprietary RMI-Broker, which works ok
> for this issue but our Broker has some other pitfalls so we are looking for
> a replacement.
>
>
> > as of scaling, the best starting point to scaling the broker is this
> > article
> >
> http://activemq.apache.org/how-do-i-configure-10s-of-1000s-of-queues-in-a-single-broker-.html
>
> During my experiments i used one queue for all clients and the IP of the
> client as a Selector.
> is it better to use one queue for every client when having many clients?
>
>
> > You may also want to consider creating a network of brokers for this >
> task.
>
> Yes this sounds very interesting.
> So for supporting 5000 clients its possible to use 10 Brokers each handling
> connections for 500 clients?
> If i understand this strategy/feature correctly it should be possible to
> support 100 of 1000s of clients?!
> Ok its some work to configure/administer all these brokers but if a network
> of brokers works reliable this is definitely something i should evaluate.
>
>
>
>

Re: facility for temporary connections?

[joe <fi...@yahoo.de>]

Hi Dejan,

 > the main question is do you really need that many clients and can you
 > redesign your system to use smaller number of clients and selectors
 > for example?

Every Client has some data which should be backuped. So if a customer 
has so many clients he wishes to backup it must be supported. The backup 
is done over direct socket communication by a legacy application, the 
broker wouldn't be involved for that task, he just has to inform the 
client about executing a job.
In general the messages handled by the broker are quite small and the 
performance is not so important.

Currently this is done by a kind of proprietary RMI-Broker, which works 
ok for this issue but our Broker has some other pitfalls so we are 
looking for a replacement.

 > as of scaling, the best starting point to scaling the broker is this
 > article
 > 
http://activemq.apache.org/how-do-i-configure-10s-of-1000s-of-queues-in-a-single-broker-.html

During my experiments i used one queue for all clients and the IP of the 
client as a Selector.
is it better to use one queue for every client when having many clients?

 > You may also want to consider creating a network of brokers for this 
 > task.

Yes this sounds very interesting.
So for supporting 5000 clients its possible to use 10 Brokers each 
handling connections for 500 clients?
If i understand this strategy/feature correctly it should be possible to 
support 100 of 1000s of clients?!
Ok its some work to configure/administer all these brokers but if a 
network of brokers works reliable this is definitely something i should 
evaluate.

Re: facility for temporary connections?

[Dejan Bosanac <de...@nighttale.net>]

Hi Joe,

the main question is do you really need that many clients and can you
redesign your system to use smaller number of clients and selectors for
example?

as of scaling, the best starting point to scaling the broker is this
article

http://activemq.apache.org/how-do-i-configure-10s-of-1000s-of-queues-in-a-single-broker-.html

and looking into

activemq-scalability.xml

provided with the ActiveMQ distribution.

You may also want to consider creating a network of brokers for this task.

As long as restraining number of connections, you can take a look here

https://issues.apache.org/activemq/browse/AMQ-1928

for the similar feature. Just that it throws an exception instead of waiting
for the connection to be stopped. Finally, you can write a plugin

http://activemq.apache.org/developing-plugins.html

that will do what you want.

Cheers
--
Dejan Bosanac - http://twitter.com/dejanb

Open Source Integration - http://fusesource.com/
ActiveMQ in Action - http://www.manning.com/snyder/
Blog - http://www.nighttale.net


On Sun, Mar 28, 2010 at 4:33 PM, joe <fi...@yahoo.de> wrote:

> Hi,
>
> I am searching for a kind of MOM technology and played around with ActiveMQ
> in the last weeks.
> The beginning was/is quite hard for me, but with the MEAP of "ActiveMQ in
> Action" and the docs on your pages i made some progress. ;)
>
> For our requirements ActiveMQ looks promising, but there is one thing i am
> not sure about.
>
> We have one central server where a scheduler is running. This scheduler
> produces jobs for clients(machines in the network). Most of the jobs are
> scheduled between 02.00-05.00 but some are scheduled at other times. This
> can be defined by the users.
>
> Cause we need to support many clients (30-5000) i am not sure the broker is
> able to handle so many client connections regarding the needed threads.
>
> Searching the web, it seems possible to tune ActiveMQ to handle so many
> connections, but it seems to be quite difficult/tricky.
>
> Consindering our requirenments, its not needed that the MOM holds a
> connection to every client at every time. For us it would be ok if the
> broker establishes a connection when the connection is needed, then sending
> the message and then close the connection.
>
> Is there a facility to use ActiveMQ in this way?
>
>