Message announcement and routing

[Eugeny N Dzhurinsky <bo...@redwerk.com>]

Hello!

Can somebody please take a look at my question below and let me know is it
possible to achieve such behavior with ActiveMQ?

I need to distribute the system which is doing heavy calculation for clients.
This means the client sends the initial query to start the calculation task to
the JMS network. In the JMS network there is several consumer hosts which have
the consumer application listening to the query. This application has a
thread pool of workers, which aimed to do the job.

Now, the flow of typical request processing is as below:

1) the client sends the request to allocate the job. One of free consumers
pick up this request, prepares the environment and then sends back to the
client's temporary queue the response about it is ready to serve.
(request-response as described in the ActiveMQ FAQ).

2) the client sends the sequence of commands and data to be processed by the
consumer as a set of messages, the consumer does it's job and provides the
producer with the set of responses.

3) the client gets finished and tells the worker it is free, so the worker can
pick up the job requests from other clients.

At the first look, allocation of "command channel" can be done with having
temporary queues: once the worker gets the initial request, it creates the
temporary queue and sends back to the client the reference to the queue in JMS
reply-to header. 

The worker does not acknowledge the JMS broker it had processed the message,
so the broker do not tries to send the initial request to another clients and
does not relay another requests from clients to this worker. I am not sure is
it works in this way?

Once the worker does the job serving the client's requests within it's
temporary queue, the worker sends the acknowledge to the broker to identify
the message is processed and the worker is ready to serve another messages.

Does everything I explained makes sense and can be done with ActiveMQ?

Thank you in advance!

-- 
Eugene N Dzhurinsky

Re: Message announcement and routing

[Eugeny N Dzhurinsky <bo...@redwerk.com>]

On Tue, Aug 26, 2008 at 10:35:31AM +0100, James Strachan wrote:
> > It's a good point, however this isn't the case we are worrying about for now,
> > we are just trying to prove the concept. If the consumer did not send the
> > acknowledgement, will the broker not route another messages to that consumer?
> 
> Yes - assuming the consumer closes or the process dies. If a valid
> consumer just sits there and never acks (assuming you're not using
> AUTO_ACK mode) then the broker will just assume your consumer is
> taking a long time to process the message (which is valid in many
> circumstances).

Perfect! Thank you!

> Though if you use temporary queues for all communication after the
> first message to the consumer - then there is no failover if a
> consumer dies - the whole conversation is dropped as the temporary
> queue is deleted when the consumer goes away; but then I think thats
> what you want.
> 
> i.e. reliable load balancing of the first incoming message - from that
> point on the rest of the conversation takes place on a temporary
> queue. If the consumer dies the client has to restart the conversation
> again.

Okay, thank you for the information, it was really helpful and we got some
things to think about and consider for finding a workaround later :)

-- 
Eugene N Dzhurinsky

Re: Message announcement and routing

[James Strachan <ja...@gmail.com>]

2008/8/26 Eugene Dzhurinsky <bo...@redwerk.com>:
> On Tue, Aug 26, 2008 at 09:17:09AM +0100, James Strachan wrote:
>> How many commands and sets of messages are required? Only normally
>> with the the worker pattern you send all the required commands to be
>> executed by a consumer in a single message.
>
> In fact the consumer may issue various commands depending on results returned
> by the worker and user input, so it is not possible to use batch of commands.
>
>> While supporting many messages is possible - its just way more chatty
>> and complex; requiring dealing with partial failure and so forth.
>
> We don't need to solve the task when the worker consumer going down during the
> processing of intermediate commands, so we just presume the consumer will be
> alive from getting the initial command to finishing the processing after the
> client is satisfied with results.
>
>> If you want reliability, you should use message acknowledgement or
>> transactions so the broker can failover to other clients if the
>> consumer fails to process the message. But if you are sending many
>> messages in a stateful conversation, you're gonna have to implement
>> your own failover protocol as you're gonna have to restart the
>> conversation and reply all the messages to another consumer if you get
>> a failure.
>
> It's a good point, however this isn't the case we are worrying about for now,
> we are just trying to prove the concept. If the consumer did not send the
> acknowledgement, will the broker not route another messages to that consumer?

Yes - assuming the consumer closes or the process dies. If a valid
consumer just sits there and never acks (assuming you're not using
AUTO_ACK mode) then the broker will just assume your consumer is
taking a long time to process the message (which is valid in many
circumstances).

Though if you use temporary queues for all communication after the
first message to the consumer - then there is no failover if a
consumer dies - the whole conversation is dropped as the temporary
queue is deleted when the consumer goes away; but then I think thats
what you want.

i.e. reliable load balancing of the first incoming message - from that
point on the rest of the conversation takes place on a temporary
queue. If the consumer dies the client has to restart the conversation
again.

-- 
James
-------
http://macstrac.blogspot.com/

Open Source Integration
http://open.iona.com

Re: Message announcement and routing

[Eugene Dzhurinsky <bo...@redwerk.com>]

On Tue, Aug 26, 2008 at 09:17:09AM +0100, James Strachan wrote:
> How many commands and sets of messages are required? Only normally
> with the the worker pattern you send all the required commands to be
> executed by a consumer in a single message.

In fact the consumer may issue various commands depending on results returned
by the worker and user input, so it is not possible to use batch of commands.

> While supporting many messages is possible - its just way more chatty
> and complex; requiring dealing with partial failure and so forth.

We don't need to solve the task when the worker consumer going down during the
processing of intermediate commands, so we just presume the consumer will be
alive from getting the initial command to finishing the processing after the
client is satisfied with results.

> If you want reliability, you should use message acknowledgement or
> transactions so the broker can failover to other clients if the
> consumer fails to process the message. But if you are sending many
> messages in a stateful conversation, you're gonna have to implement
> your own failover protocol as you're gonna have to restart the
> conversation and reply all the messages to another consumer if you get
> a failure.

It's a good point, however this isn't the case we are worrying about for now,
we are just trying to prove the concept. If the consumer did not send the
acknowledgement, will the broker not route another messages to that consumer?

> The easiest solution is to use a single request message and single
> response message; then failover and reliability are trivial and
> ActiveMQ can do all that for you.

Unfortunately, we can't encapsulate whole bunch of commands into the single
message, because it is not predictable of which commands will be issued by the
client (this depends on user input).

-- 
Eugene N Dzhurinsky

Re: Message announcement and routing

[James Strachan <ja...@gmail.com>]

2008/8/25 Eugeny N Dzhurinsky <bo...@redwerk.com>:
> Hello!
>
> Can somebody please take a look at my question below and let me know is it
> possible to achieve such behavior with ActiveMQ?
>
> I need to distribute the system which is doing heavy calculation for clients.
> This means the client sends the initial query to start the calculation task to
> the JMS network. In the JMS network there is several consumer hosts which have
> the consumer application listening to the query. This application has a
> thread pool of workers, which aimed to do the job.
>
> Now, the flow of typical request processing is as below:
>
> 1) the client sends the request to allocate the job. One of free consumers
> pick up this request, prepares the environment and then sends back to the
> client's temporary queue the response about it is ready to serve.
> (request-response as described in the ActiveMQ FAQ).
>
> 2) the client sends the sequence of commands and data to be processed by the
> consumer as a set of messages, the consumer does it's job and provides the
> producer with the set of responses.

How many commands and sets of messages are required? Only normally
with the the worker pattern you send all the required commands to be
executed by a consumer in a single message.

While supporting many messages is possible - its just way more chatty
and complex; requiring dealing with partial failure and so forth.


> 3) the client gets finished and tells the worker it is free, so the worker can
> pick up the job requests from other clients.
>
> At the first look, allocation of "command channel" can be done with having
> temporary queues: once the worker gets the initial request, it creates the
> temporary queue and sends back to the client the reference to the queue in JMS
> reply-to header.
>
> The worker does not acknowledge the JMS broker it had processed the message,
> so the broker do not tries to send the initial request to another clients and
> does not relay another requests from clients to this worker. I am not sure is
> it works in this way?

If you want reliability, you should use message acknowledgement or
transactions so the broker can failover to other clients if the
consumer fails to process the message. But if you are sending many
messages in a stateful conversation, you're gonna have to implement
your own failover protocol as you're gonna have to restart the
conversation and reply all the messages to another consumer if you get
a failure.

The easiest solution is to use a single request message and single
response message; then failover and reliability are trivial and
ActiveMQ can do all that for you.

-- 
James
-------
http://macstrac.blogspot.com/

Open Source Integration
http://open.iona.com