C++ client API help

[M Q <un...@gmail.com>]

How can I get the qpid C++ client API to do the following:

* A producer is sending a stream of messages to a single message queue. Each
message represents a unit of work to be processed by a computational
cluster.  The producer's messages must accumulate on a single FIFO message
queue, called the work queue.

* The cluster of consumers read messages off the queue (round-robin style).
The consumers process each message and output the results onto some message
queue (called the response queue).

* If a client takes a message off the work queue and fails to create a
response in a given amount of time then the message must be put back on the
work queue. For example, the client might accept the unit of work and then
crash. The unit of work must still be completed so it must be returned to
the work queue for processing by another client.

For example, I might have a series of images that need to be processed by a
server farm. Each server can processes the images and I want to distribute
the workload across multiple servers. And of course I want all images to be
eventually processed, even if one of the severs fails at some point in the
process.

In terms of the QPID C++ API, how would this work? Should I use reply-to
headers in the work queue messages? Can qpid take care of detecting
time-outs and putting failed work requests back onto the work queue?

Re: C++ client API help

[Gordon Sim <gs...@redhat.com>]

M Q wrote:
> In terms of the QPID C++ API, how would this work? Should I use reply-to
> headers in the work queue messages? 

You can use that to tell the workers of the response queue to use certainly.

> Can qpid take care of detecting
> time-outs and putting failed work requests back onto the work queue?

If you use acknowledged delivery (which is the default), then if a 
worker crashes before returning from handling a message, that message 
will be re-queued and delivered to one of the surviving workers.

However there is no time-out feature built in. You could build this into 
your application - i.e. have a timer that you 'start' prior to starting 
each task and 'stop' once the task is completed. If the task takes to 
long the timer will hit a limit and cause the workers session to be 
closed, which in turn will cause the server to re-queue the unaccepted 
message.