You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by Seth Call <se...@gmail.com> on 2010/10/04 22:46:01 UTC
Controlling how, and which messages, a JMS Consumer pulls
Hi all,
Let me describe my end-goal, and see what you all think:
I have this concept of a 'worker', which is on the consuming end of a
JMS queue. The idea is that I'd like to have many of these workers
deployed.
At the same time, there are different types of work to be done, and a
worker can do 1 or more of these types of work; however, I'd like to
limit it such that a worker can only do 1 thing at once.
So, a worker has capabilities that may or may not mesh with a task,
and workers have a maximum number of concurrent tasks they can do.
It seems to me the right thing to do is to have one queue per
unit-of-work type. So if you are a worker and support 2 types of
work, then you'd ultimately be listening to two queues. But, once you
started working on one task pulled from one of the queues, it seems
you'd want to dynamically stop listening to the other queue--and
conversely, once the task is done, you'd start listening on that queue
again. But right away I'm concerned how one could guarantee just one
message is read from either queue at a time.
Another scenario is if no workers are currently available that can
handle a particular type of task because they are all busy, but there
are other things to do in the queue. So in that case, with the first
message that no one can currently handle, I'd like to put it back on
the queue (or you could say, just never take it off the queue), but
push it back so I can have the available workers look at the next
messages in line.
You can see, though, how eventually there could be an issue if there
were even two items in the queue, and they both are unable to be
processed at the moment because all the workers are busy. With the
reordering of the queue suggestion, you could quickly get into a
situation where the queue is read and thrashed at a high rate unless
something throttles this behavior.
Any suggestions as to what I could look at? I'm not seeing anything
on the JMS Camel page, or anything in the *Processors that might be of
help, but I'm also concerned I'm going about this the wrong way.
I realize this is a possibly very loaded question and help may not be
forthcoming. But I thought I'd ask.
Regards,
Seth
Re: Controlling how, and which messages, a JMS Consumer pulls
Posted by Seth Call <se...@gmail.com>.
>From more help by Hadrian and andrewm in IRC, I got it sorted out:
The missing piece is that I needed a way to let the consumer to be a
part of the routing, which is basically not what most of the canned
'Processors' let you do when JMS Queue's are involved, because pulling
a message off of the JMS Queue is not a trivial thing to recover from,
once you've done it.
So anyway, a JMS Selector let's a worker indicate to the broker what
type of jobs it can handle on startup. All I have to do is make sure
I pass in a well-defined header in the message that indicates the job
type, and my worker's selector can easily pinpoint what type of tasks
that given worker supports.
Combine that with a single queue (which is also straightforward with
the wildcard suggestions from Illtud), and a prefetchSize of 0 or 1,
and I have it all sorted out I believe!
Thanks everyone for their help!
Seth
On Mon, Oct 4, 2010 at 4:38 PM, Seth Call <se...@gmail.com> wrote:
> Hi Illtud,
>
>
> Essentially I have no natural hierarchy. The issue is that these
> workers will generally interoping with 3rd-party services, meaning
> it's very hard to know upfront what their runtime characteristics will
> be. We may find that one API of one 3rd-party fails alot and/or
> takes a long time, and we decide we need to fire up many Workers that
> are dedicated to that service so that we can improve concurrency and
> prevent that one service from backlogging the whole system.
>
> However, the whole point is that, after runnig this thing in
> production, and realizing it makes sense to have jobs of type 'A, B,
> C' lumped in usually with one worker type, and jobs D, E should always
> have a dedicated worker, then you may then have a hierarchy like:
>
> job.combinable.A
> job.combinable.B
> job.combinable.C
> job.standalone.D
> job.standalone.E
>
> And then if I wanted a worker to do A, B, C, my path is clear. If I
> wanted to do A and D, then I have a problem, sure, but the idea here
> is that I shouldn't need to once I know my 'realistic hierarchy' if
> that makes sense.
>
> Good suggestion, thanks very much.
>
> Seth
>
>
>
> On Mon, Oct 4, 2010 at 4:25 PM, Illtud Daniel <il...@llgc.org.uk> wrote:
>> On 04/10/10 21:46, Seth Call wrote:
>>
>>> It seems to me the right thing to do is to have one queue per
>>> unit-of-work type. So if you are a worker and support 2 types of
>>> work, then you'd ultimately be listening to two queues. But, once you
>>> started working on one task pulled from one of the queues, it seems
>>> you'd want to dynamically stop listening to the other queue--and
>>> conversely, once the task is done, you'd start listening on that queue
>>> again. But right away I'm concerned how one could guarantee just one
>>> message is read from either queue at a time.
>>
>> I think this could be done by using wildcards. If you have jobs
>> of type A, B, C, D & E, and jobs A, B and C are processorder
>> jobs (for example) and D & E are dispatchorder jobs, and you
>> have 3 consumers, X, Y & Z. X can do processorders and dispatchorder
>> -type jobs, Y can do processorders, and Z can do dispatchorder,
>> then you can have queues:
>>
>> queue.processorder.A
>> queue.processorder.B
>> queue.processorder.C
>> queue.dispatchorder.D
>> queue.dispatchorder.E
>>
>> Then X could subscribe to queue.*, Y to queue.processorder.*
>> and Z to queue.dispatchorder.*
>>
>> If your workers are single-threaded, and do their work in an onmessage
>> (Java) method, then they'll automatically do the stuff you want
>> (stop listening, not block other consumers) while they work on the
>> message.
>>
>>> Another scenario is if no workers are currently available that can
>>> handle a particular type of task because they are all busy, but there
>>> are other things to do in the queue.
>>
>> My solution would work for that, since there are separate queues.
>>
>> The only issue is if your task hierarchy doesn't break down handily
>> like mine does (eg if Z could do A & E, but not the other tasks), but
>> you'd have pretty odd workers to have that scenario (or I don't have
>> much imagination).
>>
>> --
>> Illtud Daniel illtud.daniel@llgc.org.uk
>> Prif Swyddog Technegol Chief Technical Officer
>> Llyfrgell Genedlaethol Cymru National Library of Wales
>>
>
Re: Controlling how, and which messages, a JMS Consumer pulls
Posted by Seth Call <se...@gmail.com>.
Hi Illtud,
Essentially I have no natural hierarchy. The issue is that these
workers will generally interoping with 3rd-party services, meaning
it's very hard to know upfront what their runtime characteristics will
be. We may find that one API of one 3rd-party fails alot and/or
takes a long time, and we decide we need to fire up many Workers that
are dedicated to that service so that we can improve concurrency and
prevent that one service from backlogging the whole system.
However, the whole point is that, after runnig this thing in
production, and realizing it makes sense to have jobs of type 'A, B,
C' lumped in usually with one worker type, and jobs D, E should always
have a dedicated worker, then you may then have a hierarchy like:
job.combinable.A
job.combinable.B
job.combinable.C
job.standalone.D
job.standalone.E
And then if I wanted a worker to do A, B, C, my path is clear. If I
wanted to do A and D, then I have a problem, sure, but the idea here
is that I shouldn't need to once I know my 'realistic hierarchy' if
that makes sense.
Good suggestion, thanks very much.
Seth
On Mon, Oct 4, 2010 at 4:25 PM, Illtud Daniel <il...@llgc.org.uk> wrote:
> On 04/10/10 21:46, Seth Call wrote:
>
>> It seems to me the right thing to do is to have one queue per
>> unit-of-work type. So if you are a worker and support 2 types of
>> work, then you'd ultimately be listening to two queues. But, once you
>> started working on one task pulled from one of the queues, it seems
>> you'd want to dynamically stop listening to the other queue--and
>> conversely, once the task is done, you'd start listening on that queue
>> again. But right away I'm concerned how one could guarantee just one
>> message is read from either queue at a time.
>
> I think this could be done by using wildcards. If you have jobs
> of type A, B, C, D & E, and jobs A, B and C are processorder
> jobs (for example) and D & E are dispatchorder jobs, and you
> have 3 consumers, X, Y & Z. X can do processorders and dispatchorder
> -type jobs, Y can do processorders, and Z can do dispatchorder,
> then you can have queues:
>
> queue.processorder.A
> queue.processorder.B
> queue.processorder.C
> queue.dispatchorder.D
> queue.dispatchorder.E
>
> Then X could subscribe to queue.*, Y to queue.processorder.*
> and Z to queue.dispatchorder.*
>
> If your workers are single-threaded, and do their work in an onmessage
> (Java) method, then they'll automatically do the stuff you want
> (stop listening, not block other consumers) while they work on the
> message.
>
>> Another scenario is if no workers are currently available that can
>> handle a particular type of task because they are all busy, but there
>> are other things to do in the queue.
>
> My solution would work for that, since there are separate queues.
>
> The only issue is if your task hierarchy doesn't break down handily
> like mine does (eg if Z could do A & E, but not the other tasks), but
> you'd have pretty odd workers to have that scenario (or I don't have
> much imagination).
>
> --
> Illtud Daniel illtud.daniel@llgc.org.uk
> Prif Swyddog Technegol Chief Technical Officer
> Llyfrgell Genedlaethol Cymru National Library of Wales
>
Re: Controlling how, and which messages, a JMS Consumer pulls
Posted by Illtud Daniel <il...@llgc.org.uk>.
On 04/10/10 21:46, Seth Call wrote:
> It seems to me the right thing to do is to have one queue per
> unit-of-work type. So if you are a worker and support 2 types of
> work, then you'd ultimately be listening to two queues. But, once you
> started working on one task pulled from one of the queues, it seems
> you'd want to dynamically stop listening to the other queue--and
> conversely, once the task is done, you'd start listening on that queue
> again. But right away I'm concerned how one could guarantee just one
> message is read from either queue at a time.
I think this could be done by using wildcards. If you have jobs
of type A, B, C, D & E, and jobs A, B and C are processorder
jobs (for example) and D & E are dispatchorder jobs, and you
have 3 consumers, X, Y & Z. X can do processorders and dispatchorder
-type jobs, Y can do processorders, and Z can do dispatchorder,
then you can have queues:
queue.processorder.A
queue.processorder.B
queue.processorder.C
queue.dispatchorder.D
queue.dispatchorder.E
Then X could subscribe to queue.*, Y to queue.processorder.*
and Z to queue.dispatchorder.*
If your workers are single-threaded, and do their work in an onmessage
(Java) method, then they'll automatically do the stuff you want
(stop listening, not block other consumers) while they work on the
message.
> Another scenario is if no workers are currently available that can
> handle a particular type of task because they are all busy, but there
> are other things to do in the queue.
My solution would work for that, since there are separate queues.
The only issue is if your task hierarchy doesn't break down handily
like mine does (eg if Z could do A & E, but not the other tasks), but
you'd have pretty odd workers to have that scenario (or I don't have
much imagination).
--
Illtud Daniel illtud.daniel@llgc.org.uk
Prif Swyddog Technegol Chief Technical Officer
Llyfrgell Genedlaethol Cymru National Library of Wales
Re: Controlling how, and which messages, a JMS Consumer pulls
Posted by Seth Call <se...@gmail.com>.
And then I find the Dead Letter Channel. It seems that perhaps, with
this mechanism, it might make sense to just reject the message once I
started a unit of work (or if I find the current worker that pulled
the task can't handle it), and let the routing re-route to the next
worker in line.
Not sure yet. I'll definitely dig into the DLC as much as I can go.
On Mon, Oct 4, 2010 at 3:46 PM, Seth Call <se...@gmail.com> wrote:
> Hi all,
>
> Let me describe my end-goal, and see what you all think:
>
> I have this concept of a 'worker', which is on the consuming end of a
> JMS queue. The idea is that I'd like to have many of these workers
> deployed.
>
> At the same time, there are different types of work to be done, and a
> worker can do 1 or more of these types of work; however, I'd like to
> limit it such that a worker can only do 1 thing at once.
>
> So, a worker has capabilities that may or may not mesh with a task,
> and workers have a maximum number of concurrent tasks they can do.
>
> It seems to me the right thing to do is to have one queue per
> unit-of-work type. So if you are a worker and support 2 types of
> work, then you'd ultimately be listening to two queues. But, once you
> started working on one task pulled from one of the queues, it seems
> you'd want to dynamically stop listening to the other queue--and
> conversely, once the task is done, you'd start listening on that queue
> again. But right away I'm concerned how one could guarantee just one
> message is read from either queue at a time.
>
> Another scenario is if no workers are currently available that can
> handle a particular type of task because they are all busy, but there
> are other things to do in the queue. So in that case, with the first
> message that no one can currently handle, I'd like to put it back on
> the queue (or you could say, just never take it off the queue), but
> push it back so I can have the available workers look at the next
> messages in line.
>
> You can see, though, how eventually there could be an issue if there
> were even two items in the queue, and they both are unable to be
> processed at the moment because all the workers are busy. With the
> reordering of the queue suggestion, you could quickly get into a
> situation where the queue is read and thrashed at a high rate unless
> something throttles this behavior.
>
> Any suggestions as to what I could look at? I'm not seeing anything
> on the JMS Camel page, or anything in the *Processors that might be of
> help, but I'm also concerned I'm going about this the wrong way.
>
> I realize this is a possibly very loaded question and help may not be
> forthcoming. But I thought I'd ask.
>
> Regards,
> Seth
>