You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by Jim Newsham <jn...@referentia.com> on 2010/12/21 00:27:59 UTC
long-running requests over jms
Hi everyone,
We are using Camel + ActiveMQ, with InOut messages and bean() routes, as
a form of flexible remoting (remote service invocation). This has been
working out quite well for us so far. One issue that we've run into is
that while most service requests complete very quickly, some particular
service requests can take a long time to execute (due to the processing
they must perform) -- perhaps many minutes, or in extreme cases possibly
an hour or more. However, if the request exceeds the configured jms
"requestTimeout" parameter, then the requester will receive a timeout
exception.
I feel that the requestTimeout parameter alone is not flexible enough to
do what we need. requestTimeout should be a somewhat small value so
that the application is responsive to disconnects (we certainly can't
set it to an hour; the 20 sec default seems reasonable). By contrast,
there doesn't seem to be a very well-defined, reasonable upper bound on
the long-running requests -- they could take an hour, perhaps more.
I feel that what we need to ensure responsiveness while supporting
long-running requests is some form of periodic, pending request
heartbeat. Coding this on an ad-hoc basis per request would be tedious
and cumbersome. It would be great if I could set a "requestTimeout" on
the producer endpoint (let's say 20s), and configure the consumer
endpoint with a heartbeat "requestKeepalive" parameter (let's say 15s),
and the consumer would send periodic messages to the producer which
would reset its timeout counter, until the consumer finally sends the
result.
What do you think? Is such a proposal feasible? Any alternative
ideas? I took a look at the jms component code, but I'm not quite sure
where the producer timeout happens, or where a consumer keepalive
processor would go.
Thanks,
Jim
Re: long-running requests over jms
Posted by Willem Jiang <wi...@gmail.com>.
If the producer side only support sync invocation, it will block the
thread which could be a part of jms consumer thread pool.
I just checked the code of JMS prodource doesn't block the thread to
wait for the response, so you will not face the issue that I had told
with the jms enpdoints.
Willem
On 12/22/10 7:54 AM, Jim Newsham wrote:
>
> Regarding the thread pool comment... what is the impact of a
> long-running request on thread consumption? If I understand correctly:
>
> - one of the consumer threads will be occupied; so if I configure
> "concurrentConsumers=50", then it would consume one of those 50 threads.
> - the thread performing the request on the producer side will block
> waiting for the response.
>
> Is this correct?
>
> Thanks,
> Jim
>
> On 12/20/2010 9:24 PM, Willem Jiang wrote:
>> Hi
>>
>> I don't think set the time out to be hours is go choice, if you have
>> lots of this kind of long running request, they will eat up your
>> thread pool, and the camel route will not take any message any more.
>>
>> The other choice you be leverage the JMS async nature and link the
>> request and response with correlation ID. Maybe you need to improve
>> your remote interface to provides an async call back API to make sure
>> there is no blocking thread to wait for the response.
>>
>> Willem
>>
>> On 12/21/10 7:27 AM, Jim Newsham wrote:
>>>
>>> Hi everyone,
>>>
>>> We are using Camel + ActiveMQ, with InOut messages and bean() routes, as
>>> a form of flexible remoting (remote service invocation). This has been
>>> working out quite well for us so far. One issue that we've run into is
>>> that while most service requests complete very quickly, some particular
>>> service requests can take a long time to execute (due to the processing
>>> they must perform) -- perhaps many minutes, or in extreme cases possibly
>>> an hour or more. However, if the request exceeds the configured jms
>>> "requestTimeout" parameter, then the requester will receive a timeout
>>> exception.
>>>
>>> I feel that the requestTimeout parameter alone is not flexible enough to
>>> do what we need. requestTimeout should be a somewhat small value so that
>>> the application is responsive to disconnects (we certainly can't set it
>>> to an hour; the 20 sec default seems reasonable). By contrast, there
>>> doesn't seem to be a very well-defined, reasonable upper bound on the
>>> long-running requests -- they could take an hour, perhaps more.
>>>
>>> I feel that what we need to ensure responsiveness while supporting
>>> long-running requests is some form of periodic, pending request
>>> heartbeat. Coding this on an ad-hoc basis per request would be tedious
>>> and cumbersome. It would be great if I could set a "requestTimeout" on
>>> the producer endpoint (let's say 20s), and configure the consumer
>>> endpoint with a heartbeat "requestKeepalive" parameter (let's say 15s),
>>> and the consumer would send periodic messages to the producer which
>>> would reset its timeout counter, until the consumer finally sends the
>>> result.
>>>
>>> What do you think? Is such a proposal feasible? Any alternative ideas? I
>>> took a look at the jms component code, but I'm not quite sure where the
>>> producer timeout happens, or where a consumer keepalive processor
>>> would go.
>>>
>>> Thanks,
>>> Jim
>>>
>>
>>
>
>
--
Willem
----------------------------------
FuseSource
Web: http://www.fusesource.com
Blog: http://willemjiang.blogspot.com (English)
http://jnn.javaeye.com (Chinese)
Twitter: willemjiang
Re: long-running requests over jms
Posted by Jim Newsham <jn...@referentia.com>.
Regarding the thread pool comment... what is the impact of a
long-running request on thread consumption? If I understand correctly:
- one of the consumer threads will be occupied; so if I configure
"concurrentConsumers=50", then it would consume one of those 50 threads.
- the thread performing the request on the producer side will block
waiting for the response.
Is this correct?
Thanks,
Jim
On 12/20/2010 9:24 PM, Willem Jiang wrote:
> Hi
>
> I don't think set the time out to be hours is go choice, if you have
> lots of this kind of long running request, they will eat up your
> thread pool, and the camel route will not take any message any more.
>
> The other choice you be leverage the JMS async nature and link the
> request and response with correlation ID. Maybe you need to improve
> your remote interface to provides an async call back API to make sure
> there is no blocking thread to wait for the response.
>
> Willem
>
> On 12/21/10 7:27 AM, Jim Newsham wrote:
>>
>> Hi everyone,
>>
>> We are using Camel + ActiveMQ, with InOut messages and bean() routes, as
>> a form of flexible remoting (remote service invocation). This has been
>> working out quite well for us so far. One issue that we've run into is
>> that while most service requests complete very quickly, some particular
>> service requests can take a long time to execute (due to the processing
>> they must perform) -- perhaps many minutes, or in extreme cases possibly
>> an hour or more. However, if the request exceeds the configured jms
>> "requestTimeout" parameter, then the requester will receive a timeout
>> exception.
>>
>> I feel that the requestTimeout parameter alone is not flexible enough to
>> do what we need. requestTimeout should be a somewhat small value so that
>> the application is responsive to disconnects (we certainly can't set it
>> to an hour; the 20 sec default seems reasonable). By contrast, there
>> doesn't seem to be a very well-defined, reasonable upper bound on the
>> long-running requests -- they could take an hour, perhaps more.
>>
>> I feel that what we need to ensure responsiveness while supporting
>> long-running requests is some form of periodic, pending request
>> heartbeat. Coding this on an ad-hoc basis per request would be tedious
>> and cumbersome. It would be great if I could set a "requestTimeout" on
>> the producer endpoint (let's say 20s), and configure the consumer
>> endpoint with a heartbeat "requestKeepalive" parameter (let's say 15s),
>> and the consumer would send periodic messages to the producer which
>> would reset its timeout counter, until the consumer finally sends the
>> result.
>>
>> What do you think? Is such a proposal feasible? Any alternative ideas? I
>> took a look at the jms component code, but I'm not quite sure where the
>> producer timeout happens, or where a consumer keepalive processor
>> would go.
>>
>> Thanks,
>> Jim
>>
>
>
Re: long-running requests over jms
Posted by Willem Jiang <wi...@gmail.com>.
Hi
I don't think set the time out to be hours is go choice, if you have
lots of this kind of long running request, they will eat up your thread
pool, and the camel route will not take any message any more.
The other choice you be leverage the JMS async nature and link the
request and response with correlation ID. Maybe you need to improve your
remote interface to provides an async call back API to make sure there
is no blocking thread to wait for the response.
Willem
On 12/21/10 7:27 AM, Jim Newsham wrote:
>
> Hi everyone,
>
> We are using Camel + ActiveMQ, with InOut messages and bean() routes, as
> a form of flexible remoting (remote service invocation). This has been
> working out quite well for us so far. One issue that we've run into is
> that while most service requests complete very quickly, some particular
> service requests can take a long time to execute (due to the processing
> they must perform) -- perhaps many minutes, or in extreme cases possibly
> an hour or more. However, if the request exceeds the configured jms
> "requestTimeout" parameter, then the requester will receive a timeout
> exception.
>
> I feel that the requestTimeout parameter alone is not flexible enough to
> do what we need. requestTimeout should be a somewhat small value so that
> the application is responsive to disconnects (we certainly can't set it
> to an hour; the 20 sec default seems reasonable). By contrast, there
> doesn't seem to be a very well-defined, reasonable upper bound on the
> long-running requests -- they could take an hour, perhaps more.
>
> I feel that what we need to ensure responsiveness while supporting
> long-running requests is some form of periodic, pending request
> heartbeat. Coding this on an ad-hoc basis per request would be tedious
> and cumbersome. It would be great if I could set a "requestTimeout" on
> the producer endpoint (let's say 20s), and configure the consumer
> endpoint with a heartbeat "requestKeepalive" parameter (let's say 15s),
> and the consumer would send periodic messages to the producer which
> would reset its timeout counter, until the consumer finally sends the
> result.
>
> What do you think? Is such a proposal feasible? Any alternative ideas? I
> took a look at the jms component code, but I'm not quite sure where the
> producer timeout happens, or where a consumer keepalive processor would go.
>
> Thanks,
> Jim
>
--
Willem
----------------------------------
FuseSource
Web: http://www.fusesource.com
Blog: http://willemjiang.blogspot.com (English)
http://jnn.javaeye.com (Chinese)
Twitter: willemjiang
Re: long-running requests over jms
Posted by Jim Newsham <jn...@referentia.com>.
On 12/21/2010 9:37 PM, Christian Schneider wrote:
> I think on JMS you never can be sure that the consumer is working on a
> request. (When using simple request / reply).
Yeah, that's precisely the issue I was hoping could be addressed. For
direct routes, you know that the consumer is working on the request, but
for jms routes, you don't. I thought, why can't jms routes be made to
work a little more like direct routes? Long-running requests could work
in just the same way as short-running requests, and if the "link" fails,
the producer gets an exception.
> So my suggestion is to simply offer a decoupled service that can be
> asked about the requests the consumer processes at the moment. I am
> not sure if this can be generically supported by camel.
> Perhaps camel could offer a way to look into a route and see which
> requests it processes via JMX. As this would be quite camel specific I
> guess the better way would be to write the monitoring service on your
> own.
> Another simple solution would be to let the consumer write keep alive
> messages an a separate topic that also contain information on the
> requests it works on. So you could also monitor the progress.
>
> The good thing in writing this on your own is that you could even send
> a progress in percent if this is possible in your case. Then the
> client could display how far the processing went.
>
Yeah, I was hoping to avoid creating a dichotomy in the code, where the
service interfaces for longer-running requests have to be coded
differently from the service interfaces for short-running requests. Not
to mention the added implementation complexity for each service
interface, and the added complexity in the routes (I can't use InOut
routes anymore, so I have to wire up reply routes). I was hoping that
communication layer could handle the complexity internally and expose a
uniform model to the service layer. Seems like I don't have a lot of
agreement on this idea, though.
Thanks for the alternative suggestions.
Regards,
Jim
> Best regards
>
> Christian
>
>
> Am 22.12.2010 00:49, schrieb Jim Newsham:
>>
>> Thanks for the responses Claus and Willem. I couldn't find a JIRA
>> issue related to jms exchange store, etc. I opened a new JIRA issue
>> for long-running jms requests:
>> https://issues.apache.org/jira/browse/CAMEL-3456.
>>
>> I'm trying to understand the pluggable store you're referring to. I
>> think what you're saying is that the pluggable store is purely
>> consumer-side, where the consumer would notice a long-running
>> request, park the exchange in the store, and release the consumer
>> thread, then when the result becomes available, it would pick up the
>> exchange again and send it on. Did I understand correctly? And the
>> purpose would be to free the consumer thread?
>>
>> I can see the potential problem that if a lot of long-running
>> requests are made, some thread pool might be exhausted. For our own
>> purposes, I'm not too worried about this problem as the long-running
>> requests will be rare and exceptional cases and we have small enough
>> load and can afford to configure a large enough thread pool so that
>> it's not an issue. So for us, the primary issue is that there's no
>> way for the producer to be aware that the consumer is still
>> processing the request (since setting requestTimeout to a large value
>> is not a good idea).
>>
>> Regards,
>> Jim
>>
>> On 12/21/2010 4:43 AM, Claus Ibsen wrote:
>>> Hi
>>>
>>> We have some thoughts of letting camel-jms support using a pluggable
>>> store so it can persist the in flight Exchange. And then on the reply
>>> consumer side, it can peek in the store to find the correlated
>>> Exchange and load it from the store, and continue routing the
>>> Exchange.
>>>
>>> Then we can support long use-cases. Also if Camel crashes or you
>>> shutdown Camel.
>>>
>>> I think we have created a JIRA ticket for it, I may be mistaken. Fell
>>> free to see if you can dig out the JIRA.
>>> If not create a new one and refer to this thread (using nabble etc.)
>>>
>>>
>>> On Tue, Dec 21, 2010 at 12:27 AM, Jim
>>> Newsham<jn...@referentia.com> wrote:
>>>> Hi everyone,
>>>>
>>>> We are using Camel + ActiveMQ, with InOut messages and bean()
>>>> routes, as a
>>>> form of flexible remoting (remote service invocation). This has been
>>>> working out quite well for us so far. One issue that we've run
>>>> into is that
>>>> while most service requests complete very quickly, some particular
>>>> service
>>>> requests can take a long time to execute (due to the processing
>>>> they must
>>>> perform) -- perhaps many minutes, or in extreme cases possibly an
>>>> hour or
>>>> more. However, if the request exceeds the configured jms
>>>> "requestTimeout"
>>>> parameter, then the requester will receive a timeout exception.
>>>>
>>>> I feel that the requestTimeout parameter alone is not flexible
>>>> enough to do
>>>> what we need. requestTimeout should be a somewhat small value so
>>>> that the
>>>> application is responsive to disconnects (we certainly can't set it
>>>> to an
>>>> hour; the 20 sec default seems reasonable). By contrast, there
>>>> doesn't seem
>>>> to be a very well-defined, reasonable upper bound on the long-running
>>>> requests -- they could take an hour, perhaps more.
>>>>
>>>> I feel that what we need to ensure responsiveness while supporting
>>>> long-running requests is some form of periodic, pending request
>>>> heartbeat.
>>>> Coding this on an ad-hoc basis per request would be tedious and
>>>> cumbersome.
>>>> It would be great if I could set a "requestTimeout" on the producer
>>>> endpoint (let's say 20s), and configure the consumer endpoint with a
>>>> heartbeat "requestKeepalive" parameter (let's say 15s), and the
>>>> consumer
>>>> would send periodic messages to the producer which would reset its
>>>> timeout
>>>> counter, until the consumer finally sends the result.
>>>>
>>>> What do you think? Is such a proposal feasible? Any alternative
>>>> ideas? I
>>>> took a look at the jms component code, but I'm not quite sure where
>>>> the
>>>> producer timeout happens, or where a consumer keepalive processor
>>>> would go.
>>>>
>>>> Thanks,
>>>> Jim
>>>>
>>>
>>>
>>
>>
>
Re: long-running requests over jms
Posted by Christian Schneider <ch...@die-schneider.net>.
I think on JMS you never can be sure that the consumer is working on a
request. (When using simple request / reply).
So my suggestion is to simply offer a decoupled service that can be
asked about the requests the consumer processes at the moment. I am not
sure if this can be generically supported by camel.
Perhaps camel could offer a way to look into a route and see which
requests it processes via JMX. As this would be quite camel specific I
guess the better way would be to write the monitoring service on your
own.
Another simple solution would be to let the consumer write keep alive
messages an a separate topic that also contain information on the
requests it works on. So you could also monitor the progress.
The good thing in writing this on your own is that you could even send a
progress in percent if this is possible in your case. Then the client
could display how far the processing went.
Best regards
Christian
Am 22.12.2010 00:49, schrieb Jim Newsham:
>
> Thanks for the responses Claus and Willem. I couldn't find a JIRA
> issue related to jms exchange store, etc. I opened a new JIRA issue
> for long-running jms requests:
> https://issues.apache.org/jira/browse/CAMEL-3456.
>
> I'm trying to understand the pluggable store you're referring to. I
> think what you're saying is that the pluggable store is purely
> consumer-side, where the consumer would notice a long-running request,
> park the exchange in the store, and release the consumer thread, then
> when the result becomes available, it would pick up the exchange again
> and send it on. Did I understand correctly? And the purpose would be
> to free the consumer thread?
>
> I can see the potential problem that if a lot of long-running requests
> are made, some thread pool might be exhausted. For our own purposes,
> I'm not too worried about this problem as the long-running requests
> will be rare and exceptional cases and we have small enough load and
> can afford to configure a large enough thread pool so that it's not an
> issue. So for us, the primary issue is that there's no way for the
> producer to be aware that the consumer is still processing the request
> (since setting requestTimeout to a large value is not a good idea).
>
> Regards,
> Jim
>
> On 12/21/2010 4:43 AM, Claus Ibsen wrote:
>> Hi
>>
>> We have some thoughts of letting camel-jms support using a pluggable
>> store so it can persist the in flight Exchange. And then on the reply
>> consumer side, it can peek in the store to find the correlated
>> Exchange and load it from the store, and continue routing the
>> Exchange.
>>
>> Then we can support long use-cases. Also if Camel crashes or you
>> shutdown Camel.
>>
>> I think we have created a JIRA ticket for it, I may be mistaken. Fell
>> free to see if you can dig out the JIRA.
>> If not create a new one and refer to this thread (using nabble etc.)
>>
>>
>> On Tue, Dec 21, 2010 at 12:27 AM, Jim
>> Newsham<jn...@referentia.com> wrote:
>>> Hi everyone,
>>>
>>> We are using Camel + ActiveMQ, with InOut messages and bean()
>>> routes, as a
>>> form of flexible remoting (remote service invocation). This has been
>>> working out quite well for us so far. One issue that we've run into
>>> is that
>>> while most service requests complete very quickly, some particular
>>> service
>>> requests can take a long time to execute (due to the processing they
>>> must
>>> perform) -- perhaps many minutes, or in extreme cases possibly an
>>> hour or
>>> more. However, if the request exceeds the configured jms
>>> "requestTimeout"
>>> parameter, then the requester will receive a timeout exception.
>>>
>>> I feel that the requestTimeout parameter alone is not flexible
>>> enough to do
>>> what we need. requestTimeout should be a somewhat small value so
>>> that the
>>> application is responsive to disconnects (we certainly can't set it
>>> to an
>>> hour; the 20 sec default seems reasonable). By contrast, there
>>> doesn't seem
>>> to be a very well-defined, reasonable upper bound on the long-running
>>> requests -- they could take an hour, perhaps more.
>>>
>>> I feel that what we need to ensure responsiveness while supporting
>>> long-running requests is some form of periodic, pending request
>>> heartbeat.
>>> Coding this on an ad-hoc basis per request would be tedious and
>>> cumbersome.
>>> It would be great if I could set a "requestTimeout" on the producer
>>> endpoint (let's say 20s), and configure the consumer endpoint with a
>>> heartbeat "requestKeepalive" parameter (let's say 15s), and the
>>> consumer
>>> would send periodic messages to the producer which would reset its
>>> timeout
>>> counter, until the consumer finally sends the result.
>>>
>>> What do you think? Is such a proposal feasible? Any alternative
>>> ideas? I
>>> took a look at the jms component code, but I'm not quite sure where the
>>> producer timeout happens, or where a consumer keepalive processor
>>> would go.
>>>
>>> Thanks,
>>> Jim
>>>
>>
>>
>
>
--
----
http://www.liquid-reality.de
Re: long-running requests over jms
Posted by Ashwin Karpe <ak...@fusesource.com>.
Hi Jim,
Long running requests can be challenging and I am not sure that depending on
JMS or any other protocol is the way to go.
The typical standard way to handle these situations is to use callback
handler sent along with a one-way request so that the server can
role-reverse to client and sent back notifications at any arbitrary point in
time (for e.g. long running, stock tickers etc). In messaging systems where
there is no way to assure a response in a given window, the standard way is
to do this via send correlation ids in messages and the onus is then on the
producer to ensure that the request is matched against a response and
appropriate action taken. Again in this scenario the time taken for a
response to be sent does not matter so long as the producer has no
additional QoS needs.
I do not think engineering Camel capability to support this is warranted.
Cheers,
Ashwin...
-----
---------------------------------------------------------
Ashwin Karpe
Apache Camel Committer & Sr Principal Consultant
FUSESource (a Progress Software Corporation subsidiary)
http://fusesource.com http://fusesource.com
Blog: http://opensourceknowledge.blogspot.com
http://opensourceknowledge.blogspot.com
---------------------------------------------------------
--
View this message in context: http://camel.465427.n5.nabble.com/long-running-requests-over-jms-tp3312711p3314947.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: long-running requests over jms
Posted by Jim Newsham <jn...@referentia.com>.
Thanks for the responses Claus and Willem. I couldn't find a JIRA issue
related to jms exchange store, etc. I opened a new JIRA issue for
long-running jms requests:
https://issues.apache.org/jira/browse/CAMEL-3456.
I'm trying to understand the pluggable store you're referring to. I
think what you're saying is that the pluggable store is purely
consumer-side, where the consumer would notice a long-running request,
park the exchange in the store, and release the consumer thread, then
when the result becomes available, it would pick up the exchange again
and send it on. Did I understand correctly? And the purpose would be
to free the consumer thread?
I can see the potential problem that if a lot of long-running requests
are made, some thread pool might be exhausted. For our own purposes,
I'm not too worried about this problem as the long-running requests will
be rare and exceptional cases and we have small enough load and can
afford to configure a large enough thread pool so that it's not an
issue. So for us, the primary issue is that there's no way for the
producer to be aware that the consumer is still processing the request
(since setting requestTimeout to a large value is not a good idea).
Regards,
Jim
On 12/21/2010 4:43 AM, Claus Ibsen wrote:
> Hi
>
> We have some thoughts of letting camel-jms support using a pluggable
> store so it can persist the in flight Exchange. And then on the reply
> consumer side, it can peek in the store to find the correlated
> Exchange and load it from the store, and continue routing the
> Exchange.
>
> Then we can support long use-cases. Also if Camel crashes or you shutdown Camel.
>
> I think we have created a JIRA ticket for it, I may be mistaken. Fell
> free to see if you can dig out the JIRA.
> If not create a new one and refer to this thread (using nabble etc.)
>
>
> On Tue, Dec 21, 2010 at 12:27 AM, Jim Newsham<jn...@referentia.com> wrote:
>> Hi everyone,
>>
>> We are using Camel + ActiveMQ, with InOut messages and bean() routes, as a
>> form of flexible remoting (remote service invocation). This has been
>> working out quite well for us so far. One issue that we've run into is that
>> while most service requests complete very quickly, some particular service
>> requests can take a long time to execute (due to the processing they must
>> perform) -- perhaps many minutes, or in extreme cases possibly an hour or
>> more. However, if the request exceeds the configured jms "requestTimeout"
>> parameter, then the requester will receive a timeout exception.
>>
>> I feel that the requestTimeout parameter alone is not flexible enough to do
>> what we need. requestTimeout should be a somewhat small value so that the
>> application is responsive to disconnects (we certainly can't set it to an
>> hour; the 20 sec default seems reasonable). By contrast, there doesn't seem
>> to be a very well-defined, reasonable upper bound on the long-running
>> requests -- they could take an hour, perhaps more.
>>
>> I feel that what we need to ensure responsiveness while supporting
>> long-running requests is some form of periodic, pending request heartbeat.
>> Coding this on an ad-hoc basis per request would be tedious and cumbersome.
>> It would be great if I could set a "requestTimeout" on the producer
>> endpoint (let's say 20s), and configure the consumer endpoint with a
>> heartbeat "requestKeepalive" parameter (let's say 15s), and the consumer
>> would send periodic messages to the producer which would reset its timeout
>> counter, until the consumer finally sends the result.
>>
>> What do you think? Is such a proposal feasible? Any alternative ideas? I
>> took a look at the jms component code, but I'm not quite sure where the
>> producer timeout happens, or where a consumer keepalive processor would go.
>>
>> Thanks,
>> Jim
>>
>
>
Re: long-running requests over jms
Posted by Claus Ibsen <cl...@gmail.com>.
Hi
We have some thoughts of letting camel-jms support using a pluggable
store so it can persist the in flight Exchange. And then on the reply
consumer side, it can peek in the store to find the correlated
Exchange and load it from the store, and continue routing the
Exchange.
Then we can support long use-cases. Also if Camel crashes or you shutdown Camel.
I think we have created a JIRA ticket for it, I may be mistaken. Fell
free to see if you can dig out the JIRA.
If not create a new one and refer to this thread (using nabble etc.)
On Tue, Dec 21, 2010 at 12:27 AM, Jim Newsham <jn...@referentia.com> wrote:
>
> Hi everyone,
>
> We are using Camel + ActiveMQ, with InOut messages and bean() routes, as a
> form of flexible remoting (remote service invocation). This has been
> working out quite well for us so far. One issue that we've run into is that
> while most service requests complete very quickly, some particular service
> requests can take a long time to execute (due to the processing they must
> perform) -- perhaps many minutes, or in extreme cases possibly an hour or
> more. However, if the request exceeds the configured jms "requestTimeout"
> parameter, then the requester will receive a timeout exception.
>
> I feel that the requestTimeout parameter alone is not flexible enough to do
> what we need. requestTimeout should be a somewhat small value so that the
> application is responsive to disconnects (we certainly can't set it to an
> hour; the 20 sec default seems reasonable). By contrast, there doesn't seem
> to be a very well-defined, reasonable upper bound on the long-running
> requests -- they could take an hour, perhaps more.
>
> I feel that what we need to ensure responsiveness while supporting
> long-running requests is some form of periodic, pending request heartbeat.
> Coding this on an ad-hoc basis per request would be tedious and cumbersome.
> It would be great if I could set a "requestTimeout" on the producer
> endpoint (let's say 20s), and configure the consumer endpoint with a
> heartbeat "requestKeepalive" parameter (let's say 15s), and the consumer
> would send periodic messages to the producer which would reset its timeout
> counter, until the consumer finally sends the result.
>
> What do you think? Is such a proposal feasible? Any alternative ideas? I
> took a look at the jms component code, but I'm not quite sure where the
> producer timeout happens, or where a consumer keepalive processor would go.
>
> Thanks,
> Jim
>
--
Claus Ibsen
-----------------
FuseSource
Email: cibsen@fusesource.com
Web: http://fusesource.com
Twitter: davsclaus
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/