You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@camel.apache.org by Bilgin Ibryam <bi...@gmail.com> on 2013/02/21 18:12:33 UTC
Seda component not handing over the completions
Hi,
I have a route where I consume from a file endpoint and put them in VM
endpoint, so that another camel route from different camel app can process
it a little later asynchronously.
The file endpoint has move=processed for removing the processed files, but
even the exchanges are delivered to VM endpoint, the exchange is not
considered completed, so the original file stays locked and cannot process
other files before I start the second camel application and process the
exchanges from VM endpoint.
SEDA has waitForTaskToComplete option, which controls whether the caller
should wait for the async task to complete or not before continuing. But
even when waitForTaskToComplete=Never is set, the completions are handed
over to VM queue, so it is not possible to complete the exchange.
Is there a way to complete the exchange when it is put in VM queue? Or
there is a possibility for extending SEDA component with additional option
for not handling over the completions when waitForTaskToComplete=Never is
set?
WDYT
Bilgin
Re: Seda component not handing over the completions
Posted by Willem jiang <wi...@gmail.com>.
InOnly is just setting the Message Exchange pattern, I don't think it will effect the completions handing over things.
BTW, If we don't hand over the completions, we still find a way to call the onComplete method.
In most case, the user doesn't need to know any thing about it, but if there are some transaction work need to do, the onComplete method is a good start point.
How about we add a option to let the Seda or VM endpoint to call the onComplete method once it send the exchange to the queue?
--
Willem Jiang
Red Hat, Inc.
FuseSource is now part of Red Hat
Web: http://www.fusesource.com | http://www.redhat.com
Blog: http://willemjiang.blogspot.com (http://willemjiang.blogspot.com/) (English)
http://jnn.iteye.com (http://jnn.javaeye.com/) (Chinese)
Twitter: willemjiang
Weibo: 姜宁willem
On Monday, February 25, 2013 at 10:11 PM, Raul Kripalani wrote:
> >
> > >
> > > The safest option is to convert the body to a String, and either use the
> > > threads DSL or inOnly(vm:...) to modify the exchange pattern onwards.
> >
> >
> > I've converted the body to a String, and then send the exchange to vm
> > endpoint using in only patters. At this point I expected to use VM endpoint
> > as an inmemory VM queue for exchanges, but cannot do that, because the
> > completions are also handled to that queue. Do I miss something here?
>
>
>
>
> That sounds fishy. Functionally speaking, I'd expect an InOnly dispatch to
> not hand over any completions, as we explicitly tell Camel we don't care
> what happens thereafter by using inOnly(); but then again, on some level
> it's sensible to provide the guarantee that whatever is behind the VM
> endpoint won't find an incoherent state whenever it executes.
>
> If this is really like this, it would make sense to add a
> handoverCompletions option to the Seda endpoint.
>
> For your immediate problem, did you try using the threads() DSL?
>
> Regards,
>
> *Raúl Kripalani*
> Apache Camel Committer
> Enterprise Architect, Program Manager, Open Source Integration specialist
> http://about.me/raulkripalani | http://www.linkedin.com/in/raulkripalani
> http://blog.raulkr.net | twitter: @raulvk <http://twitter.com/raulvk>
>
> On Mon, Feb 25, 2013 at 11:21 AM, Bilgin Ibryam <bibryam@gmail.com (mailto:bibryam@gmail.com)> wrote:
>
> > On 25 February 2013 10:45, Raul Kripalani <raul@evosent.com (mailto:raul@evosent.com)> wrote:
> >
> > > Well, it's not safe to execute the completions before the Vm consumer has
> > > finished processing, because the InputStream could be closed by the time
> > > the Vm consumer starts processing the exchange.
> > >
> > > I agree with Willem that the current implement is correct.
> >
> > I see, you both are right here, but what I actually do is exactly your
> > example below:
> >
> >
> > >
> > > The safest option is to convert the body to a String, and either use the
> > > threads DSL or inOnly(vm:...) to modify the exchange pattern onwards.
> >
> >
> > I've converted the body to a String, and then send the exchange to vm
> > endpoint using in only patters. At this point I expected to use VM endpoint
> > as an inmemory VM queue for exchanges, but cannot do that, because the
> > completions are also handled to that queue. Do I miss something here?
> >
> > Thanks for your help guys.
> > Bilgin
> >
> >
> > > Raúl.
> > > On 21 Feb 2013 17:13, "Bilgin Ibryam" <bibryam@gmail.com (mailto:bibryam@gmail.com)> wrote:
> > >
> > > > Hi,
> > > >
> > > > I have a route where I consume from a file endpoint and put them in VM
> > > > endpoint, so that another camel route from different camel app can
> > >
> > >
> > > process
> > > > it a little later asynchronously.
> > > >
> > > > The file endpoint has move=processed for removing the processed files,
> > > but
> > > > even the exchanges are delivered to VM endpoint, the exchange is not
> > > > considered completed, so the original file stays locked and cannot
> > >
> > >
> > > process
> > > > other files before I start the second camel application and process the
> > > > exchanges from VM endpoint.
> > > >
> > > > SEDA has waitForTaskToComplete option, which controls whether the
> > caller
> > > > should wait for the async task to complete or not before continuing.
> > >
> >
> >
> > But
> > > > even when waitForTaskToComplete=Never is set, the completions are
> > >
> >
> >
> > handed
> > > > over to VM queue, so it is not possible to complete the exchange.
> > > >
> > > > Is there a way to complete the exchange when it is put in VM queue? Or
> > > > there is a possibility for extending SEDA component with additional
> > >
> > >
> > > option
> > > > for not handling over the completions when waitForTaskToComplete=Never
> > >
> >
> >
> > is
> > > > set?
> > > >
> > > > WDYT
> > > > Bilgin
> > >
> >
>
Re: Seda component not handing over the completions
Posted by Raul Kripalani <ra...@evosent.com>.
>
> >
> > The safest option is to convert the body to a String, and either use the
> > threads DSL or inOnly(vm:...) to modify the exchange pattern onwards.
> >
> >
> I've converted the body to a String, and then send the exchange to vm
> endpoint using in only patters. At this point I expected to use VM endpoint
> as an inmemory VM queue for exchanges, but cannot do that, because the
> completions are also handled to that queue. Do I miss something here?
That sounds fishy. Functionally speaking, I'd expect an InOnly dispatch to
not hand over any completions, as we explicitly tell Camel we don't care
what happens thereafter by using inOnly(); but then again, on some level
it's sensible to provide the guarantee that whatever is behind the VM
endpoint won't find an incoherent state whenever it executes.
If this is really like this, it would make sense to add a
handoverCompletions option to the Seda endpoint.
For your immediate problem, did you try using the threads() DSL?
Regards,
*Raúl Kripalani*
Apache Camel Committer
Enterprise Architect, Program Manager, Open Source Integration specialist
http://about.me/raulkripalani | http://www.linkedin.com/in/raulkripalani
http://blog.raulkr.net | twitter: @raulvk <http://twitter.com/raulvk>
On Mon, Feb 25, 2013 at 11:21 AM, Bilgin Ibryam <bi...@gmail.com> wrote:
> On 25 February 2013 10:45, Raul Kripalani <ra...@evosent.com> wrote:
>
> > Well, it's not safe to execute the completions before the Vm consumer has
> > finished processing, because the InputStream could be closed by the time
> > the Vm consumer starts processing the exchange.
> >
> > I agree with Willem that the current implement is correct.
> >
>
> I see, you both are right here, but what I actually do is exactly your
> example below:
>
>
> >
> > The safest option is to convert the body to a String, and either use the
> > threads DSL or inOnly(vm:...) to modify the exchange pattern onwards.
> >
> >
> I've converted the body to a String, and then send the exchange to vm
> endpoint using in only patters. At this point I expected to use VM endpoint
> as an inmemory VM queue for exchanges, but cannot do that, because the
> completions are also handled to that queue. Do I miss something here?
>
> Thanks for your help guys.
> Bilgin
>
>
> > Raúl.
> > On 21 Feb 2013 17:13, "Bilgin Ibryam" <bi...@gmail.com> wrote:
> >
> > > Hi,
> > >
> > > I have a route where I consume from a file endpoint and put them in VM
> > > endpoint, so that another camel route from different camel app can
> > process
> > > it a little later asynchronously.
> > >
> > > The file endpoint has move=processed for removing the processed files,
> > but
> > > even the exchanges are delivered to VM endpoint, the exchange is not
> > > considered completed, so the original file stays locked and cannot
> > process
> > > other files before I start the second camel application and process the
> > > exchanges from VM endpoint.
> > >
> > > SEDA has waitForTaskToComplete option, which controls whether the
> caller
> > > should wait for the async task to complete or not before continuing.
> But
> > > even when waitForTaskToComplete=Never is set, the completions are
> handed
> > > over to VM queue, so it is not possible to complete the exchange.
> > >
> > > Is there a way to complete the exchange when it is put in VM queue? Or
> > > there is a possibility for extending SEDA component with additional
> > option
> > > for not handling over the completions when waitForTaskToComplete=Never
> is
> > > set?
> > >
> > > WDYT
> > > Bilgin
> > >
> >
>
Re: Seda component not handing over the completions
Posted by Bilgin Ibryam <bi...@gmail.com>.
On 25 February 2013 10:45, Raul Kripalani <ra...@evosent.com> wrote:
> Well, it's not safe to execute the completions before the Vm consumer has
> finished processing, because the InputStream could be closed by the time
> the Vm consumer starts processing the exchange.
>
> I agree with Willem that the current implement is correct.
>
I see, you both are right here, but what I actually do is exactly your
example below:
>
> The safest option is to convert the body to a String, and either use the
> threads DSL or inOnly(vm:...) to modify the exchange pattern onwards.
>
>
I've converted the body to a String, and then send the exchange to vm
endpoint using in only patters. At this point I expected to use VM endpoint
as an inmemory VM queue for exchanges, but cannot do that, because the
completions are also handled to that queue. Do I miss something here?
Thanks for your help guys.
Bilgin
> Raúl.
> On 21 Feb 2013 17:13, "Bilgin Ibryam" <bi...@gmail.com> wrote:
>
> > Hi,
> >
> > I have a route where I consume from a file endpoint and put them in VM
> > endpoint, so that another camel route from different camel app can
> process
> > it a little later asynchronously.
> >
> > The file endpoint has move=processed for removing the processed files,
> but
> > even the exchanges are delivered to VM endpoint, the exchange is not
> > considered completed, so the original file stays locked and cannot
> process
> > other files before I start the second camel application and process the
> > exchanges from VM endpoint.
> >
> > SEDA has waitForTaskToComplete option, which controls whether the caller
> > should wait for the async task to complete or not before continuing. But
> > even when waitForTaskToComplete=Never is set, the completions are handed
> > over to VM queue, so it is not possible to complete the exchange.
> >
> > Is there a way to complete the exchange when it is put in VM queue? Or
> > there is a possibility for extending SEDA component with additional
> option
> > for not handling over the completions when waitForTaskToComplete=Never is
> > set?
> >
> > WDYT
> > Bilgin
> >
>
Re: Seda component not handing over the completions
Posted by Raul Kripalani <ra...@evosent.com>.
Well, it's not safe to execute the completions before the Vm consumer has
finished processing, because the InputStream could be closed by the time
the Vm consumer starts processing the exchange.
I agree with Willem that the current implement is correct.
The safest option is to convert the body to a String, and either use the
threads DSL or inOnly(vm:...) to modify the exchange pattern onwards.
Raúl.
On 21 Feb 2013 17:13, "Bilgin Ibryam" <bi...@gmail.com> wrote:
> Hi,
>
> I have a route where I consume from a file endpoint and put them in VM
> endpoint, so that another camel route from different camel app can process
> it a little later asynchronously.
>
> The file endpoint has move=processed for removing the processed files, but
> even the exchanges are delivered to VM endpoint, the exchange is not
> considered completed, so the original file stays locked and cannot process
> other files before I start the second camel application and process the
> exchanges from VM endpoint.
>
> SEDA has waitForTaskToComplete option, which controls whether the caller
> should wait for the async task to complete or not before continuing. But
> even when waitForTaskToComplete=Never is set, the completions are handed
> over to VM queue, so it is not possible to complete the exchange.
>
> Is there a way to complete the exchange when it is put in VM queue? Or
> there is a possibility for extending SEDA component with additional option
> for not handling over the completions when waitForTaskToComplete=Never is
> set?
>
> WDYT
> Bilgin
>
Re: Seda component not handing over the completions
Posted by Willem jiang <wi...@gmail.com>.
Hi Bilgin,
I think current camel behavior is right. If the exchange is handled rightly, the file will not be moved.
If the further processor doesn't start, the file should not be moved at the same time.
It could be very helpful if we restart the camel context at this time, and we will lose the file which is not handled.
--
Willem Jiang
Red Hat, Inc.
FuseSource is now part of Red Hat
Web: http://www.fusesource.com | http://www.redhat.com
Blog: http://willemjiang.blogspot.com (http://willemjiang.blogspot.com/) (English)
http://jnn.iteye.com (http://jnn.javaeye.com/) (Chinese)
Twitter: willemjiang
Weibo: 姜宁willem
On Monday, February 25, 2013 at 6:22 AM, Bilgin Ibryam wrote:
> Hi Willem,
>
> If I set waitForTaskToComplete=Alway or
> waitForTaskToComplete=IfReplyExpectet SEDA will will not handover the
> completions, but it will also block and wait for the exchange to be
> processed and the it will complete the exchange. So in this case, if there
> is no seda consumer to process the exchange, the seda producer will be
> blocked and waiting for timeout.
>
> waitForTaskToComplete=Never is a little better, it is not blocking the seda
> producer, but it hands over the completions too. In that case the seda
> producer is not blocked and can process other exchanges, but this causes
> issues with some components (like the file consumer) which relies on the
> completions to be completed before processing the next exchange.
>
> What I propose is additional option, which will be used to control whether
> the completions to be handed over or not when waitForTaskToComplete=Never
> is set.
>
> Thanks
> Bilgin
>
>
> On 22 February 2013 08:06, Willem jiang <willem.jiang@gmail.com (mailto:willem.jiang@gmail.com)> wrote:
>
> > It looks like you need to set the option waitForTaskToComplete=Alway or
> > waitForTaskToComplete=IfReplyExpected to make sure the Seda component
> > doesn't hand over the completions.
> > I think current Camel feature just make sure the file isn't deleted until
> > the file is processed.
> >
> >
> > --
> > Willem Jiang
> >
> > Red Hat, Inc.
> > FuseSource is now part of Red Hat
> > Web: http://www.fusesource.com | http://www.redhat.com
> > Blog: http://willemjiang.blogspot.com (http://willemjiang.blogspot.com/)
> > (English)
> > http://jnn.iteye.com (http://jnn.javaeye.com/) (Chinese)
> > Twitter: willemjiang
> > Weibo: 姜宁willem
> >
> >
> >
> >
> >
> > On Friday, February 22, 2013 at 1:12 AM, Bilgin Ibryam wrote:
> >
> > > Hi,
> > >
> > > I have a route where I consume from a file endpoint and put them in VM
> > > endpoint, so that another camel route from different camel app can
> >
> >
> > process
> > > it a little later asynchronously.
> > >
> > > The file endpoint has move=processed for removing the processed files,
> > but
> > > even the exchanges are delivered to VM endpoint, the exchange is not
> > > considered completed, so the original file stays locked and cannot
> >
> >
> > process
> > > other files before I start the second camel application and process the
> > > exchanges from VM endpoint.
> > >
> > > SEDA has waitForTaskToComplete option, which controls whether the caller
> > > should wait for the async task to complete or not before continuing. But
> > > even when waitForTaskToComplete=Never is set, the completions are handed
> > > over to VM queue, so it is not possible to complete the exchange.
> > >
> > > Is there a way to complete the exchange when it is put in VM queue? Or
> > > there is a possibility for extending SEDA component with additional
> >
> >
> > option
> > > for not handling over the completions when waitForTaskToComplete=Never is
> > > set?
> > >
> > > WDYT
> > > Bilgin
> >
>
Re: Seda component not handing over the completions
Posted by Bilgin Ibryam <bi...@gmail.com>.
Hi Willem,
If I set waitForTaskToComplete=Alway or
waitForTaskToComplete=IfReplyExpectet SEDA will will not handover the
completions, but it will also block and wait for the exchange to be
processed and the it will complete the exchange. So in this case, if there
is no seda consumer to process the exchange, the seda producer will be
blocked and waiting for timeout.
waitForTaskToComplete=Never is a little better, it is not blocking the seda
producer, but it hands over the completions too. In that case the seda
producer is not blocked and can process other exchanges, but this causes
issues with some components (like the file consumer) which relies on the
completions to be completed before processing the next exchange.
What I propose is additional option, which will be used to control whether
the completions to be handed over or not when waitForTaskToComplete=Never
is set.
Thanks
Bilgin
On 22 February 2013 08:06, Willem jiang <wi...@gmail.com> wrote:
> It looks like you need to set the option waitForTaskToComplete=Alway or
> waitForTaskToComplete=IfReplyExpected to make sure the Seda component
> doesn't hand over the completions.
> I think current Camel feature just make sure the file isn't deleted until
> the file is processed.
>
>
> --
> Willem Jiang
>
> Red Hat, Inc.
> FuseSource is now part of Red Hat
> Web: http://www.fusesource.com | http://www.redhat.com
> Blog: http://willemjiang.blogspot.com (http://willemjiang.blogspot.com/)
> (English)
> http://jnn.iteye.com (http://jnn.javaeye.com/) (Chinese)
> Twitter: willemjiang
> Weibo: 姜宁willem
>
>
>
>
>
> On Friday, February 22, 2013 at 1:12 AM, Bilgin Ibryam wrote:
>
> > Hi,
> >
> > I have a route where I consume from a file endpoint and put them in VM
> > endpoint, so that another camel route from different camel app can
> process
> > it a little later asynchronously.
> >
> > The file endpoint has move=processed for removing the processed files,
> but
> > even the exchanges are delivered to VM endpoint, the exchange is not
> > considered completed, so the original file stays locked and cannot
> process
> > other files before I start the second camel application and process the
> > exchanges from VM endpoint.
> >
> > SEDA has waitForTaskToComplete option, which controls whether the caller
> > should wait for the async task to complete or not before continuing. But
> > even when waitForTaskToComplete=Never is set, the completions are handed
> > over to VM queue, so it is not possible to complete the exchange.
> >
> > Is there a way to complete the exchange when it is put in VM queue? Or
> > there is a possibility for extending SEDA component with additional
> option
> > for not handling over the completions when waitForTaskToComplete=Never is
> > set?
> >
> > WDYT
> > Bilgin
>
>
>
>
Re: Seda component not handing over the completions
Posted by Willem jiang <wi...@gmail.com>.
It looks like you need to set the option waitForTaskToComplete=Alway or waitForTaskToComplete=IfReplyExpected to make sure the Seda component doesn't hand over the completions.
I think current Camel feature just make sure the file isn't deleted until the file is processed.
--
Willem Jiang
Red Hat, Inc.
FuseSource is now part of Red Hat
Web: http://www.fusesource.com | http://www.redhat.com
Blog: http://willemjiang.blogspot.com (http://willemjiang.blogspot.com/) (English)
http://jnn.iteye.com (http://jnn.javaeye.com/) (Chinese)
Twitter: willemjiang
Weibo: 姜宁willem
On Friday, February 22, 2013 at 1:12 AM, Bilgin Ibryam wrote:
> Hi,
>
> I have a route where I consume from a file endpoint and put them in VM
> endpoint, so that another camel route from different camel app can process
> it a little later asynchronously.
>
> The file endpoint has move=processed for removing the processed files, but
> even the exchanges are delivered to VM endpoint, the exchange is not
> considered completed, so the original file stays locked and cannot process
> other files before I start the second camel application and process the
> exchanges from VM endpoint.
>
> SEDA has waitForTaskToComplete option, which controls whether the caller
> should wait for the async task to complete or not before continuing. But
> even when waitForTaskToComplete=Never is set, the completions are handed
> over to VM queue, so it is not possible to complete the exchange.
>
> Is there a way to complete the exchange when it is put in VM queue? Or
> there is a possibility for extending SEDA component with additional option
> for not handling over the completions when waitForTaskToComplete=Never is
> set?
>
> WDYT
> Bilgin