You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by Scott Parkerson <sc...@parkerson.net> on 2013/08/05 16:58:50 UTC
Unit of Work Scope and direct/SEDA/vm components
Hi, fellow Camel riders.
I was wondering where Camel's notion of UnitOfWork begins and ends with
respect to processing a route or set of routes.
For example, I have a route that starts with a SFTP poller that consumes
files off of a server. The contents of these files are then handed to
another route via the direct component, which is responsible for filtering
and routing the data to other processors that are on multiple camel
contexts. As such, I have been using the vm component to pass that data to
the appropriate processing route.
What's strange is that it seems like the files are not marked as processed
(i.e. the original SFTP component does not move the files to the processed
directory until the message is completely handled by the second route. This
is a problem because the second route actually splits and aggregates data
in the file, and, as such, the data may not be released from the aggregator
until later. Furthermore, the SFTP connection in the body may not be valid
anymore, and even if it is, it seems like splitting the data also splits
the notion that that data came from a SFTP consumer, which makes lots of
stack traces.
What I want to know is: how can I make sure that the file on the SFTP
server is marked processed before it goes off to the handling route? Do I
need to send the data to a file or an JMS/ActiveMQ queue first?
Thanks,
Scott Parkerson
Re: Unit of Work Scope and direct/SEDA/vm components
Posted by Christian Posta <ch...@gmail.com>.
Could you also do the following?
from(...)......to("vm:name").process(new Processor() {
public void process(Exchange exchange) {
exchange.getUnitOfWork().done();
}
});
On Tue, Aug 6, 2013 at 7:11 AM, Scott Parkerson <sc...@parkerson.net> wrote:
> Bilgin,
>
> Thank you -- that's what I expected, so I'll give using a JMS queue a try.
>
> --sgp
>
>
> On Mon, Aug 5, 2013 at 2:45 PM, Bilgin Ibryam <bi...@gmail.com> wrote:
>
> > Hi,
> > The file component uses onCompletion for moving files after the exchange
> > has completed. As long as the completions are passed with the exchange
> the
> > files wont move. The VM component passes the completions to the next
> camel
> > context so the files are not moved... You have to use a queue or
> something
> > like wiretap to stop completions.
> >
> > HTH
> > Bilgin
> >
> > Sent from mobile
> > On 5 Aug 2013 18:07, "Scott Parkerson" <sc...@parkerson.net> wrote:
> >
> > > Hi, fellow Camel riders.
> > >
> > > I was wondering where Camel's notion of UnitOfWork begins and ends with
> > > respect to processing a route or set of routes.
> > >
> > > For example, I have a route that starts with a SFTP poller that
> consumes
> > > files off of a server. The contents of these files are then handed to
> > > another route via the direct component, which is responsible for
> > filtering
> > > and routing the data to other processors that are on multiple camel
> > > contexts. As such, I have been using the vm component to pass that data
> > to
> > > the appropriate processing route.
> > >
> > > What's strange is that it seems like the files are not marked as
> > processed
> > > (i.e. the original SFTP component does not move the files to the
> > processed
> > > directory until the message is completely handled by the second route.
> > This
> > > is a problem because the second route actually splits and aggregates
> data
> > > in the file, and, as such, the data may not be released from the
> > aggregator
> > > until later. Furthermore, the SFTP connection in the body may not be
> > valid
> > > anymore, and even if it is, it seems like splitting the data also
> splits
> > > the notion that that data came from a SFTP consumer, which makes lots
> of
> > > stack traces.
> > >
> > > What I want to know is: how can I make sure that the file on the SFTP
> > > server is marked processed before it goes off to the handling route?
> Do I
> > > need to send the data to a file or an JMS/ActiveMQ queue first?
> > >
> > > Thanks,
> > > Scott Parkerson
> > >
> >
>
--
*Christian Posta*
http://www.christianposta.com/blog
twitter: @christianposta
Re: Unit of Work Scope and direct/SEDA/vm components
Posted by Scott Parkerson <sc...@parkerson.net>.
Bilgin,
Thank you -- that's what I expected, so I'll give using a JMS queue a try.
--sgp
On Mon, Aug 5, 2013 at 2:45 PM, Bilgin Ibryam <bi...@gmail.com> wrote:
> Hi,
> The file component uses onCompletion for moving files after the exchange
> has completed. As long as the completions are passed with the exchange the
> files wont move. The VM component passes the completions to the next camel
> context so the files are not moved... You have to use a queue or something
> like wiretap to stop completions.
>
> HTH
> Bilgin
>
> Sent from mobile
> On 5 Aug 2013 18:07, "Scott Parkerson" <sc...@parkerson.net> wrote:
>
> > Hi, fellow Camel riders.
> >
> > I was wondering where Camel's notion of UnitOfWork begins and ends with
> > respect to processing a route or set of routes.
> >
> > For example, I have a route that starts with a SFTP poller that consumes
> > files off of a server. The contents of these files are then handed to
> > another route via the direct component, which is responsible for
> filtering
> > and routing the data to other processors that are on multiple camel
> > contexts. As such, I have been using the vm component to pass that data
> to
> > the appropriate processing route.
> >
> > What's strange is that it seems like the files are not marked as
> processed
> > (i.e. the original SFTP component does not move the files to the
> processed
> > directory until the message is completely handled by the second route.
> This
> > is a problem because the second route actually splits and aggregates data
> > in the file, and, as such, the data may not be released from the
> aggregator
> > until later. Furthermore, the SFTP connection in the body may not be
> valid
> > anymore, and even if it is, it seems like splitting the data also splits
> > the notion that that data came from a SFTP consumer, which makes lots of
> > stack traces.
> >
> > What I want to know is: how can I make sure that the file on the SFTP
> > server is marked processed before it goes off to the handling route? Do I
> > need to send the data to a file or an JMS/ActiveMQ queue first?
> >
> > Thanks,
> > Scott Parkerson
> >
>
Re: Unit of Work Scope and direct/SEDA/vm components
Posted by Bilgin Ibryam <bi...@gmail.com>.
Hi,
The file component uses onCompletion for moving files after the exchange
has completed. As long as the completions are passed with the exchange the
files wont move. The VM component passes the completions to the next camel
context so the files are not moved... You have to use a queue or something
like wiretap to stop completions.
HTH
Bilgin
Sent from mobile
On 5 Aug 2013 18:07, "Scott Parkerson" <sc...@parkerson.net> wrote:
> Hi, fellow Camel riders.
>
> I was wondering where Camel's notion of UnitOfWork begins and ends with
> respect to processing a route or set of routes.
>
> For example, I have a route that starts with a SFTP poller that consumes
> files off of a server. The contents of these files are then handed to
> another route via the direct component, which is responsible for filtering
> and routing the data to other processors that are on multiple camel
> contexts. As such, I have been using the vm component to pass that data to
> the appropriate processing route.
>
> What's strange is that it seems like the files are not marked as processed
> (i.e. the original SFTP component does not move the files to the processed
> directory until the message is completely handled by the second route. This
> is a problem because the second route actually splits and aggregates data
> in the file, and, as such, the data may not be released from the aggregator
> until later. Furthermore, the SFTP connection in the body may not be valid
> anymore, and even if it is, it seems like splitting the data also splits
> the notion that that data came from a SFTP consumer, which makes lots of
> stack traces.
>
> What I want to know is: how can I make sure that the file on the SFTP
> server is marked processed before it goes off to the handling route? Do I
> need to send the data to a file or an JMS/ActiveMQ queue first?
>
> Thanks,
> Scott Parkerson
>