You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by David Hoffer <dh...@gmail.com> on 2016/07/12 12:53:55 UTC
Re: How to retain exchange headers through RecipientList destination
As I look into this more, I think I see what the problem is. I haven't
seen any good docs on the exchange lifecycle but from the perspective of
the original RecipientList (or RoutingSlip) since it delivered the file to
the destination its job is done and it's a new exchange that handles it
from there. Below is an example of the route/endpoint that handles the
file once it gets to its destination. It is in the backupProcessor
processor that I need to have the metadata about the file however since its
a new exchange the data is lost at this point.
So it seems what I need is a different 'from' URI in the example below.
This URI has to retain the metadata so the backupProcessor can use it. So
some options it seems to be would be to extend the file URI so it writes
both the normal file but also a separate metadata file. Then
my backupProcessor would just read both files. Or is there a different URI
that can store the file and metadata? We have a 200ms delay here so the
file/metadata are only here for a fraction of a second.
Or another idea is that the from URI would become the backupProcessor as
well. E.g. My app has no need for the temporary file URI shown below, that
was just to satisfy Camel. What we really want is the file logic
in backupProcessor but it has to know about the Camel headers/metadata
somehow.
<routeContext id="backup-endpoint"
xmlns="http://camel.apache.org/schema/spring">
<route id="to_backup" autoStartup="{{gp.camel.backup.isConfigured}}">
<from uri="file://{{gangplank.home}}/work/toBackup/?delete=true&moveFailed=.error&filter=#partialFilter&delay=200"/>
<process ref="backupProcessor" />
</route>
</routeContext>
On Mon, Jul 11, 2016 at 2:05 PM, David Hoffer <dh...@gmail.com> wrote:
> I'm back now trying to find a solution to this problem.
>
> First I changed the RoutingSlip to be a RecipientList, not sure that is
> significant but wanted to mention it.
>
> Let me try to better explain what this application does. We use Camel to
> route/transfer files from one location to a dynamic list of other
> locations. It's dynamic in the sense that we have a GUI app that enables
> destinations. E.g. any input source can be routed to any number of
> destination locations. We use the RecipientList to specify the
> dynamic/multiple destinations.
>
> So for every source folder we use a custom CamelContextAware bean to be
> the RecipientList (actually it contains the RecipientList instance). This
> bean (routeManager in example below) has a route(Exchange exchange) method
> that is called in each of the source routes. We define all routes in XML,
> e.g.
>
> <route id="from_externalDropbox"
> autoStartup="{{gp.camel.externalDropbox.isConfigured}}">
> <from
> uri="file://{{gangplank.data}}/../fromExternalDropbox/?delete=true&exclusiveReadLockStrategy=#fileLock&delay=500"/>
>
> <convertBodyTo type="byte[]"/>
>
> <bean ref="routeManager" method="route(*)"/>
> </route>
>
> So in the CamelContextAware bean's route method we set the exchange's
> getFromRouteId() value as a property and/or header on the exchange and/or
> input message. E.g.
>
> exchange.getIn().setHeader(GP_DYNAMIC_FROM_ID, fromId);
> exchange.setProperty(GP_DYNAMIC_FROM_ID, fromId);
>
>
> Then in one of the destination routes we use a custom Processor to handle
> the file being routed. It's in this custom processor that I need to
> extract the FromRouteId that was stored in the exchange property and/or
> input message header. However this value is always null here, and that is
> the problem. E.g.
>
> String fromIdHeader = (String)
> exchange.getIn().getHeader(GP_DYNAMIC_FROM_ID);
> String fromIdProperty = (String) exchange.getProperty(GP_DYNAMIC_FROM_ID);
>
> I need to propagate this metadata about the file to the Processor but
> Camel is removing the headers & properties and I don't know why.
> (Both fromIdHeader & fromIdProperty are null).
>
> What is the right way to pass metadata between related routes?
>
> Note I printed out the ExchangeId in both the source and destination and I
> see the trailing 2 digits are different.
>
> RouteManagerImpl ExchangeId=ID-ISS206486-54382-1468259188590-0-24
> CopyFileProcessor ExchangeId=ID-ISS206486-54382-1468259188590-0-32
>
> Also we don't use exchange.getOut() at all so I don't think that is why
> headers/properties are getting lost.
>
> -Dave
>
>
>
>
>
>
> On Sun, Oct 4, 2015 at 11:36 PM, Claus Ibsen <cl...@gmail.com>
> wrote:
>
>> Hi
>>
>> Sorry I think you confuse us. You talk about routing slip and show
>> some code of a route that starts from a file, and without any routing
>> slip.
>>
>> Also see this FAQ
>>
>> http://camel.apache.org/why-do-my-message-lose-its-headers-during-routing.html
>>
>> On Sun, Oct 4, 2015 at 4:20 PM, David Hoffer <dh...@gmail.com> wrote:
>> > I'm not trying to add any headers to the file when reading and
>> > writing...I'm just adding them to the Camel Exchange so that when the
>> file
>> > is written in my Processor I have some 'metadata' about the file. I
>> need
>> > the 'metadata' so I know where to write the file. I get the bytes to
>> write
>> > from the exchange it seems only natural to get the 'metadata' from the
>> same
>> > exchange. Why does Camel throw the headers away? I'm pretty new to
>> > Camel...not sure how to add a custom message transformation...any
>> examples
>> > of that?
>> >
>> > Btw, I have no need for the header information after I write the file
>> in my
>> > processor...only before.
>> >
>> > -Dave
>> >
>> > On Sun, Oct 4, 2015 at 12:15 AM, Claus Ibsen <cl...@gmail.com>
>> wrote:
>> >
>> >> Hi
>> >>
>> >> Ah yeah read from a file and a file does not include any headers when
>> >> you write to a file. The body is the content of the file. And all
>> >> headers is not stored. You need to do some kind of custom message
>> >> transformation if you want to include headers, and also when you read
>> >> the file. eg where should the headers be stored in the file?
>> >>
>> >> On Sat, Oct 3, 2015 at 2:33 PM, David Hoffer <dh...@gmail.com>
>> wrote:
>> >> > We add the headers where we create and/or call process on the routing
>> >> > slip...
>> >> >
>> >> > exchange.getIn().setHeader("customHeader1", somedata1);
>> >> > exchange.getIn().setHeader("customHeader2", somedata2);
>> >> > exchange.getIn().setHeader(routingSlipHeaderName, toUris);
>> >> > routingSlip.process(exchange);
>> >> >
>> >> > But then when when the message is handled by a processor bean we try
>> to
>> >> get
>> >> > the header back via
>> >> >
>> >> > String data1 = (String) exchange.getIn().getHeader("customHeader1");
>> >> >
>> >> > This always returns null and it removed the 'routingSlipHeaderName'
>> >> header
>> >> > too that used used by the routing slip. (Its okay that it removed
>> this
>> >> > one...but need the others.)
>> >> >
>> >> > Here is the XML config for the receiving route. The
>> CopyFileProcessor is
>> >> > the one that does not get the custom headers.
>> >> >
>> >> > <bean id="backupProcessor"
>> >> > class="com.issinc.jms.gangplank.impl.CopyFileProcessor">
>> >> > <property name="path" value="${gp.camel.backup.dir}"/>
>> >> > </bean>
>> >> >
>> >> > <routeContext id="backup-endpoint"
>> >> > xmlns="http://camel.apache.org/schema/spring">
>> >> > <route id="to_backup"
>> autoStartup="{{gp.camel.backup.isConfigured}}">
>> >> > <from
>> >>
>> uri="file://{{gangplank.data}}/toBackup/?delete=true&moveFailed=.error&filter=#partialFilter&delay=200"/>
>> >> > <process ref="backupProcessor" />
>> >> > <onException redeliveryPolicyRef="redeliveryProfile">
>> >> > <exception>java.lang.Exception</exception>
>> >> > <handled>
>> >> > <constant>false</constant>
>> >> > </handled>
>> >> > </onException>
>> >> > </route>
>> >> > </routeContext>
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > -Dave
>> >> >
>> >> > On Sat, Oct 3, 2015 at 2:27 AM, Claus Ibsen <cl...@gmail.com>
>> >> wrote:
>> >> >
>> >> >> Where do you set those custom headers?
>> >> >>
>> >> >> On Fri, Oct 2, 2015 at 8:20 PM, David Hoffer <dh...@gmail.com>
>> >> wrote:
>> >> >> > I'm using a RoutingSlip to route files but all custom headers set
>> on
>> >> the
>> >> >> > Exchange are discarded. I assume that's because Camel assumes
>> custom
>> >> >> > headers are not useful for file messages but I need a way to pass
>> >> source
>> >> >> > information from the RoutingSlip to the file Processors.
>> >> >> >
>> >> >> > How can I do this? Is there a way I can override the default
>> >> behavior of
>> >> >> > removing all custom headers?
>> >> >> >
>> >> >> > -Dave
>> >> >>
>> >> >>
>> >> >>
>> >> >> --
>> >> >> Claus Ibsen
>> >> >> -----------------
>> >> >> http://davsclaus.com @davsclaus
>> >> >> Camel in Action 2nd edition:
>> >> >> https://www.manning.com/books/camel-in-action-second-edition
>> >> >>
>> >>
>> >>
>> >>
>> >> --
>> >> Claus Ibsen
>> >> -----------------
>> >> http://davsclaus.com @davsclaus
>> >> Camel in Action 2nd edition:
>> >> https://www.manning.com/books/camel-in-action-second-edition
>> >>
>>
>>
>>
>> --
>> Claus Ibsen
>> -----------------
>> http://davsclaus.com @davsclaus
>> Camel in Action 2nd edition:
>> https://www.manning.com/books/camel-in-action-second-edition
>>
>
>