You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by Andy Bourke <an...@pcmsgroup.com> on 2010/01/22 13:20:18 UTC
Renamer failing to rename 'From' File when using multiple endpoints
via Multicast
Hi all,
I am new to camel and have hit a problem that I hope someone can help me
with.
I have a route that looks for a file in a directory => unmarshals the file
using <CSV> => transforms the output from the unmarshal (which is
List<List<String>>) to a List of data objetcs => passes the List of data
objects to a processor which does the intelligent mapping of the input data
to the output which is a List of Objects => passes the List of Objects to a
<multicast> pipeline who's endpoints traverse the List of Objects and
extract the data, that they are interested in, formatting the output in to a
List of Map objects so that the <CSV> marshaller can marshal to a named
file.
This is the route:
<route id="iq">
<from uri="file:/data/iq/inbound/"/>
<unmarshal>
<csv id="pipeCsvDataFormat"/>
</unmarshal>
<bean ref="iqTransform" method="doTransform"/>
<to uri="bean:iqProcessor?method=process"/>
<multicast parallelProcessing="true">
<pipeline>
<to uri="bean:formatOutput?method=formatHeader"/>
<marshal>
<csv />
</marshal>
<to
uri="file:/data/iq/outbound/?fileName=Txn_Header.txt"/>
</pipeline>
<pipeline>
<to uri="bean:formatOutput?method=formatLineHeader"/>
<marshal>
<csv />
</marshal>
<to
uri="file:/data/iq/outbound/?fileName=Line_Header.txt"/>
</pipeline>
</multicast>
</route>
As expected, I get a correctly formatted file named Txn_Header.txt and a
file named Line_Header.txt in the /outbound directory. So the routing seems
to have worked fine.
The problem is that the original file does not get renamed in to the .camel
subdirectory and so the file gets processed again and again.
If I remove the second endpoint in the multicast so that only the
Txn_Header.txt file is produced then the original file does get renamed and
the route ends successfully.
Can someone help me understand what I have done wrong please?
--
View this message in context: http://old.nabble.com/Renamer-failing-to-rename-%27From%27-File-when-using-multiple-endpoints-via-Multicast-tp27272407p27272407.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Renamer failing to rename 'From' File when using multiple
endpoints via Multicast
Posted by Andy Bourke <an...@pcmsgroup.com>.
> Which OS are you running? Could you write that down in the JIRA as well.
I'm running Windows Vista Business Service Pack 2, 32 bit. I have put this
in the Jira (#CAMEL-2394)
https://issues.apache.org/activemq/browse/CAMEL-2394
--
View this message in context: http://old.nabble.com/Renamer-failing-to-rename-%27From%27-File-when-using-multiple-endpoints-via-Multicast-tp27272407p27275060.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Renamer failing to rename 'From' File when using multiple
endpoints via Multicast
Posted by Claus Ibsen <cl...@gmail.com>.
On Fri, Jan 22, 2010 at 3:54 PM, Andy Bourke <an...@pcmsgroup.com> wrote:
>
>
>> To use the log component you should just do
>> <to uri="log:fooOrWhatYouLike"/>
>
>> http://camel.apache.org/log.html
>
> Ok, I've got it now. However, sending the output to the log rather than the
> file does not stop the rename issue. If it helps the actual error from the
> trace log is:
>
> org.apache.camel.component.file.GenericFileOperationFailedException: Cannot
> rename file: GenericFile[extract3.txt] to: GenericFile[\\.camel]
>
Which OS are you running? Could you write that down in the JIRA as well.
>
>> How do I create a ticket in Jira? I have never used it.
>>
>
>> No problem we will get you up to speed. JIRA is used a lot in the OS
>> world so you may as well start learning
>> it to be able to submit bugs or requests for new features etc.
>
>> Support page here with link to Issue Tracker = JIRA
>> http://camel.apache.org/support.html
>
>> You create an account and then there is a "create new issue button" on
>> the "menu bar".
>
> Ok, I'll get right on it.
>
>
>>
>>
>> Claus Ibsen-2 wrote:
>>>
>>> On Fri, Jan 22, 2010 at 2:47 PM, Claus Ibsen <cl...@gmail.com>
>>> wrote:
>>>> On Fri, Jan 22, 2010 at 2:41 PM, Andy Bourke <an...@pcmsgroup.com>
>>>> wrote:
>>>>>
>>>>> If I understand it right, the multicast allows the same payload to be
>>>>> processed by multiple end points? If I remove the multicast then won't
>>>>> the
>>>>> second enpoint (in the now non-multicast pipeline) try to process the
>>>>> output
>>>>> from the first one?
>>>>>
>>>>
>>>> Correct but just to see if the file gets renamed.
>>>> Also try the multicast without parallel as well.
>>>>
>>>
>>> Could you also try NOT to write to file in the multicast. Instead just
>>> do a "log:foo".
>>>
>>> I suspect it interferes with the original file.
>>> Which may also be a good time to spend the day coding to implement a
>>> stack based file context
>>> so they wont interfere each other. Will also fix the file -> pollEnrich
>>> as
>>> well.
>>>
>>> After your findings could you create a ticket in JIRA? And a link back
>>> to this forum, eg find it from nabble.
>>> After coming back from a business trip I should have to to work on this
>>> issue.
>>>
>>>
>>>>
>>>>>
>>>>> Claus Ibsen-2 wrote:
>>>>>>
>>>>>> On Fri, Jan 22, 2010 at 1:20 PM, Andy Bourke
>>>>>> <an...@pcmsgroup.com>
>>>>>> wrote:
>>>>>>>
>>>>>>> Hi all,
>>>>>>>
>>>>>>> I am new to camel and have hit a problem that I hope someone can help
>>>>>>> me
>>>>>>> with.
>>>>>>>
>>>>>>> I have a route that looks for a file in a directory => unmarshals the
>>>>>>> file
>>>>>>> using <CSV> => transforms the output from the unmarshal (which is
>>>>>>> List<List<String>>) to a List of data objetcs => passes the List of
>>>>>>> data
>>>>>>> objects to a processor which does the intelligent mapping of the
>>>>>>> input
>>>>>>> data
>>>>>>> to the output which is a List of Objects => passes the List of
>>>>>>> Objects
>>>>>>> to
>>>>>>> a
>>>>>>> <multicast> pipeline who's endpoints traverse the List of Objects and
>>>>>>> extract the data, that they are interested in, formatting the output
>>>>>>> in
>>>>>>> to a
>>>>>>> List of Map objects so that the <CSV> marshaller can marshal to a
>>>>>>> named
>>>>>>> file.
>>>>>>>
>>>>>>> This is the route:
>>>>>>>
>>>>>>> <route id="iq">
>>>>>>> <from uri="file:/data/iq/inbound/"/>
>>>>>>> <unmarshal>
>>>>>>> <csv id="pipeCsvDataFormat"/>
>>>>>>> </unmarshal>
>>>>>>> <bean ref="iqTransform" method="doTransform"/>
>>>>>>> <to uri="bean:iqProcessor?method=process"/>
>>>>>>> <multicast parallelProcessing="true">
>>>>>>> <pipeline>
>>>>>>> <to uri="bean:formatOutput?method=formatHeader"/>
>>>>>>> <marshal>
>>>>>>> <csv />
>>>>>>> </marshal>
>>>>>>> <to
>>>>>>> uri="file:/data/iq/outbound/?fileName=Txn_Header.txt"/>
>>>>>>> </pipeline>
>>>>>>> <pipeline>
>>>>>>> <to
>>>>>>> uri="bean:formatOutput?method=formatLineHeader"/>
>>>>>>> <marshal>
>>>>>>> <csv />
>>>>>>> </marshal>
>>>>>>> <to
>>>>>>> uri="file:/data/iq/outbound/?fileName=Line_Header.txt"/>
>>>>>>> </pipeline>
>>>>>>> </multicast>
>>>>>>> </route>
>>>>>>>
>>>>>>>
>>>>>>> As expected, I get a correctly formatted file named Txn_Header.txt
>>>>>>> and
>>>>>>> a
>>>>>>> file named Line_Header.txt in the /outbound directory. So the routing
>>>>>>> seems
>>>>>>> to have worked fine.
>>>>>>>
>>>>>>> The problem is that the original file does not get renamed in to the
>>>>>>> .camel
>>>>>>> subdirectory and so the file gets processed again and again.
>>>>>>>
>>>>>>> If I remove the second endpoint in the multicast so that only the
>>>>>>> Txn_Header.txt file is produced then the original file does get
>>>>>>> renamed
>>>>>>> and
>>>>>>> the route ends successfully.
>>>>>>>
>>>>>>
>>>>>> And with no mulitcast have you tried that?
>>>>>>
>>>>>>
>>>>>>> Can someone help me understand what I have done wrong please?
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> View this message in context:
>>>>>>> http://old.nabble.com/Renamer-failing-to-rename-%27From%27-File-when-using-multiple-endpoints-via-Multicast-tp27272407p27272407.html
>>>>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Claus Ibsen
>>>>>> Apache Camel Committer
>>>>>>
>>>>>> Author of Camel in Action: http://www.manning.com/ibsen/
>>>>>> Open Source Integration: http://fusesource.com
>>>>>> Blog: http://davsclaus.blogspot.com/
>>>>>> Twitter: http://twitter.com/davsclaus
>>>>>>
>>>>>>
>>>>>
>>>>> --
>>>>> View this message in context:
>>>>> http://old.nabble.com/Renamer-failing-to-rename-%27From%27-File-when-using-multiple-endpoints-via-Multicast-tp27272407p27273273.html
>>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Claus Ibsen
>>>> Apache Camel Committer
>>>>
>>>> Author of Camel in Action: http://www.manning.com/ibsen/
>>>> Open Source Integration: http://fusesource.com
>>>> Blog: http://davsclaus.blogspot.com/
>>>> Twitter: http://twitter.com/davsclaus
>>>>
>>>
>>>
>>>
>>> --
>>> Claus Ibsen
>>> Apache Camel Committer
>>>
>>> Author of Camel in Action: http://www.manning.com/ibsen/
>>> Open Source Integration: http://fusesource.com
>>> Blog: http://davsclaus.blogspot.com/
>>> Twitter: http://twitter.com/davsclaus
>>>
>>>
>>
>> --
>> View this message in context:
>> http://old.nabble.com/Renamer-failing-to-rename-%27From%27-File-when-using-multiple-endpoints-via-Multicast-tp27272407p27273955.html
>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>
>>
>
>
>
> --
> Claus Ibsen
> Apache Camel Committer
>
> Author of Camel in Action: http://www.manning.com/ibsen/
> Open Source Integration: http://fusesource.com
> Blog: http://davsclaus.blogspot.com/
> Twitter: http://twitter.com/davsclaus
>
>
>
> --
> View this message in context: http://old.nabble.com/Renamer-failing-to-rename-%27From%27-File-when-using-multiple-endpoints-via-Multicast-tp27272407p27274388.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
>
--
Claus Ibsen
Apache Camel Committer
Author of Camel in Action: http://www.manning.com/ibsen/
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus
Re: Renamer failing to rename 'From' File when using multiple
endpoints via Multicast
Posted by Andy Bourke <an...@pcmsgroup.com>.
> To use the log component you should just do
> <to uri="log:fooOrWhatYouLike"/>
> http://camel.apache.org/log.html
Ok, I've got it now. However, sending the output to the log rather than the
file does not stop the rename issue. If it helps the actual error from the
trace log is:
org.apache.camel.component.file.GenericFileOperationFailedException: Cannot
rename file: GenericFile[extract3.txt] to: GenericFile[\\.camel]
> How do I create a ticket in Jira? I have never used it.
>
> No problem we will get you up to speed. JIRA is used a lot in the OS
> world so you may as well start learning
> it to be able to submit bugs or requests for new features etc.
> Support page here with link to Issue Tracker = JIRA
> http://camel.apache.org/support.html
> You create an account and then there is a "create new issue button" on
> the "menu bar".
Ok, I'll get right on it.
>
>
> Claus Ibsen-2 wrote:
>>
>> On Fri, Jan 22, 2010 at 2:47 PM, Claus Ibsen <cl...@gmail.com>
>> wrote:
>>> On Fri, Jan 22, 2010 at 2:41 PM, Andy Bourke <an...@pcmsgroup.com>
>>> wrote:
>>>>
>>>> If I understand it right, the multicast allows the same payload to be
>>>> processed by multiple end points? If I remove the multicast then won't
>>>> the
>>>> second enpoint (in the now non-multicast pipeline) try to process the
>>>> output
>>>> from the first one?
>>>>
>>>
>>> Correct but just to see if the file gets renamed.
>>> Also try the multicast without parallel as well.
>>>
>>
>> Could you also try NOT to write to file in the multicast. Instead just
>> do a "log:foo".
>>
>> I suspect it interferes with the original file.
>> Which may also be a good time to spend the day coding to implement a
>> stack based file context
>> so they wont interfere each other. Will also fix the file -> pollEnrich
>> as
>> well.
>>
>> After your findings could you create a ticket in JIRA? And a link back
>> to this forum, eg find it from nabble.
>> After coming back from a business trip I should have to to work on this
>> issue.
>>
>>
>>>
>>>>
>>>> Claus Ibsen-2 wrote:
>>>>>
>>>>> On Fri, Jan 22, 2010 at 1:20 PM, Andy Bourke
>>>>> <an...@pcmsgroup.com>
>>>>> wrote:
>>>>>>
>>>>>> Hi all,
>>>>>>
>>>>>> I am new to camel and have hit a problem that I hope someone can help
>>>>>> me
>>>>>> with.
>>>>>>
>>>>>> I have a route that looks for a file in a directory => unmarshals the
>>>>>> file
>>>>>> using <CSV> => transforms the output from the unmarshal (which is
>>>>>> List<List<String>>) to a List of data objetcs => passes the List of
>>>>>> data
>>>>>> objects to a processor which does the intelligent mapping of the
>>>>>> input
>>>>>> data
>>>>>> to the output which is a List of Objects => passes the List of
>>>>>> Objects
>>>>>> to
>>>>>> a
>>>>>> <multicast> pipeline who's endpoints traverse the List of Objects and
>>>>>> extract the data, that they are interested in, formatting the output
>>>>>> in
>>>>>> to a
>>>>>> List of Map objects so that the <CSV> marshaller can marshal to a
>>>>>> named
>>>>>> file.
>>>>>>
>>>>>> This is the route:
>>>>>>
>>>>>> <route id="iq">
>>>>>> <from uri="file:/data/iq/inbound/"/>
>>>>>> <unmarshal>
>>>>>> <csv id="pipeCsvDataFormat"/>
>>>>>> </unmarshal>
>>>>>> <bean ref="iqTransform" method="doTransform"/>
>>>>>> <to uri="bean:iqProcessor?method=process"/>
>>>>>> <multicast parallelProcessing="true">
>>>>>> <pipeline>
>>>>>> <to uri="bean:formatOutput?method=formatHeader"/>
>>>>>> <marshal>
>>>>>> <csv />
>>>>>> </marshal>
>>>>>> <to
>>>>>> uri="file:/data/iq/outbound/?fileName=Txn_Header.txt"/>
>>>>>> </pipeline>
>>>>>> <pipeline>
>>>>>> <to
>>>>>> uri="bean:formatOutput?method=formatLineHeader"/>
>>>>>> <marshal>
>>>>>> <csv />
>>>>>> </marshal>
>>>>>> <to
>>>>>> uri="file:/data/iq/outbound/?fileName=Line_Header.txt"/>
>>>>>> </pipeline>
>>>>>> </multicast>
>>>>>> </route>
>>>>>>
>>>>>>
>>>>>> As expected, I get a correctly formatted file named Txn_Header.txt
>>>>>> and
>>>>>> a
>>>>>> file named Line_Header.txt in the /outbound directory. So the routing
>>>>>> seems
>>>>>> to have worked fine.
>>>>>>
>>>>>> The problem is that the original file does not get renamed in to the
>>>>>> .camel
>>>>>> subdirectory and so the file gets processed again and again.
>>>>>>
>>>>>> If I remove the second endpoint in the multicast so that only the
>>>>>> Txn_Header.txt file is produced then the original file does get
>>>>>> renamed
>>>>>> and
>>>>>> the route ends successfully.
>>>>>>
>>>>>
>>>>> And with no mulitcast have you tried that?
>>>>>
>>>>>
>>>>>> Can someone help me understand what I have done wrong please?
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> View this message in context:
>>>>>> http://old.nabble.com/Renamer-failing-to-rename-%27From%27-File-when-using-multiple-endpoints-via-Multicast-tp27272407p27272407.html
>>>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Claus Ibsen
>>>>> Apache Camel Committer
>>>>>
>>>>> Author of Camel in Action: http://www.manning.com/ibsen/
>>>>> Open Source Integration: http://fusesource.com
>>>>> Blog: http://davsclaus.blogspot.com/
>>>>> Twitter: http://twitter.com/davsclaus
>>>>>
>>>>>
>>>>
>>>> --
>>>> View this message in context:
>>>> http://old.nabble.com/Renamer-failing-to-rename-%27From%27-File-when-using-multiple-endpoints-via-Multicast-tp27272407p27273273.html
>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Claus Ibsen
>>> Apache Camel Committer
>>>
>>> Author of Camel in Action: http://www.manning.com/ibsen/
>>> Open Source Integration: http://fusesource.com
>>> Blog: http://davsclaus.blogspot.com/
>>> Twitter: http://twitter.com/davsclaus
>>>
>>
>>
>>
>> --
>> Claus Ibsen
>> Apache Camel Committer
>>
>> Author of Camel in Action: http://www.manning.com/ibsen/
>> Open Source Integration: http://fusesource.com
>> Blog: http://davsclaus.blogspot.com/
>> Twitter: http://twitter.com/davsclaus
>>
>>
>
> --
> View this message in context:
> http://old.nabble.com/Renamer-failing-to-rename-%27From%27-File-when-using-multiple-endpoints-via-Multicast-tp27272407p27273955.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
>
--
Claus Ibsen
Apache Camel Committer
Author of Camel in Action: http://www.manning.com/ibsen/
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus
--
View this message in context: http://old.nabble.com/Renamer-failing-to-rename-%27From%27-File-when-using-multiple-endpoints-via-Multicast-tp27272407p27274388.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Renamer failing to rename 'From' File when using multiple
endpoints via Multicast
Posted by Claus Ibsen <cl...@gmail.com>.
On Fri, Jan 22, 2010 at 3:27 PM, Andy Bourke <an...@pcmsgroup.com> wrote:
>
> Please excuse my ignorance Claus, but I don't know what "log:foo" means, I
> have tried using it as you have written but I get an exception:
>
> org.xml.sax.SAXParseException: The prefix "log" for element "log:info" is
> not bound
>
To use the log component you should just do
<to uri="log:fooOrWhatYouLike"/>
http://camel.apache.org/log.html
> I have tried without using parallelProcessing="true" and the problem
> remains.
>
Okay thanks. Not a race condition then.
> How do I create a ticket in Jira? I have never used it.
>
No problem we will get you up to speed. JIRA is used a lot in the OS
world so you may as well start learning
it to be able to submit bugs or requests for new features etc.
Support page here with link to Issue Tracker = JIRA
http://camel.apache.org/support.html
You create an account and then there is a "create new issue button" on
the "menu bar".
>
>
> Claus Ibsen-2 wrote:
>>
>> On Fri, Jan 22, 2010 at 2:47 PM, Claus Ibsen <cl...@gmail.com>
>> wrote:
>>> On Fri, Jan 22, 2010 at 2:41 PM, Andy Bourke <an...@pcmsgroup.com>
>>> wrote:
>>>>
>>>> If I understand it right, the multicast allows the same payload to be
>>>> processed by multiple end points? If I remove the multicast then won't
>>>> the
>>>> second enpoint (in the now non-multicast pipeline) try to process the
>>>> output
>>>> from the first one?
>>>>
>>>
>>> Correct but just to see if the file gets renamed.
>>> Also try the multicast without parallel as well.
>>>
>>
>> Could you also try NOT to write to file in the multicast. Instead just
>> do a "log:foo".
>>
>> I suspect it interferes with the original file.
>> Which may also be a good time to spend the day coding to implement a
>> stack based file context
>> so they wont interfere each other. Will also fix the file -> pollEnrich as
>> well.
>>
>> After your findings could you create a ticket in JIRA? And a link back
>> to this forum, eg find it from nabble.
>> After coming back from a business trip I should have to to work on this
>> issue.
>>
>>
>>>
>>>>
>>>> Claus Ibsen-2 wrote:
>>>>>
>>>>> On Fri, Jan 22, 2010 at 1:20 PM, Andy Bourke
>>>>> <an...@pcmsgroup.com>
>>>>> wrote:
>>>>>>
>>>>>> Hi all,
>>>>>>
>>>>>> I am new to camel and have hit a problem that I hope someone can help
>>>>>> me
>>>>>> with.
>>>>>>
>>>>>> I have a route that looks for a file in a directory => unmarshals the
>>>>>> file
>>>>>> using <CSV> => transforms the output from the unmarshal (which is
>>>>>> List<List<String>>) to a List of data objetcs => passes the List of
>>>>>> data
>>>>>> objects to a processor which does the intelligent mapping of the input
>>>>>> data
>>>>>> to the output which is a List of Objects => passes the List of Objects
>>>>>> to
>>>>>> a
>>>>>> <multicast> pipeline who's endpoints traverse the List of Objects and
>>>>>> extract the data, that they are interested in, formatting the output
>>>>>> in
>>>>>> to a
>>>>>> List of Map objects so that the <CSV> marshaller can marshal to a
>>>>>> named
>>>>>> file.
>>>>>>
>>>>>> This is the route:
>>>>>>
>>>>>> <route id="iq">
>>>>>> <from uri="file:/data/iq/inbound/"/>
>>>>>> <unmarshal>
>>>>>> <csv id="pipeCsvDataFormat"/>
>>>>>> </unmarshal>
>>>>>> <bean ref="iqTransform" method="doTransform"/>
>>>>>> <to uri="bean:iqProcessor?method=process"/>
>>>>>> <multicast parallelProcessing="true">
>>>>>> <pipeline>
>>>>>> <to uri="bean:formatOutput?method=formatHeader"/>
>>>>>> <marshal>
>>>>>> <csv />
>>>>>> </marshal>
>>>>>> <to
>>>>>> uri="file:/data/iq/outbound/?fileName=Txn_Header.txt"/>
>>>>>> </pipeline>
>>>>>> <pipeline>
>>>>>> <to
>>>>>> uri="bean:formatOutput?method=formatLineHeader"/>
>>>>>> <marshal>
>>>>>> <csv />
>>>>>> </marshal>
>>>>>> <to
>>>>>> uri="file:/data/iq/outbound/?fileName=Line_Header.txt"/>
>>>>>> </pipeline>
>>>>>> </multicast>
>>>>>> </route>
>>>>>>
>>>>>>
>>>>>> As expected, I get a correctly formatted file named Txn_Header.txt and
>>>>>> a
>>>>>> file named Line_Header.txt in the /outbound directory. So the routing
>>>>>> seems
>>>>>> to have worked fine.
>>>>>>
>>>>>> The problem is that the original file does not get renamed in to the
>>>>>> .camel
>>>>>> subdirectory and so the file gets processed again and again.
>>>>>>
>>>>>> If I remove the second endpoint in the multicast so that only the
>>>>>> Txn_Header.txt file is produced then the original file does get
>>>>>> renamed
>>>>>> and
>>>>>> the route ends successfully.
>>>>>>
>>>>>
>>>>> And with no mulitcast have you tried that?
>>>>>
>>>>>
>>>>>> Can someone help me understand what I have done wrong please?
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> View this message in context:
>>>>>> http://old.nabble.com/Renamer-failing-to-rename-%27From%27-File-when-using-multiple-endpoints-via-Multicast-tp27272407p27272407.html
>>>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Claus Ibsen
>>>>> Apache Camel Committer
>>>>>
>>>>> Author of Camel in Action: http://www.manning.com/ibsen/
>>>>> Open Source Integration: http://fusesource.com
>>>>> Blog: http://davsclaus.blogspot.com/
>>>>> Twitter: http://twitter.com/davsclaus
>>>>>
>>>>>
>>>>
>>>> --
>>>> View this message in context:
>>>> http://old.nabble.com/Renamer-failing-to-rename-%27From%27-File-when-using-multiple-endpoints-via-Multicast-tp27272407p27273273.html
>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Claus Ibsen
>>> Apache Camel Committer
>>>
>>> Author of Camel in Action: http://www.manning.com/ibsen/
>>> Open Source Integration: http://fusesource.com
>>> Blog: http://davsclaus.blogspot.com/
>>> Twitter: http://twitter.com/davsclaus
>>>
>>
>>
>>
>> --
>> Claus Ibsen
>> Apache Camel Committer
>>
>> Author of Camel in Action: http://www.manning.com/ibsen/
>> Open Source Integration: http://fusesource.com
>> Blog: http://davsclaus.blogspot.com/
>> Twitter: http://twitter.com/davsclaus
>>
>>
>
> --
> View this message in context: http://old.nabble.com/Renamer-failing-to-rename-%27From%27-File-when-using-multiple-endpoints-via-Multicast-tp27272407p27273955.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
>
--
Claus Ibsen
Apache Camel Committer
Author of Camel in Action: http://www.manning.com/ibsen/
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus
Re: Renamer failing to rename 'From' File when using multiple
endpoints via Multicast
Posted by Andy Bourke <an...@pcmsgroup.com>.
Please excuse my ignorance Claus, but I don't know what "log:foo" means, I
have tried using it as you have written but I get an exception:
org.xml.sax.SAXParseException: The prefix "log" for element "log:info" is
not bound
I have tried without using parallelProcessing="true" and the problem
remains.
How do I create a ticket in Jira? I have never used it.
Claus Ibsen-2 wrote:
>
> On Fri, Jan 22, 2010 at 2:47 PM, Claus Ibsen <cl...@gmail.com>
> wrote:
>> On Fri, Jan 22, 2010 at 2:41 PM, Andy Bourke <an...@pcmsgroup.com>
>> wrote:
>>>
>>> If I understand it right, the multicast allows the same payload to be
>>> processed by multiple end points? If I remove the multicast then won't
>>> the
>>> second enpoint (in the now non-multicast pipeline) try to process the
>>> output
>>> from the first one?
>>>
>>
>> Correct but just to see if the file gets renamed.
>> Also try the multicast without parallel as well.
>>
>
> Could you also try NOT to write to file in the multicast. Instead just
> do a "log:foo".
>
> I suspect it interferes with the original file.
> Which may also be a good time to spend the day coding to implement a
> stack based file context
> so they wont interfere each other. Will also fix the file -> pollEnrich as
> well.
>
> After your findings could you create a ticket in JIRA? And a link back
> to this forum, eg find it from nabble.
> After coming back from a business trip I should have to to work on this
> issue.
>
>
>>
>>>
>>> Claus Ibsen-2 wrote:
>>>>
>>>> On Fri, Jan 22, 2010 at 1:20 PM, Andy Bourke
>>>> <an...@pcmsgroup.com>
>>>> wrote:
>>>>>
>>>>> Hi all,
>>>>>
>>>>> I am new to camel and have hit a problem that I hope someone can help
>>>>> me
>>>>> with.
>>>>>
>>>>> I have a route that looks for a file in a directory => unmarshals the
>>>>> file
>>>>> using <CSV> => transforms the output from the unmarshal (which is
>>>>> List<List<String>>) to a List of data objetcs => passes the List of
>>>>> data
>>>>> objects to a processor which does the intelligent mapping of the input
>>>>> data
>>>>> to the output which is a List of Objects => passes the List of Objects
>>>>> to
>>>>> a
>>>>> <multicast> pipeline who's endpoints traverse the List of Objects and
>>>>> extract the data, that they are interested in, formatting the output
>>>>> in
>>>>> to a
>>>>> List of Map objects so that the <CSV> marshaller can marshal to a
>>>>> named
>>>>> file.
>>>>>
>>>>> This is the route:
>>>>>
>>>>> <route id="iq">
>>>>> <from uri="file:/data/iq/inbound/"/>
>>>>> <unmarshal>
>>>>> <csv id="pipeCsvDataFormat"/>
>>>>> </unmarshal>
>>>>> <bean ref="iqTransform" method="doTransform"/>
>>>>> <to uri="bean:iqProcessor?method=process"/>
>>>>> <multicast parallelProcessing="true">
>>>>> <pipeline>
>>>>> <to uri="bean:formatOutput?method=formatHeader"/>
>>>>> <marshal>
>>>>> <csv />
>>>>> </marshal>
>>>>> <to
>>>>> uri="file:/data/iq/outbound/?fileName=Txn_Header.txt"/>
>>>>> </pipeline>
>>>>> <pipeline>
>>>>> <to
>>>>> uri="bean:formatOutput?method=formatLineHeader"/>
>>>>> <marshal>
>>>>> <csv />
>>>>> </marshal>
>>>>> <to
>>>>> uri="file:/data/iq/outbound/?fileName=Line_Header.txt"/>
>>>>> </pipeline>
>>>>> </multicast>
>>>>> </route>
>>>>>
>>>>>
>>>>> As expected, I get a correctly formatted file named Txn_Header.txt and
>>>>> a
>>>>> file named Line_Header.txt in the /outbound directory. So the routing
>>>>> seems
>>>>> to have worked fine.
>>>>>
>>>>> The problem is that the original file does not get renamed in to the
>>>>> .camel
>>>>> subdirectory and so the file gets processed again and again.
>>>>>
>>>>> If I remove the second endpoint in the multicast so that only the
>>>>> Txn_Header.txt file is produced then the original file does get
>>>>> renamed
>>>>> and
>>>>> the route ends successfully.
>>>>>
>>>>
>>>> And with no mulitcast have you tried that?
>>>>
>>>>
>>>>> Can someone help me understand what I have done wrong please?
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> View this message in context:
>>>>> http://old.nabble.com/Renamer-failing-to-rename-%27From%27-File-when-using-multiple-endpoints-via-Multicast-tp27272407p27272407.html
>>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Claus Ibsen
>>>> Apache Camel Committer
>>>>
>>>> Author of Camel in Action: http://www.manning.com/ibsen/
>>>> Open Source Integration: http://fusesource.com
>>>> Blog: http://davsclaus.blogspot.com/
>>>> Twitter: http://twitter.com/davsclaus
>>>>
>>>>
>>>
>>> --
>>> View this message in context:
>>> http://old.nabble.com/Renamer-failing-to-rename-%27From%27-File-when-using-multiple-endpoints-via-Multicast-tp27272407p27273273.html
>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>
>>>
>>
>>
>>
>> --
>> Claus Ibsen
>> Apache Camel Committer
>>
>> Author of Camel in Action: http://www.manning.com/ibsen/
>> Open Source Integration: http://fusesource.com
>> Blog: http://davsclaus.blogspot.com/
>> Twitter: http://twitter.com/davsclaus
>>
>
>
>
> --
> Claus Ibsen
> Apache Camel Committer
>
> Author of Camel in Action: http://www.manning.com/ibsen/
> Open Source Integration: http://fusesource.com
> Blog: http://davsclaus.blogspot.com/
> Twitter: http://twitter.com/davsclaus
>
>
--
View this message in context: http://old.nabble.com/Renamer-failing-to-rename-%27From%27-File-when-using-multiple-endpoints-via-Multicast-tp27272407p27273955.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Renamer failing to rename 'From' File when using multiple
endpoints via Multicast
Posted by Claus Ibsen <cl...@gmail.com>.
On Fri, Jan 22, 2010 at 2:47 PM, Claus Ibsen <cl...@gmail.com> wrote:
> On Fri, Jan 22, 2010 at 2:41 PM, Andy Bourke <an...@pcmsgroup.com> wrote:
>>
>> If I understand it right, the multicast allows the same payload to be
>> processed by multiple end points? If I remove the multicast then won't the
>> second enpoint (in the now non-multicast pipeline) try to process the output
>> from the first one?
>>
>
> Correct but just to see if the file gets renamed.
> Also try the multicast without parallel as well.
>
Could you also try NOT to write to file in the multicast. Instead just
do a "log:foo".
I suspect it interferes with the original file.
Which may also be a good time to spend the day coding to implement a
stack based file context
so they wont interfere each other. Will also fix the file -> pollEnrich as well.
After your findings could you create a ticket in JIRA? And a link back
to this forum, eg find it from nabble.
After coming back from a business trip I should have to to work on this issue.
>
>>
>> Claus Ibsen-2 wrote:
>>>
>>> On Fri, Jan 22, 2010 at 1:20 PM, Andy Bourke <an...@pcmsgroup.com>
>>> wrote:
>>>>
>>>> Hi all,
>>>>
>>>> I am new to camel and have hit a problem that I hope someone can help me
>>>> with.
>>>>
>>>> I have a route that looks for a file in a directory => unmarshals the
>>>> file
>>>> using <CSV> => transforms the output from the unmarshal (which is
>>>> List<List<String>>) to a List of data objetcs => passes the List of data
>>>> objects to a processor which does the intelligent mapping of the input
>>>> data
>>>> to the output which is a List of Objects => passes the List of Objects to
>>>> a
>>>> <multicast> pipeline who's endpoints traverse the List of Objects and
>>>> extract the data, that they are interested in, formatting the output in
>>>> to a
>>>> List of Map objects so that the <CSV> marshaller can marshal to a named
>>>> file.
>>>>
>>>> This is the route:
>>>>
>>>> <route id="iq">
>>>> <from uri="file:/data/iq/inbound/"/>
>>>> <unmarshal>
>>>> <csv id="pipeCsvDataFormat"/>
>>>> </unmarshal>
>>>> <bean ref="iqTransform" method="doTransform"/>
>>>> <to uri="bean:iqProcessor?method=process"/>
>>>> <multicast parallelProcessing="true">
>>>> <pipeline>
>>>> <to uri="bean:formatOutput?method=formatHeader"/>
>>>> <marshal>
>>>> <csv />
>>>> </marshal>
>>>> <to
>>>> uri="file:/data/iq/outbound/?fileName=Txn_Header.txt"/>
>>>> </pipeline>
>>>> <pipeline>
>>>> <to uri="bean:formatOutput?method=formatLineHeader"/>
>>>> <marshal>
>>>> <csv />
>>>> </marshal>
>>>> <to
>>>> uri="file:/data/iq/outbound/?fileName=Line_Header.txt"/>
>>>> </pipeline>
>>>> </multicast>
>>>> </route>
>>>>
>>>>
>>>> As expected, I get a correctly formatted file named Txn_Header.txt and a
>>>> file named Line_Header.txt in the /outbound directory. So the routing
>>>> seems
>>>> to have worked fine.
>>>>
>>>> The problem is that the original file does not get renamed in to the
>>>> .camel
>>>> subdirectory and so the file gets processed again and again.
>>>>
>>>> If I remove the second endpoint in the multicast so that only the
>>>> Txn_Header.txt file is produced then the original file does get renamed
>>>> and
>>>> the route ends successfully.
>>>>
>>>
>>> And with no mulitcast have you tried that?
>>>
>>>
>>>> Can someone help me understand what I have done wrong please?
>>>>
>>>>
>>>>
>>>> --
>>>> View this message in context:
>>>> http://old.nabble.com/Renamer-failing-to-rename-%27From%27-File-when-using-multiple-endpoints-via-Multicast-tp27272407p27272407.html
>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Claus Ibsen
>>> Apache Camel Committer
>>>
>>> Author of Camel in Action: http://www.manning.com/ibsen/
>>> Open Source Integration: http://fusesource.com
>>> Blog: http://davsclaus.blogspot.com/
>>> Twitter: http://twitter.com/davsclaus
>>>
>>>
>>
>> --
>> View this message in context: http://old.nabble.com/Renamer-failing-to-rename-%27From%27-File-when-using-multiple-endpoints-via-Multicast-tp27272407p27273273.html
>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>
>>
>
>
>
> --
> Claus Ibsen
> Apache Camel Committer
>
> Author of Camel in Action: http://www.manning.com/ibsen/
> Open Source Integration: http://fusesource.com
> Blog: http://davsclaus.blogspot.com/
> Twitter: http://twitter.com/davsclaus
>
--
Claus Ibsen
Apache Camel Committer
Author of Camel in Action: http://www.manning.com/ibsen/
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus
Re: Renamer failing to rename 'From' File when using multiple
endpoints via Multicast
Posted by Claus Ibsen <cl...@gmail.com>.
On Fri, Jan 22, 2010 at 2:41 PM, Andy Bourke <an...@pcmsgroup.com> wrote:
>
> If I understand it right, the multicast allows the same payload to be
> processed by multiple end points? If I remove the multicast then won't the
> second enpoint (in the now non-multicast pipeline) try to process the output
> from the first one?
>
Correct but just to see if the file gets renamed.
Also try the multicast without parallel as well.
>
> Claus Ibsen-2 wrote:
>>
>> On Fri, Jan 22, 2010 at 1:20 PM, Andy Bourke <an...@pcmsgroup.com>
>> wrote:
>>>
>>> Hi all,
>>>
>>> I am new to camel and have hit a problem that I hope someone can help me
>>> with.
>>>
>>> I have a route that looks for a file in a directory => unmarshals the
>>> file
>>> using <CSV> => transforms the output from the unmarshal (which is
>>> List<List<String>>) to a List of data objetcs => passes the List of data
>>> objects to a processor which does the intelligent mapping of the input
>>> data
>>> to the output which is a List of Objects => passes the List of Objects to
>>> a
>>> <multicast> pipeline who's endpoints traverse the List of Objects and
>>> extract the data, that they are interested in, formatting the output in
>>> to a
>>> List of Map objects so that the <CSV> marshaller can marshal to a named
>>> file.
>>>
>>> This is the route:
>>>
>>> <route id="iq">
>>> <from uri="file:/data/iq/inbound/"/>
>>> <unmarshal>
>>> <csv id="pipeCsvDataFormat"/>
>>> </unmarshal>
>>> <bean ref="iqTransform" method="doTransform"/>
>>> <to uri="bean:iqProcessor?method=process"/>
>>> <multicast parallelProcessing="true">
>>> <pipeline>
>>> <to uri="bean:formatOutput?method=formatHeader"/>
>>> <marshal>
>>> <csv />
>>> </marshal>
>>> <to
>>> uri="file:/data/iq/outbound/?fileName=Txn_Header.txt"/>
>>> </pipeline>
>>> <pipeline>
>>> <to uri="bean:formatOutput?method=formatLineHeader"/>
>>> <marshal>
>>> <csv />
>>> </marshal>
>>> <to
>>> uri="file:/data/iq/outbound/?fileName=Line_Header.txt"/>
>>> </pipeline>
>>> </multicast>
>>> </route>
>>>
>>>
>>> As expected, I get a correctly formatted file named Txn_Header.txt and a
>>> file named Line_Header.txt in the /outbound directory. So the routing
>>> seems
>>> to have worked fine.
>>>
>>> The problem is that the original file does not get renamed in to the
>>> .camel
>>> subdirectory and so the file gets processed again and again.
>>>
>>> If I remove the second endpoint in the multicast so that only the
>>> Txn_Header.txt file is produced then the original file does get renamed
>>> and
>>> the route ends successfully.
>>>
>>
>> And with no mulitcast have you tried that?
>>
>>
>>> Can someone help me understand what I have done wrong please?
>>>
>>>
>>>
>>> --
>>> View this message in context:
>>> http://old.nabble.com/Renamer-failing-to-rename-%27From%27-File-when-using-multiple-endpoints-via-Multicast-tp27272407p27272407.html
>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>
>>>
>>
>>
>>
>> --
>> Claus Ibsen
>> Apache Camel Committer
>>
>> Author of Camel in Action: http://www.manning.com/ibsen/
>> Open Source Integration: http://fusesource.com
>> Blog: http://davsclaus.blogspot.com/
>> Twitter: http://twitter.com/davsclaus
>>
>>
>
> --
> View this message in context: http://old.nabble.com/Renamer-failing-to-rename-%27From%27-File-when-using-multiple-endpoints-via-Multicast-tp27272407p27273273.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
>
--
Claus Ibsen
Apache Camel Committer
Author of Camel in Action: http://www.manning.com/ibsen/
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus
Re: Renamer failing to rename 'From' File when using multiple
endpoints via Multicast
Posted by Andy Bourke <an...@pcmsgroup.com>.
If I understand it right, the multicast allows the same payload to be
processed by multiple end points? If I remove the multicast then won't the
second enpoint (in the now non-multicast pipeline) try to process the output
from the first one?
Claus Ibsen-2 wrote:
>
> On Fri, Jan 22, 2010 at 1:20 PM, Andy Bourke <an...@pcmsgroup.com>
> wrote:
>>
>> Hi all,
>>
>> I am new to camel and have hit a problem that I hope someone can help me
>> with.
>>
>> I have a route that looks for a file in a directory => unmarshals the
>> file
>> using <CSV> => transforms the output from the unmarshal (which is
>> List<List<String>>) to a List of data objetcs => passes the List of data
>> objects to a processor which does the intelligent mapping of the input
>> data
>> to the output which is a List of Objects => passes the List of Objects to
>> a
>> <multicast> pipeline who's endpoints traverse the List of Objects and
>> extract the data, that they are interested in, formatting the output in
>> to a
>> List of Map objects so that the <CSV> marshaller can marshal to a named
>> file.
>>
>> This is the route:
>>
>> <route id="iq">
>> <from uri="file:/data/iq/inbound/"/>
>> <unmarshal>
>> <csv id="pipeCsvDataFormat"/>
>> </unmarshal>
>> <bean ref="iqTransform" method="doTransform"/>
>> <to uri="bean:iqProcessor?method=process"/>
>> <multicast parallelProcessing="true">
>> <pipeline>
>> <to uri="bean:formatOutput?method=formatHeader"/>
>> <marshal>
>> <csv />
>> </marshal>
>> <to
>> uri="file:/data/iq/outbound/?fileName=Txn_Header.txt"/>
>> </pipeline>
>> <pipeline>
>> <to uri="bean:formatOutput?method=formatLineHeader"/>
>> <marshal>
>> <csv />
>> </marshal>
>> <to
>> uri="file:/data/iq/outbound/?fileName=Line_Header.txt"/>
>> </pipeline>
>> </multicast>
>> </route>
>>
>>
>> As expected, I get a correctly formatted file named Txn_Header.txt and a
>> file named Line_Header.txt in the /outbound directory. So the routing
>> seems
>> to have worked fine.
>>
>> The problem is that the original file does not get renamed in to the
>> .camel
>> subdirectory and so the file gets processed again and again.
>>
>> If I remove the second endpoint in the multicast so that only the
>> Txn_Header.txt file is produced then the original file does get renamed
>> and
>> the route ends successfully.
>>
>
> And with no mulitcast have you tried that?
>
>
>> Can someone help me understand what I have done wrong please?
>>
>>
>>
>> --
>> View this message in context:
>> http://old.nabble.com/Renamer-failing-to-rename-%27From%27-File-when-using-multiple-endpoints-via-Multicast-tp27272407p27272407.html
>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>
>>
>
>
>
> --
> Claus Ibsen
> Apache Camel Committer
>
> Author of Camel in Action: http://www.manning.com/ibsen/
> Open Source Integration: http://fusesource.com
> Blog: http://davsclaus.blogspot.com/
> Twitter: http://twitter.com/davsclaus
>
>
--
View this message in context: http://old.nabble.com/Renamer-failing-to-rename-%27From%27-File-when-using-multiple-endpoints-via-Multicast-tp27272407p27273273.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Renamer failing to rename 'From' File when using multiple
endpoints via Multicast
Posted by Claus Ibsen <cl...@gmail.com>.
On Fri, Jan 22, 2010 at 1:20 PM, Andy Bourke <an...@pcmsgroup.com> wrote:
>
> Hi all,
>
> I am new to camel and have hit a problem that I hope someone can help me
> with.
>
> I have a route that looks for a file in a directory => unmarshals the file
> using <CSV> => transforms the output from the unmarshal (which is
> List<List<String>>) to a List of data objetcs => passes the List of data
> objects to a processor which does the intelligent mapping of the input data
> to the output which is a List of Objects => passes the List of Objects to a
> <multicast> pipeline who's endpoints traverse the List of Objects and
> extract the data, that they are interested in, formatting the output in to a
> List of Map objects so that the <CSV> marshaller can marshal to a named
> file.
>
> This is the route:
>
> <route id="iq">
> <from uri="file:/data/iq/inbound/"/>
> <unmarshal>
> <csv id="pipeCsvDataFormat"/>
> </unmarshal>
> <bean ref="iqTransform" method="doTransform"/>
> <to uri="bean:iqProcessor?method=process"/>
> <multicast parallelProcessing="true">
> <pipeline>
> <to uri="bean:formatOutput?method=formatHeader"/>
> <marshal>
> <csv />
> </marshal>
> <to
> uri="file:/data/iq/outbound/?fileName=Txn_Header.txt"/>
> </pipeline>
> <pipeline>
> <to uri="bean:formatOutput?method=formatLineHeader"/>
> <marshal>
> <csv />
> </marshal>
> <to
> uri="file:/data/iq/outbound/?fileName=Line_Header.txt"/>
> </pipeline>
> </multicast>
> </route>
>
>
> As expected, I get a correctly formatted file named Txn_Header.txt and a
> file named Line_Header.txt in the /outbound directory. So the routing seems
> to have worked fine.
>
> The problem is that the original file does not get renamed in to the .camel
> subdirectory and so the file gets processed again and again.
>
> If I remove the second endpoint in the multicast so that only the
> Txn_Header.txt file is produced then the original file does get renamed and
> the route ends successfully.
>
And with no mulitcast have you tried that?
> Can someone help me understand what I have done wrong please?
>
>
>
> --
> View this message in context: http://old.nabble.com/Renamer-failing-to-rename-%27From%27-File-when-using-multiple-endpoints-via-Multicast-tp27272407p27272407.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
>
--
Claus Ibsen
Apache Camel Committer
Author of Camel in Action: http://www.manning.com/ibsen/
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus