You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by cwhistler <ch...@sungard.com> on 2011/07/20 18:17:59 UTC
file move option not working when using split
I am using camel version 2.7.2
I am watching an inbound folder for files. when I pick up a file I need to
determine if I should do anything with it or just ignore it. If it's a file
I want then I need to split the data in the file into multiple output files.
After a file is processed I want to move the original file to a bkup folder
and append a timestamp to the file name.
So my java route is basically:
from(file://inbound?move=//bkup/${file:name}-${date:now:yyyyMMddHHmmssSSSS})
.choice()
.when(some criteria).to("direct:processFile")
.otherwise().to("direct:donotprocessFile");
from("direct:processFile")
.split().method("javasplitclass", "splitmethod")
.to(file://outbound)
from("direct:donotprocessFile")
.process(javanoprocessclass);
in the javanoprocessclass I am simply printing out the file name:
public void process(Exchange exchange)
{
String fileName = (String)
exchange.getIn().getHeader(Exchange.FILE_NAME_ONLY);
log.info("Skipping unknown file - " + fileName);
}
The files that go to the donotprocessFile route are correctly moved to the
bkup folder and renamed.
The files that go through the split are split correctly, but the original
file remains in the input folder where it is processed repeatedly.
In the javasplitclass, I need to split the file into multiple output files.
so my split method returns a List of Message objects. Each Message
represents the records for a separate file which I name in the message
Header.
public List<Message> splitBody(Exchange exchange)
{
String fileName = (String)
exchange.getIn().getHeader(Exchange.FILE_NAME_ONLY);
log.info("Processing file - " + fileName);
BufferedReader inputReader =
exchange.getIn().getBody(BufferedReader.class);
StringBuffer sb = new StringBuffer();
List<Message> messages = new ArrayList<Message>();
String line = null;
String participant = null;
SsbXref ssbXref = null;
try
{
while (null != (line = inputReader.readLine()))
{
if (isFileHeader(line))
{
// we just skip this
log.debug("File header " + line);
}
else if (isParticipantHeader(line))
{
// we are on a new participant so write out the previous records first
if (sb.length() > 0)
{
messages.add(createOutput(fileName, participant, sb));
sb = new StringBuffer();
}
sb.append(line);
sb.append("\n");
}
else
{
// need to append the line
sb.append(line);
sb.append("\n");
}
}
if (sb.length() > 0)
{
messages.add(createOutput(fileName, participant, sb));
}
}
catch (Exception e)
{
e.printStackTrace();
}
log.info("Completed processing file - " + fileName);
return messages;
}
private Message createOutput(String fileName, String participant,
StringBuffer sb)
{
Message message = new DefaultMessage();
message.setBody(sb.toString());
message.setHeader(Exchange.FILE_NAME, fileName + "." + participant);
return message;
}
so have I done something that is preventing the original file from moving to
the bkup folder after splitting the records out of it?? What can I do to
fix this?
thanks
--
View this message in context: http://camel.465427.n5.nabble.com/file-move-option-not-working-when-using-split-tp4616425p4616425.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: file move option not working when using split
Posted by Claus Ibsen <cl...@gmail.com>.
On Tue, Jul 26, 2011 at 4:06 PM, cwhistler <ch...@sungard.com> wrote:
> FYI the answer to this issue is the same as the problem I had yesterday, I
> simply needed to close my BufferedReader at the end of the process!
>
Thanks for reporting back. I had a look in the source, and Camel ought
to react accordingly and be able to move the original file. But I will
rework the code just a slight bit, to ensure any side effects from end
users who may tamper with the exchange/message wont influence the file
commit strategy.
> --
> View this message in context: http://camel.465427.n5.nabble.com/file-move-option-not-working-when-using-split-tp4616425p4634904.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
--
Claus Ibsen
-----------------
FuseSource
Email: cibsen@fusesource.com
Web: http://fusesource.com
Twitter: davsclaus, fusenews
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/
Re: file move option not working when using split
Posted by cwhistler <ch...@sungard.com>.
FYI the answer to this issue is the same as the problem I had yesterday, I
simply needed to close my BufferedReader at the end of the process!
--
View this message in context: http://camel.465427.n5.nabble.com/file-move-option-not-working-when-using-split-tp4616425p4634904.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: file move option not working when using split
Posted by Claus Ibsen <cl...@gmail.com>.
Hi
I have logged a ticket to improved this in Camel
https://issues.apache.org/jira/browse/CAMEL-4270
On Thu, Jul 21, 2011 at 1:53 PM, cwhistler <ch...@sungard.com> wrote:
> Do you mean to propagate the entire inbound message into the outbound List so
> that it is available when this List is returned?
>
> public List splitBody(Exchange exchange)
> {
> String fileName = (String)
> exchange.getIn().getHeader(Exchange.FILE_NAME_ONLY);
> log.info("Processing file - " + fileName);
> BufferedReader inputReader =
> exchange.getIn().getBody(BufferedReader.class);
> StringBuffer sb = new StringBuffer();
> List messages = new ArrayList();
> *messages.add(exchange.getIn());*
> . . . .
> }
>
> or do you mean to add all of the headers from the inbound message to each of
> the new messages that I am creating and putting in the list?
>
> private Message createOutput(*Exchange exchange*, String fileName, String
> participant, StringBuffer sb)
> {
> Message message = new DefaultMessage();
> message.setBody(sb.toString());
> *message.setHeaders(exchange.getIn().getHeaders());*
> message.setHeader(Exchange.FILE_NAME, fileName + "." + participant);
> return message;
> }
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/file-move-option-not-working-when-using-split-tp4616425p4619103.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
--
Claus Ibsen
-----------------
FuseSource
Email: cibsen@fusesource.com
Web: http://fusesource.com
Twitter: davsclaus, fusenews
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/
Re: file move option not working when using split
Posted by cwhistler <ch...@sungard.com>.
Do you mean to propagate the entire inbound message into the outbound List so
that it is available when this List is returned?
public List splitBody(Exchange exchange)
{
String fileName = (String)
exchange.getIn().getHeader(Exchange.FILE_NAME_ONLY);
log.info("Processing file - " + fileName);
BufferedReader inputReader =
exchange.getIn().getBody(BufferedReader.class);
StringBuffer sb = new StringBuffer();
List messages = new ArrayList();
*messages.add(exchange.getIn());*
. . . .
}
or do you mean to add all of the headers from the inbound message to each of
the new messages that I am creating and putting in the list?
private Message createOutput(*Exchange exchange*, String fileName, String
participant, StringBuffer sb)
{
Message message = new DefaultMessage();
message.setBody(sb.toString());
*message.setHeaders(exchange.getIn().getHeaders());*
message.setHeader(Exchange.FILE_NAME, fileName + "." + participant);
return message;
}
--
View this message in context: http://camel.465427.n5.nabble.com/file-move-option-not-working-when-using-split-tp4616425p4619103.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: file move option not working when using split
Posted by Claus Ibsen <cl...@gmail.com>.
Hi
You may have a corner case when you decide the return Message
instances in the split bean.
Could you ensure all headers is propagated from the Exchange IN
message in all the returned Message's in that list?
On Wed, Jul 20, 2011 at 6:17 PM, cwhistler <ch...@sungard.com> wrote:
> I am using camel version 2.7.2
>
> I am watching an inbound folder for files. when I pick up a file I need to
> determine if I should do anything with it or just ignore it. If it's a file
> I want then I need to split the data in the file into multiple output files.
> After a file is processed I want to move the original file to a bkup folder
> and append a timestamp to the file name.
>
> So my java route is basically:
> from(file://inbound?move=//bkup/${file:name}-${date:now:yyyyMMddHHmmssSSSS})
> .choice()
> .when(some criteria).to("direct:processFile")
> .otherwise().to("direct:donotprocessFile");
>
> from("direct:processFile")
> .split().method("javasplitclass", "splitmethod")
> .to(file://outbound)
>
> from("direct:donotprocessFile")
> .process(javanoprocessclass);
>
> in the javanoprocessclass I am simply printing out the file name:
> public void process(Exchange exchange)
> {
> String fileName = (String)
> exchange.getIn().getHeader(Exchange.FILE_NAME_ONLY);
> log.info("Skipping unknown file - " + fileName);
>
> }
>
>
> The files that go to the donotprocessFile route are correctly moved to the
> bkup folder and renamed.
>
> The files that go through the split are split correctly, but the original
> file remains in the input folder where it is processed repeatedly.
>
> In the javasplitclass, I need to split the file into multiple output files.
> so my split method returns a List of Message objects. Each Message
> represents the records for a separate file which I name in the message
> Header.
>
> public List<Message> splitBody(Exchange exchange)
> {
> String fileName = (String)
> exchange.getIn().getHeader(Exchange.FILE_NAME_ONLY);
> log.info("Processing file - " + fileName);
> BufferedReader inputReader =
> exchange.getIn().getBody(BufferedReader.class);
> StringBuffer sb = new StringBuffer();
> List<Message> messages = new ArrayList<Message>();
> String line = null;
> String participant = null;
> SsbXref ssbXref = null;
> try
> {
> while (null != (line = inputReader.readLine()))
> {
> if (isFileHeader(line))
> {
> // we just skip this
> log.debug("File header " + line);
> }
> else if (isParticipantHeader(line))
> {
> // we are on a new participant so write out the previous records first
> if (sb.length() > 0)
> {
> messages.add(createOutput(fileName, participant, sb));
> sb = new StringBuffer();
> }
> sb.append(line);
> sb.append("\n");
> }
> else
> {
> // need to append the line
> sb.append(line);
> sb.append("\n");
> }
> }
> if (sb.length() > 0)
> {
> messages.add(createOutput(fileName, participant, sb));
> }
> }
> catch (Exception e)
> {
>
> e.printStackTrace();
> }
>
> log.info("Completed processing file - " + fileName);
> return messages;
>
> }
>
> private Message createOutput(String fileName, String participant,
> StringBuffer sb)
> {
> Message message = new DefaultMessage();
> message.setBody(sb.toString());
> message.setHeader(Exchange.FILE_NAME, fileName + "." + participant);
> return message;
> }
>
> so have I done something that is preventing the original file from moving to
> the bkup folder after splitting the records out of it?? What can I do to
> fix this?
>
> thanks
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/file-move-option-not-working-when-using-split-tp4616425p4616425.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
--
Claus Ibsen
-----------------
FuseSource
Email: cibsen@fusesource.com
Web: http://fusesource.com
Twitter: davsclaus, fusenews
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/