You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by atluris <at...@gmail.com> on 2013/08/10 12:48:12 UTC
moveFailed is failing when streaming is on
Hi,
I am new to camel. My requirement is to read a huge file (> 100 KB) and
process it. If process fails due to some reason, file should be moved to
error folder. since it is a huge file I am using split() with streaming().
Below is my route definition.
onException(Exception.class).to("direct:handleFailure");
from("file:C:/UFM/test?delay=10000&readLock=changed&move=processed&moveFailed=error")
.log("Starting to process file: ${header.CamelFileName}")
.setProperty("UfmProcessLogId", method(ufmProcessService,
"logUFMProcessStart(${header.CamelFileName})"))
.split(body().tokenize("\n"), new
UFMAggregationStrategy()).streaming().stopOnException()
.to("direct:processUFM")
.end()
.bean(ufmProcessService,
"logUFMProcessSuccess(${property.UfmProcessLogId},
${property.CamelSplitSize})")
.log("End of process file: ${header.CamelFileName}");
from("direct:processUFM")
.filter(body().startsWith(TRANSACTION_PREFIX))
.unmarshal().bindy(BindyType.Fixed, UFMVehiclePassage.class)
.to("stream:out");
from("direct:handleFailure")
.bean(ufmProcessService,
"logUFMProcessFailure(${property.UfmProcessLogId},
${property.CamelSplitIndex}, ${body}, ${exception.stacktrace})");
When the process failed, file is failed to move into error folder due to
below exception.
Caused by: java.io.IOException: Renaming file from:
C:\UFM\test\FDOT9010UFMSample1.txt to:
C:\UFM\test\error\FDOT9010UFMSample1.txt failed due cannot delete from file:
C:\UFM\test\FDOT9010UFMSample1.txt after copy succeeded
File is being successfully moved to error folder when streaming() is removed
or if I use convertBodyToType(String).
Since it is huge file, I have to use streaming() and I can't use
convertBodyToType(String) because it loads whole file into memory which I
want to avoid in the first place.
Please suggest me if there is any solution for this.
OS: Windows
Camel: 2.11.1 on JDK7.
Thanks & Regards,
Srinivas Atluri
--
View this message in context: http://camel.465427.n5.nabble.com/moveFailed-is-failing-when-streaming-is-on-tp5737065.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: moveFailed is failing when streaming is on
Posted by Claus Ibsen <cl...@gmail.com>.
I logged a ticket about the problem with moving to failed due the
stream is locked.
https://issues.apache.org/jira/browse/CAMEL-6627
The reason is that you have stopOnExcepition=true, and the splitter
EIP didn't close the stream at this point, only at the end.
This problem is only on windows platforms where the windows file
system will lock the file.
On Mon, Aug 12, 2013 at 11:08 AM, atluris <at...@gmail.com> wrote:
> shareUnitOfWork=true moved file to 'moveFailed' directory, but onException (
> onException(Exception.class).to("direct:handleFailure"); ) is not invoked. I
> tried deadLetterChannel(
> errorHandler(deadLetterChannel("direct:handleUFMFailure")); ) instead.
> handleUFMFailure is invoked this time, but file is moved to 'move' directory
> instead of 'moveFailed'.
>
> I want "direct:handleFailure" to be invoked along with moving file to error
> folder.
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/moveFailed-is-failing-when-streaming-is-on-tp5737065p5737131.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
--
Claus Ibsen
-----------------
Red Hat, Inc.
Email: cibsen@redhat.com
Twitter: davsclaus
Blog: http://davsclaus.com
Author of Camel in Action: http://www.manning.com/ibsen
Re: moveFailed is failing when streaming is on
Posted by atluris <at...@gmail.com>.
shareUnitOfWork=true moved file to 'moveFailed' directory, but onException (
onException(Exception.class).to("direct:handleFailure"); ) is not invoked. I
tried deadLetterChannel(
errorHandler(deadLetterChannel("direct:handleUFMFailure")); ) instead.
handleUFMFailure is invoked this time, but file is moved to 'move' directory
instead of 'moveFailed'.
I want "direct:handleFailure" to be invoked along with moving file to error
folder.
--
View this message in context: http://camel.465427.n5.nabble.com/moveFailed-is-failing-when-streaming-is-on-tp5737065p5737131.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: moveFailed is failing when streaming is on
Posted by Claus Ibsen <cl...@gmail.com>.
Can you try setting shareUnitOfWork=true on the splitter.
On Sat, Aug 10, 2013 at 12:48 PM, atluris <at...@gmail.com> wrote:
> Hi,
>
> I am new to camel. My requirement is to read a huge file (> 100 KB) and
> process it. If process fails due to some reason, file should be moved to
> error folder. since it is a huge file I am using split() with streaming().
> Below is my route definition.
>
> onException(Exception.class).to("direct:handleFailure");
>
>
> from("file:C:/UFM/test?delay=10000&readLock=changed&move=processed&moveFailed=error")
> .log("Starting to process file: ${header.CamelFileName}")
> .setProperty("UfmProcessLogId", method(ufmProcessService,
> "logUFMProcessStart(${header.CamelFileName})"))
> .split(body().tokenize("\n"), new
> UFMAggregationStrategy()).streaming().stopOnException()
> .to("direct:processUFM")
> .end()
> .bean(ufmProcessService,
> "logUFMProcessSuccess(${property.UfmProcessLogId},
> ${property.CamelSplitSize})")
> .log("End of process file: ${header.CamelFileName}");
>
> from("direct:processUFM")
> .filter(body().startsWith(TRANSACTION_PREFIX))
> .unmarshal().bindy(BindyType.Fixed, UFMVehiclePassage.class)
> .to("stream:out");
>
> from("direct:handleFailure")
> .bean(ufmProcessService,
> "logUFMProcessFailure(${property.UfmProcessLogId},
> ${property.CamelSplitIndex}, ${body}, ${exception.stacktrace})");
>
> When the process failed, file is failed to move into error folder due to
> below exception.
>
> Caused by: java.io.IOException: Renaming file from:
> C:\UFM\test\FDOT9010UFMSample1.txt to:
> C:\UFM\test\error\FDOT9010UFMSample1.txt failed due cannot delete from file:
> C:\UFM\test\FDOT9010UFMSample1.txt after copy succeeded
>
> File is being successfully moved to error folder when streaming() is removed
> or if I use convertBodyToType(String).
>
> Since it is huge file, I have to use streaming() and I can't use
> convertBodyToType(String) because it loads whole file into memory which I
> want to avoid in the first place.
>
> Please suggest me if there is any solution for this.
>
> OS: Windows
> Camel: 2.11.1 on JDK7.
>
> Thanks & Regards,
> Srinivas Atluri
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/moveFailed-is-failing-when-streaming-is-on-tp5737065.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
--
Claus Ibsen
-----------------
Red Hat, Inc.
Email: cibsen@redhat.com
Twitter: davsclaus
Blog: http://davsclaus.com
Author of Camel in Action: http://www.manning.com/ibsen