You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by "Andreas A." <an...@gmail.com> on 2010/11/09 14:13:19 UTC
Processed file being read again, and is empty.
Hi
I'm struggling with a problem that I think occurs because of two threads
trying to do something to the same file but I'm not sure.
Route1 processes a large file and spends around 30 seconds doing this.
...
<split>
<simple>body</simple>
<log message="sending message to queue" />
<to uri="jms:{{jms.queue.in}}" />
<log message="Waiting ${properties:message.interval} before sending next
message to queue" />
<delay>
<simple>${properties:message.interval}</simple>
</delay>
</split>
<to uri="seda:MyAggregatorQueue" />
</when>
<!-- Route ends -->
Towards the end I send the message to an aggregator that zips the message
and writes it do disk.
Sometimes I then get a GenericFileOperation exception stating that camel
cannot move the file to .camel\filename and at the same time, the route
tries to read the file again, but now the route fails because the file seems
to be empty, at least it cannot be converted to string.
I tried converting the message to String before sending it to the aggregator
because I thought maybe the aggregator would lock the file while the first
route would try to move it? But this has not helped.
--
View this message in context: http://camel.465427.n5.nabble.com/Processed-file-being-read-again-and-is-empty-tp3256771p3256771.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Processed file being read again, and is empty.
Posted by "Andreas A." <an...@gmail.com>.
Hi
Yeah I will do that. I have observed that on Camel 2.4 the error actually
comes from the Splitter exactly like in this post:
http://camel.465427.n5.nabble.com/File-can-t-be-moved-or-deleted-on-Windows-td512484.html#a512484
<route>
<from uri="file:data/in2?move=backup&moveFailed=error" />
<split>
<tokenize token="," />
<to uri="mock:test" />
</split>
</route>
This will fail.
On Camel 2.5 this seems to be working, but when I introduce an Aggregator it
fails. It sometimes fails renaming a file to "backup" dir. It never releases
a .camelLock file on the first file of the batch.
--
View this message in context: http://camel.465427.n5.nabble.com/Processed-file-being-read-again-and-is-empty-tp3256771p3258701.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Processed file being read again, and is empty.
Posted by Claus Ibsen <cl...@gmail.com>.
Hi
Can you create a JIRA ticket and link to this thread (from nabble)
Then we wont forget and can look into it when we got a bit of time.
On Wed, Nov 10, 2010 at 1:41 PM, Andreas A. <an...@gmail.com> wrote:
>
> There's a difference to using 2.4 or 2.5. With 2.5 it is always the last file
> of a batch that fails. When I use 2.4 it is random which file fails and the
> exception is different.
>
> 2010-11-10 13:33:11,139 ERROR [Camel Thread 2 - seda://AggregatorQueue]
> [org.apache.camel.component.file.GenericFileOnCompletion]: Caused by:
> [org.apache.camel.component.file.GenericFileOperationFailedException -
> Cannot rename file: GenericFile[test - Copy (10).txt] to:
> GenericFile[backup\test - Copy (10).txt]]
> org.apache.camel.component.file.GenericFileOperationFailedException: Cannot
> rename file: GenericFile[test - Copy (10).txt] to: GenericFile[backup\test -
> Copy (10).txt]
> at
> org.apache.camel.component.file.strategy.GenericFileProcessStrategySupport.renameFile(GenericFileProcessStrategySupport.java:100)
> at
> org.apache.camel.component.file.strategy.GenericFileRenameProcessStrategy.commit(GenericFileRenameProcessStrategy.java:72)
> at
> org.apache.camel.component.file.GenericFileOnCompletion.processStrategyCommit(GenericFileOnCompletion.java:121)
> at
> org.apache.camel.component.file.GenericFileOnCompletion.onCompletion(GenericFileOnCompletion.java:83)
> at
> org.apache.camel.component.file.GenericFileOnCompletion.onComplete(GenericFileOnCompletion.java:52)
> at org.apache.camel.impl.DefaultUnitOfWork.done(DefaultUnitOfWork.java:175)
> at
> org.apache.camel.processor.UnitOfWorkProcessor.doneUow(UnitOfWorkProcessor.java:121)
> at
> org.apache.camel.processor.UnitOfWorkProcessor.access$000(UnitOfWorkProcessor.java:36)
> at
> org.apache.camel.processor.UnitOfWorkProcessor$1.done(UnitOfWorkProcessor.java:106)
> at
> org.apache.camel.processor.DefaultChannel$1.done(DefaultChannel.java:262)
> at
> org.apache.camel.processor.RedeliveryErrorHandler.processErrorHandler(RedeliveryErrorHandler.java:321)
> at
> org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:202)
> at
> org.apache.camel.processor.DefaultChannel.process(DefaultChannel.java:256)
> at
> org.apache.camel.processor.UnitOfWorkProcessor.process(UnitOfWorkProcessor.java:99)
> at
> org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:70)
> at
> org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:98)
> at
> org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:89)
> at
> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:68)
> at
> org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:70)
> at
> org.apache.camel.component.seda.SedaConsumer.sendToConsumers(SedaConsumer.java:170)
> at org.apache.camel.component.seda.SedaConsumer.run(SedaConsumer.java:112)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:619)
> --
> View this message in context: http://camel.465427.n5.nabble.com/Processed-file-being-read-again-and-is-empty-tp3256771p3258592.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
--
Claus Ibsen
-----------------
FuseSource
Email: cibsen@fusesource.com
Web: http://fusesource.com
Twitter: davsclaus
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/
Re: Processed file being read again, and is empty.
Posted by "Andreas A." <an...@gmail.com>.
There's a difference to using 2.4 or 2.5. With 2.5 it is always the last file
of a batch that fails. When I use 2.4 it is random which file fails and the
exception is different.
2010-11-10 13:33:11,139 ERROR [Camel Thread 2 - seda://AggregatorQueue]
[org.apache.camel.component.file.GenericFileOnCompletion]: Caused by:
[org.apache.camel.component.file.GenericFileOperationFailedException -
Cannot rename file: GenericFile[test - Copy (10).txt] to:
GenericFile[backup\test - Copy (10).txt]]
org.apache.camel.component.file.GenericFileOperationFailedException: Cannot
rename file: GenericFile[test - Copy (10).txt] to: GenericFile[backup\test -
Copy (10).txt]
at
org.apache.camel.component.file.strategy.GenericFileProcessStrategySupport.renameFile(GenericFileProcessStrategySupport.java:100)
at
org.apache.camel.component.file.strategy.GenericFileRenameProcessStrategy.commit(GenericFileRenameProcessStrategy.java:72)
at
org.apache.camel.component.file.GenericFileOnCompletion.processStrategyCommit(GenericFileOnCompletion.java:121)
at
org.apache.camel.component.file.GenericFileOnCompletion.onCompletion(GenericFileOnCompletion.java:83)
at
org.apache.camel.component.file.GenericFileOnCompletion.onComplete(GenericFileOnCompletion.java:52)
at org.apache.camel.impl.DefaultUnitOfWork.done(DefaultUnitOfWork.java:175)
at
org.apache.camel.processor.UnitOfWorkProcessor.doneUow(UnitOfWorkProcessor.java:121)
at
org.apache.camel.processor.UnitOfWorkProcessor.access$000(UnitOfWorkProcessor.java:36)
at
org.apache.camel.processor.UnitOfWorkProcessor$1.done(UnitOfWorkProcessor.java:106)
at
org.apache.camel.processor.DefaultChannel$1.done(DefaultChannel.java:262)
at
org.apache.camel.processor.RedeliveryErrorHandler.processErrorHandler(RedeliveryErrorHandler.java:321)
at
org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:202)
at
org.apache.camel.processor.DefaultChannel.process(DefaultChannel.java:256)
at
org.apache.camel.processor.UnitOfWorkProcessor.process(UnitOfWorkProcessor.java:99)
at
org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:70)
at
org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:98)
at
org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:89)
at
org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:68)
at
org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:70)
at
org.apache.camel.component.seda.SedaConsumer.sendToConsumers(SedaConsumer.java:170)
at org.apache.camel.component.seda.SedaConsumer.run(SedaConsumer.java:112)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
--
View this message in context: http://camel.465427.n5.nabble.com/Processed-file-being-read-again-and-is-empty-tp3256771p3258592.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Processed file being read again, and is empty.
Posted by "Andreas A." <an...@gmail.com>.
Hi, I have created a test sample that recreates this issue. Maybe I should
not be using an Aggregator to solve this. Basically I just want to zip
together an incoming message with a message(a response) that I generate on
the basis of the incoming message.
Is this a bug, or am I misusing the Aggregator?
The route is as such:
<route>
<from uri="file:data/in2?move=backup&moveFailed=error" />
<split>
<tokenize token="," />
<to uri="mock:test" />
<delay>
<constant>50</constant>
</delay>
</split>
<to uri="seda:AggregatorQueue" />
</route>
<route>
<from uri="file:data/in?move=backup&moveFailed=error" />
<split>
<tokenize token="," />
<to uri="mock:test" />
<delay>
<constant>150</constant>
</delay>
</split>
<to uri="seda:AggregatorQueue" />
</route>
<route>
<from uri="seda:AggregatorQueue" />
<aggregate strategyRef="myStrat" completionSize="2">
<correlationExpression>
<simple>${file:name}</simple>
</correlationExpression>
<to uri="file:data/out" />
</aggregate>
</route>
Aggregator:
public class MyAggregator implements AggregationStrategy {
public Exchange aggregate(Exchange oldExchange, Exchange newExchange) {
if (oldExchange == null) {
return newExchange;
}
String oldBody = oldExchange.getIn().getBody(String.class);
String newBody = newExchange.getIn().getBody(String.class);
String body = oldBody + "\n+\n" + newBody;
oldExchange.getIn().setBody(body);
oldExchange.setProperty("CamelFileNameOnly", new
ControlGenerator().getControl() + ".txt");
return oldExchange;
}
}
Stacktrace:
2010-11-10 11:22:42,062 ERROR [Camel Thread 0 - file://data/in2]
[org.apache.camel.component.file.GenericFileOnCompletion]: Caused by:
[org.apache.camel.RuntimeCamelException - java.io.FileNotFoundException:
data\in2\test - Copy (9).txt (The system cannot find the file specified)]
org.apache.camel.RuntimeCamelException: java.io.FileNotFoundException:
data\in2\test - Copy (9).txt (The system cannot find the file specified)
at org.apache.camel.util.ObjectHelper.getScanner(ObjectHelper.java:1246)
at org.apache.camel.util.ObjectHelper.getScanner(ObjectHelper.java:1232)
at
org.apache.camel.builder.ExpressionBuilder$40.evaluate(ExpressionBuilder.java:934)
at
org.apache.camel.impl.ExpressionAdapter.evaluate(ExpressionAdapter.java:36)
at
org.apache.camel.processor.Splitter.createProcessorExchangePairs(Splitter.java:99)
at
org.apache.camel.processor.MulticastProcessor.process(MulticastProcessor.java:181)
at org.apache.camel.processor.Splitter.process(Splitter.java:94)
at
org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:70)
at
org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:98)
at
org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:89)
at
org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:68)
at
org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:70)
at
org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:98)
at
org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:89)
at
org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:99)
at
org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:70)
at
org.apache.camel.processor.RedeliveryErrorHandler.processErrorHandler(RedeliveryErrorHandler.java:299)
at
org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:208)
at
org.apache.camel.processor.DefaultChannel.process(DefaultChannel.java:256)
at
org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:70)
at org.apache.camel.processor.Pipeline.process(Pipeline.java:143)
at org.apache.camel.processor.Pipeline.process(Pipeline.java:78)
at
org.apache.camel.processor.UnitOfWorkProcessor.process(UnitOfWorkProcessor.java:99)
at
org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:70)
at
org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:98)
at
org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:89)
at
org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:68)
at
org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:322)
at
org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
at
org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
at
org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at
java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.io.FileNotFoundException: data\in2\test - Copy (9).txt (The
system cannot find the file specified)
at java.io.FileInputStream.open(Native Method)
at java.io.FileInputStream.<init>(FileInputStream.java:106)
at java.util.Scanner.<init>(Scanner.java:636)
at org.apache.camel.util.ObjectHelper.getScanner(ObjectHelper.java:1244)
... 39 more
2010-11-10 11:22:42,067 WARN [Camel Thread 0 - file://data/in2]
[org.apache.camel.component.file.GenericFileOnCompletion]: Rollback file
strategy:
org.apache.camel.component.file.strategy.GenericFileRenameProcessStrategy@1e36c0b
for file: GenericFile[test - Copy (9).txt]
2010-11-10 11:22:42,070 ERROR [Camel Thread 0 - file://data/in2]
[org.apache.camel.component.file.GenericFileOnCompletion]: Caused by:
[org.apache.camel.component.file.GenericFileOperationFailedException -
Cannot rename file: GenericFile[test - Copy (9).txt] to:
GenericFile[error\test - Copy (9).txt]]
org.apache.camel.component.file.GenericFileOperationFailedException: Cannot
rename file: GenericFile[test - Copy (9).txt] to: GenericFile[error\test -
Copy (9).txt]
at
org.apache.camel.component.file.strategy.GenericFileProcessStrategySupport.renameFile(GenericFileProcessStrategySupport.java:100)
at
org.apache.camel.component.file.strategy.GenericFileRenameProcessStrategy.rollback(GenericFileRenameProcessStrategy.java:65)
at
org.apache.camel.component.file.GenericFileOnCompletion.processStrategyRollback(GenericFileOnCompletion.java:141)
at
org.apache.camel.component.file.GenericFileOnCompletion.onCompletion(GenericFileOnCompletion.java:93)
at
org.apache.camel.component.file.GenericFileOnCompletion.onFailure(GenericFileOnCompletion.java:56)
at
org.apache.camel.util.UnitOfWorkHelper.doneSynchronizations(UnitOfWorkHelper.java:50)
at org.apache.camel.impl.DefaultUnitOfWork.done(DefaultUnitOfWork.java:172)
at
org.apache.camel.processor.UnitOfWorkProcessor.doneUow(UnitOfWorkProcessor.java:121)
at
org.apache.camel.processor.UnitOfWorkProcessor.access$000(UnitOfWorkProcessor.java:36)
at
org.apache.camel.processor.UnitOfWorkProcessor$1.done(UnitOfWorkProcessor.java:106)
at org.apache.camel.processor.Pipeline.process(Pipeline.java:130)
at
org.apache.camel.processor.UnitOfWorkProcessor.process(UnitOfWorkProcessor.java:99)
at
org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:70)
at
org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:98)
at
org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:89)
at
org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:68)
at
org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:322)
at
org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
at
org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
at
org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at
java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
--
View this message in context: http://camel.465427.n5.nabble.com/Processed-file-being-read-again-and-is-empty-tp3256771p3258441.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Processed file being read again, and is empty.
Posted by "Andreas A." <an...@gmail.com>.
Hi
I think disabling the lock-files would just be treating the symptoms and not
the real cause.
The AggregationStrategy:
public Exchange aggregate(Exchange oldExchange, Exchange newExchange) {
if (oldExchange == null) {
Map<String, byte[]> messageMap = new HashMap<String, byte[]>();
if (!"empty".equals(newExchange.getIn().getBody(String.class)))
messageMap.put(newExchange.getIn().getHeader("CamelFileNameOnly",
String.class), newExchange.getIn().getBody(byte[].class));
newExchange.getIn().setBody(messageMap);
return newExchange;
}
Map<String, byte[]> messageMap = oldExchange.getIn().getBody(Map.class);
if (!"empty".equals(newExchange.getIn().getBody(String.class)))
messageMap.put(newExchange.getIn().getHeader("CamelFileNameOnly",
String.class), newExchange.getIn().getBody(byte[].class));
oldExchange.getIn().setBody(messageMap);
return oldExchange;
}
The route where it is used:
<route id="AggregateInMsgAndControlQueue">
<from uri="seda:AggregateInMsgAndControlQueue" />
<aggregate completionSize="2" strategyRef="InMsgAndControlAggregator">
<correlationExpression>
<header>envelopeId</header>
</correlationExpression>
<to uri="ZipMessageListProcessor" />
<to
uri="file:{{directory.incoming.backup}}?fileName={{edifact.messagePrefix}}${header.envelopeId}.zip"
/>
</aggregate>
</route>
--
View this message in context: http://camel.465427.n5.nabble.com/Processed-file-being-read-again-and-is-empty-tp3256771p3258288.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Processed file being read again, and is empty.
Posted by Claus Ibsen <cl...@gmail.com>.
Hi
Yeah you gotta be more careful when using the AggregationStrategy
because you need to ensure the on completions is preserved.
So it may actually matter whether or not you return oldExchange or newExchange.
Can you post your AggregationStrategy implementation?
But if you do not need the read locks at all then just set it to
readLock=none on the file endpoint, and you wont have issues with
.camelLock files.
On Tue, Nov 9, 2010 at 3:01 PM, Andreas A. <an...@gmail.com> wrote:
>
> Fails in 2.5 also, as well as leaving the camelLock file behind. But could
> this have anything to do with the aggregator having control over the file
> that the original route is trying to move, or should Camel be able to handle
> this? Would wiretapping over into the queue do any difference?
> --
> View this message in context: http://camel.465427.n5.nabble.com/Processed-file-being-read-again-and-is-empty-tp3256771p3256862.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
--
Claus Ibsen
-----------------
FuseSource
Email: cibsen@fusesource.com
Web: http://fusesource.com
Twitter: davsclaus
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/
Re: Processed file being read again, and is empty.
Posted by "Andreas A." <an...@gmail.com>.
Fails in 2.5 also, as well as leaving the camelLock file behind. But could
this have anything to do with the aggregator having control over the file
that the original route is trying to move, or should Camel be able to handle
this? Would wiretapping over into the queue do any difference?
--
View this message in context: http://camel.465427.n5.nabble.com/Processed-file-being-read-again-and-is-empty-tp3256771p3256862.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Processed file being read again, and is empty.
Posted by "Andreas A." <an...@gmail.com>.
Hi
This is Camel 2.4, I've tried 2.5 also, but this version for some reason
leaves behind a .camelLock file for the first (alphabetically) file of a
batch. Completes the rest (don't know if sometimes fails the same way as
2.4) of the files, with no camelLock file (?). After processing is done the
.camelLock file for the first file of the batch is still left behind and
this obstructs the next time we poll for a batch.
--
View this message in context: http://camel.465427.n5.nabble.com/Processed-file-being-read-again-and-is-empty-tp3256771p3256818.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Processed file being read again, and is empty.
Posted by Claus Ibsen <cl...@gmail.com>.
Hi
What version of Camel are you using?
And as always try with latest version as well to test if its been fixed already.
On Tue, Nov 9, 2010 at 2:13 PM, Andreas A. <an...@gmail.com> wrote:
>
> Hi
>
> I'm struggling with a problem that I think occurs because of two threads
> trying to do something to the same file but I'm not sure.
>
> Route1 processes a large file and spends around 30 seconds doing this.
>
> ...
> <split>
> <simple>body</simple>
> <log message="sending message to queue" />
> <to uri="jms:{{jms.queue.in}}" />
> <log message="Waiting ${properties:message.interval} before sending next
> message to queue" />
> <delay>
> <simple>${properties:message.interval}</simple>
> </delay>
> </split>
> <to uri="seda:MyAggregatorQueue" />
> </when>
> <!-- Route ends -->
>
> Towards the end I send the message to an aggregator that zips the message
> and writes it do disk.
>
> Sometimes I then get a GenericFileOperation exception stating that camel
> cannot move the file to .camel\filename and at the same time, the route
> tries to read the file again, but now the route fails because the file seems
> to be empty, at least it cannot be converted to string.
>
> I tried converting the message to String before sending it to the aggregator
> because I thought maybe the aggregator would lock the file while the first
> route would try to move it? But this has not helped.
> --
> View this message in context: http://camel.465427.n5.nabble.com/Processed-file-being-read-again-and-is-empty-tp3256771p3256771.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
--
Claus Ibsen
-----------------
FuseSource
Email: cibsen@fusesource.com
Web: http://fusesource.com
Twitter: davsclaus
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/
Re: Processed file being read again, and is empty.
Posted by "Andreas A." <an...@gmail.com>.
2010-11-09 14:06:53,197 INFO [Camel Thread 0 - seda://ErrorHandlingQueueIn]
[ErrorHandlingRouteIn]: Enter ErrorHandlingRouteIn
2010-11-09 14:06:53,206 ERROR [Camel Thread 0 - seda://ErrorHandlingQueueIn]
[org.apache.camel.component.file.GenericFileOnCompletion]: Caused by:
[org.apache.camel.component.file.GenericFileOperationFailedException -
Cannot rename file: GenericFile[op12.txt] to:
GenericFile[c:\testcm\op12.txt]]
org.apache.camel.component.file.GenericFileOperationFailedException: Cannot
rename file: GenericFile[op12.txt] to: GenericFile[c:\testcm\op12.txt]
at
org.apache.camel.component.file.strategy.GenericFileProcessStrategySupport.renameFile(GenericFileProcessStrategySupport.java:100)
at
org.apache.camel.component.file.strategy.GenericFileRenameProcessStrategy.commit(GenericFileRenameProcessStrategy.java:72)
at
org.apache.camel.component.file.GenericFileOnCompletion.processStrategyCommit(GenericFileOnCompletion.java:121)
at
org.apache.camel.component.file.GenericFileOnCompletion.onCompletion(GenericFileOnCompletion.java:83)
at
org.apache.camel.component.file.GenericFileOnCompletion.onComplete(GenericFileOnCompletion.java:52)
at org.apache.camel.impl.DefaultUnitOfWork.done(DefaultUnitOfWork.java:175)
at
org.apache.camel.processor.UnitOfWorkProcessor.doneUow(UnitOfWorkProcessor.java:121)
at
org.apache.camel.processor.UnitOfWorkProcessor.access$000(UnitOfWorkProcessor.java:36)
at
org.apache.camel.processor.UnitOfWorkProcessor$1.done(UnitOfWorkProcessor.java:106)
at org.apache.camel.processor.Pipeline.process(Pipeline.java:130)
at
org.apache.camel.processor.UnitOfWorkProcessor.process(UnitOfWorkProcessor.java:99)
at
org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:70)
at
org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:98)
at
org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:89)
at
org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:68)
at
org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:70)
at
org.apache.camel.component.seda.SedaConsumer.sendToConsumers(SedaConsumer.java:170)
at org.apache.camel.component.seda.SedaConsumer.run(SedaConsumer.java:112)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
--
View this message in context: http://camel.465427.n5.nabble.com/Processed-file-being-read-again-and-is-empty-tp3256771p3256781.html
Sent from the Camel - Users mailing list archive at Nabble.com.