You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by martin11 <ma...@gmail.com> on 2013/12/02 15:59:26 UTC
Random unsuccessful read from file
Hello,
I have a route triggered by timer where I read data (zulu time) from file
(using poolEnrich) and at the end of route I replace data in file for next
route iteration.
Now after deploy on server I found following problem:
- sometimes data are not loaded from file and poolEnrich return only 0x0a
(Line Feed) character!
- from log I see that at the end of route valid data was ready for persist
into file
- I don`t understand why this happened after cca 200 successful iteration of
this route
- this anomaly happens 2 or 3 times per day (timer is set to 60s on server)
- there is no exception in log (Permission denied, Cannot store file, etc.)
Any idea why this could happen? Why pollEnrich read empty file? No other
process access to this new created file.
I use camel 2.11.0 and java 7.
Could not reproduce this problem on devel environment:
<camel:endpoint id="timeFile"
uri="file://{{working.folder}}/?fileName=filename.time&noop=true&idempotent=false&readLock=none&sendEmptyMessageWhenIdle=true&fileExist=Override"/>
<camel:routeContext id="testRoutes">
<camel:route id="route-test-file">
<camel:from
uri="timer://test?fixedRate=true&period=1s&delay={{timer.delay}}"/>
<camel:pollEnrich ref="timeFile" strategyRef="enrichAggregation"/>
<camel:log message="Data from file:${body}"/>
<camel:convertBodyTo type="java.lang.String" />
<camel:transform>
<camel:groovy>request.body != null ? request.body.trim() :
null</camel:groovy>
</camel:transform>
...
...
...
<camel:setBody>
<camel:constant>20131130125927Z/camel:constant>
</camel:setBody>
<camel:log message="Save data to file:${body}"/>
<camel:to ref="timeFile"/>
</camel:route>
</camel:routeContext>
/**
* Enrich aggregator strategy implementation.
* It keep original exchange message with all headers and properties
* and only replace body with new value.
*/
public class EnrichAggregationStrategy implements AggregationStrategy {
@Override
public Exchange aggregate(Exchange originalExchange, Exchange
newExchange) {
if (originalExchange == null) {
originalExchange = newExchange;
} else {
originalExchange.getIn().setBody(newExchange.getIn().getBody());
}
return originalExchange;
}
}
Thanks for some hint.
--
View this message in context: http://camel.465427.n5.nabble.com/Random-unsuccessful-read-from-file-tp5744181.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Random unsuccessful read from file
Posted by Claus Ibsen <cl...@gmail.com>.
If you always need to poll the same file, then you can also do
<transform>
<simple resultType="java.io.File>{{working.folder}}/filename.time</simple>
</transform>
To set the body as a java.io.File of the file.
On Tue, Dec 3, 2013 at 11:46 AM, martin11 <ma...@gmail.com> wrote:
> Hello Claus,
>
> I set timeout on poolEnrich but the error occured again.
> <camel:pollEnrich ref="timeFile" strategyRef="enrichAggregation"
> timeout="10000"/>
>
> Should it be problem (or feature) when using sendEmptyMessageWhenIdle=true
> on file endpoint?
>
> The server is always in high load and when I try to read data from file it`s
> possible that file system is not responding immediately . Do I also change
> configuration of file endpoint? e.g readLockTimeout?
> <camel:endpoint id="timeFile"
> uri="file://{{working.folder}}/?fileName=filename.time&noop=true&idempotent=false&readLock=none&sendEmptyMessageWhenIdle=true&fileExist=Override"/>
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Random-unsuccessful-read-from-file-tp5744181p5744230.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
--
Claus Ibsen
-----------------
Red Hat, Inc.
Email: cibsen@redhat.com
Twitter: davsclaus
Blog: http://davsclaus.com
Author of Camel in Action: http://www.manning.com/ibsen
Re: Random unsuccessful read from file
Posted by martin11 <ma...@gmail.com>.
Hello Claus,
I set timeout on poolEnrich but the error occured again.
<camel:pollEnrich ref="timeFile" strategyRef="enrichAggregation"
timeout="10000"/>
Should it be problem (or feature) when using sendEmptyMessageWhenIdle=true
on file endpoint?
The server is always in high load and when I try to read data from file it`s
possible that file system is not responding immediately . Do I also change
configuration of file endpoint? e.g readLockTimeout?
<camel:endpoint id="timeFile"
uri="file://{{working.folder}}/?fileName=filename.time&noop=true&idempotent=false&readLock=none&sendEmptyMessageWhenIdle=true&fileExist=Override"/>
--
View this message in context: http://camel.465427.n5.nabble.com/Random-unsuccessful-read-from-file-tp5744181p5744230.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Random unsuccessful read from file
Posted by Claus Ibsen <cl...@gmail.com>.
I suggest to set a timeout option on the pollEnrich as suggested
http://camel.apache.org/content-enricher.html
Depending on the Camel release AFAIR then without a timeout option it
may use receiveNoWait which could potentially not detect any new
files/messages. With the timeout it ensures to give the poll enrich
amble time to detect and pickup any file.
On Mon, Dec 2, 2013 at 3:59 PM, martin11 <ma...@gmail.com> wrote:
> Hello,
>
> I have a route triggered by timer where I read data (zulu time) from file
> (using poolEnrich) and at the end of route I replace data in file for next
> route iteration.
>
> Now after deploy on server I found following problem:
> - sometimes data are not loaded from file and poolEnrich return only 0x0a
> (Line Feed) character!
> - from log I see that at the end of route valid data was ready for persist
> into file
> - I don`t understand why this happened after cca 200 successful iteration of
> this route
> - this anomaly happens 2 or 3 times per day (timer is set to 60s on server)
> - there is no exception in log (Permission denied, Cannot store file, etc.)
>
> Any idea why this could happen? Why pollEnrich read empty file? No other
> process access to this new created file.
> I use camel 2.11.0 and java 7.
>
> Could not reproduce this problem on devel environment:
>
> <camel:endpoint id="timeFile"
> uri="file://{{working.folder}}/?fileName=filename.time&noop=true&idempotent=false&readLock=none&sendEmptyMessageWhenIdle=true&fileExist=Override"/>
> <camel:routeContext id="testRoutes">
> <camel:route id="route-test-file">
> <camel:from
> uri="timer://test?fixedRate=true&period=1s&delay={{timer.delay}}"/>
> <camel:pollEnrich ref="timeFile" strategyRef="enrichAggregation"/>
> <camel:log message="Data from file:${body}"/>
> <camel:convertBodyTo type="java.lang.String" />
>
> <camel:transform>
> <camel:groovy>request.body != null ? request.body.trim() :
> null</camel:groovy>
> </camel:transform>
> ...
> ...
> ...
> <camel:setBody>
> <camel:constant>20131130125927Z/camel:constant>
> </camel:setBody>
> <camel:log message="Save data to file:${body}"/>
> <camel:to ref="timeFile"/>
> </camel:route>
> </camel:routeContext>
>
> /**
> * Enrich aggregator strategy implementation.
> * It keep original exchange message with all headers and properties
> * and only replace body with new value.
> */
> public class EnrichAggregationStrategy implements AggregationStrategy {
>
> @Override
> public Exchange aggregate(Exchange originalExchange, Exchange
> newExchange) {
> if (originalExchange == null) {
> originalExchange = newExchange;
> } else {
> originalExchange.getIn().setBody(newExchange.getIn().getBody());
> }
>
> return originalExchange;
> }
> }
>
> Thanks for some hint.
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Random-unsuccessful-read-from-file-tp5744181.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
--
Claus Ibsen
-----------------
Red Hat, Inc.
Email: cibsen@redhat.com
Twitter: davsclaus
Blog: http://davsclaus.com
Author of Camel in Action: http://www.manning.com/ibsen