You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by rrajen2 <rr...@capario.com> on 2013/09/28 02:40:50 UTC
File component consumer to consume files by timestamp based
Hi
My requirements for the file component consumer is to just get list of
filenames from a directory that have last modified timestamp which is 1 hr
and greater and append the list of files to a file say data/outputlist.txt.
How would I do that? Could you plz advice?
The pseudo code that I have so far is
Route1://create a data/outputlist.txt containing list of files that are 1 hr
and greater in the data/report directory.
from("file:data/report?noop=true&recursive=true&delay=3600000&filter=MyFileFilter")
.processor(Exchange e)
{
exchange.getIn().setBody(${CamelFileAbsolutePath});
}
.to("file:data/outputlist.txt?doneFileName=${file:name}.done&fileExist=Append");
Route2: //this route actually has to email the data/outputlist.txt file
after its completely done by Route1 and delete the file.
from("file:data/outputlist.txt?doneFileName=${file:name}.done&delay=3400000}
.... //where i email the outputlist.txt file after it is completely
done and finally delete the outputlist.txt
The MyFilter is my custom filter that I am intending to do similar to the
below:
public class MyFileFilter<T> implements GenericFileFilter<T> {
public boolean accept(GenericFile<T> file) {
// we want all directories
if (file.isDirectory()) {
return true;
}
// we dont accept any files starting with skip in the name
return !file.getFileName().startsWith("skip");
}
}
But I don't know how in the MyFileFilter I could filter based on the
timestamp?
--
View this message in context: http://camel.465427.n5.nabble.com/File-component-consumer-to-consume-files-by-timestamp-based-tp5740319.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: File component consumer to consume files by timestamp based
Posted by rrajen2 <rr...@capario.com>.
The solution for the first route listed above has evolved as below with some
issues. Please assist me in fixing it
from("file:{{fmsSys.root.File.InProgress.In}}?noop=true&recursive=true&filter=#myInFMSDirectoryFilter&antInclude={{fmsSys.root.ClientFile.Mailbox.AntIncludeDir}}&delay=500").routeId("InFMSReader")
.to("log:?level=INFO&showExchangeId=true&showHeaders=true&showProperties=true&multiline=true")
.setBody().simple("${in.header.CamelFileAbsolutePath}")
.setBody(body().append("\n"))
.log("Message is ${in.header.CamelFileAbsolutePath}")
.setHeader(Exchange.FILE_NAME,constant("outputlist.txt"))
.to("file:data/FMSTest/outbox?fileExist=Append&doneFileName=outputlist.txt.done")
//if this statement is commented I see all the files that are old 1 hr and
above.
;
Based on the above solution, could you please answer the below questions
1)How can I append all the file names that are 1 hr and above to
outputlist.txt ? Currently with the above solution it only has some file
names written to the outputlist.txt .But if I comment out the last
".to("File:data/FMSTest...)", I am able to see that the route gets all the
file names that are required.
2)I want to create a done file at the end of creation of the outputlist.txt.
I suspect my above solution will not work for my requirement because it
looks like the file reader serially writes one file name at a time to the
outputlist.txt and also writes a outputlist.txt.done and for the next
filename it goes on to append to the outputlist.txt and overwrites the
outputlist.txt.done. Please advice on how I could implement that?
--
View this message in context: http://camel.465427.n5.nabble.com/File-component-consumer-to-consume-files-by-timestamp-based-tp5740319p5745290.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: File component consumer to consume files by timestamp based
Posted by Claus Ibsen <cl...@gmail.com>.
You can check if its a file or directory, and then always accept
directories, and then only file the timestamp on files.
There is a file.isDirectory() you can use.
On Tue, Oct 1, 2013 at 2:48 AM, rrajen2 <rr...@capario.com> wrote:
> Thx Claus.
>
> In the i/p file directory structure data/report/infolder ,it has several zip
> files underneath it. Say when one of the zip file gets modified, the
> infolder directory timestamp also accordingly gets modified.
> So lets say there are 5 zip files in the infolder which was modified 1hr or
> more and 1 zip file in the infolder which is modified within the 1 hr. What
> I am expecting is a list of the 5 zip files. But based on my below
> implementation I don't get any files since the infolder's timestamp has
> undergone change within the 1 hr and is getting excluded out completely.
> Could you please advice on how to solve it, Below is my file component
> consumer implemenation and its associated filter
>
> from("file:data/report?noop=true&recursive=true&delay=3600000&filter=MyFileFilter")
> ......
> ......
>
> Below is my MyFileFilter
> public class MyFileFilter<T> implements GenericFileFilter<T>
> {
> public boolean accept (GenericFile<T> file)
> {
> long lModified=file.getLastModified();
> long onehourBefoTime= (System.currentTimeMillis()-3600*1000);
> if (onehourBefoTime-lModified>=0) return true;
> return false;
> }
> }
>
> Thanks
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/File-component-consumer-to-consume-files-by-timestamp-based-tp5740319p5740622.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
--
Claus Ibsen
-----------------
Red Hat, Inc.
Email: cibsen@redhat.com
Twitter: davsclaus
Blog: http://davsclaus.com
Author of Camel in Action: http://www.manning.com/ibsen
Re: File component consumer to consume files by timestamp based
Posted by rrajen2 <rr...@capario.com>.
Thx Claus.
In the i/p file directory structure data/report/infolder ,it has several zip
files underneath it. Say when one of the zip file gets modified, the
infolder directory timestamp also accordingly gets modified.
So lets say there are 5 zip files in the infolder which was modified 1hr or
more and 1 zip file in the infolder which is modified within the 1 hr. What
I am expecting is a list of the 5 zip files. But based on my below
implementation I don't get any files since the infolder's timestamp has
undergone change within the 1 hr and is getting excluded out completely.
Could you please advice on how to solve it, Below is my file component
consumer implemenation and its associated filter
from("file:data/report?noop=true&recursive=true&delay=3600000&filter=MyFileFilter")
......
......
Below is my MyFileFilter
public class MyFileFilter<T> implements GenericFileFilter<T>
{
public boolean accept (GenericFile<T> file)
{
long lModified=file.getLastModified();
long onehourBefoTime= (System.currentTimeMillis()-3600*1000);
if (onehourBefoTime-lModified>=0) return true;
return false;
}
}
Thanks
--
View this message in context: http://camel.465427.n5.nabble.com/File-component-consumer-to-consume-files-by-timestamp-based-tp5740319p5740622.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: File component consumer to consume files by timestamp based
Posted by Claus Ibsen <cl...@gmail.com>.
Hi
There is a getLastModified method on the generic file instance, you
can use in the filter.
You can use that to check if its 1 hour old.
Just google how to check file last modified in standard java to see
how to implement such logic.
On Sat, Sep 28, 2013 at 2:40 AM, rrajen2 <rr...@capario.com> wrote:
> Hi
> My requirements for the file component consumer is to just get list of
> filenames from a directory that have last modified timestamp which is 1 hr
> and greater and append the list of files to a file say data/outputlist.txt.
> How would I do that? Could you plz advice?
>
> The pseudo code that I have so far is
> Route1://create a data/outputlist.txt containing list of files that are 1 hr
> and greater in the data/report directory.
>
> from("file:data/report?noop=true&recursive=true&delay=3600000&filter=MyFileFilter")
> .processor(Exchange e)
> {
> exchange.getIn().setBody(${CamelFileAbsolutePath});
> }
>
> .to("file:data/outputlist.txt?doneFileName=${file:name}.done&fileExist=Append");
> Route2: //this route actually has to email the data/outputlist.txt file
> after its completely done by Route1 and delete the file.
>
> from("file:data/outputlist.txt?doneFileName=${file:name}.done&delay=3400000}
> .... //where i email the outputlist.txt file after it is completely
> done and finally delete the outputlist.txt
>
>
> The MyFilter is my custom filter that I am intending to do similar to the
> below:
> public class MyFileFilter<T> implements GenericFileFilter<T> {
> public boolean accept(GenericFile<T> file) {
> // we want all directories
> if (file.isDirectory()) {
> return true;
> }
> // we dont accept any files starting with skip in the name
> return !file.getFileName().startsWith("skip");
> }
> }
> But I don't know how in the MyFileFilter I could filter based on the
> timestamp?
>
>
>
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/File-component-consumer-to-consume-files-by-timestamp-based-tp5740319.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
--
Claus Ibsen
-----------------
Red Hat, Inc.
Email: cibsen@redhat.com
Twitter: davsclaus
Blog: http://davsclaus.com
Author of Camel in Action: http://www.manning.com/ibsen