You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by hayden74 <hd...@gmail.com> on 2015/08/09 23:18:24 UTC
smb endpoint - modified files not copied to destination
Hi Camel experts.
The issue I am facing is that Camel doesn't seem to copy a modified file
after being copied the first time to its destination. The route used is:
from("smb://networkDriveOne?noop=true&fileName=inputFile.csv")
.to("smb://networkDriveTwo?fileExist=Override")
When Camel is restarted, the inputFile.csv file is copied to its destination
as expected. When inputFile.csv modified and saved in the input directory,
however, Camel doesn't pick it up and override the one in the destination.
Versions:
Camel: 2.14.0
ActiveMQ: 5.11.0
Spring MVC: 4.1.6.RELEASE
--
View this message in context: http://camel.465427.n5.nabble.com/smb-endpoint-modified-files-not-copied-to-destination-tp5770499.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: smb endpoint - modified files not copied to destination
Posted by hayden74 <hd...@gmail.com>.
I finally had time to try this. The result was not what I expected though.
My route looks like this:
from("file://C:/applications/data?fileName=${date:now:yyyyMMdd}/dataFile.csv")
.log("New file received");
and the result was:
Route: route4 started and consuming from:
Endpoint[file://C:/applications/data?fileName=%24%7Bdate%3Anow%3AyyyyMMdd%7D%2FdataFile.csv]
What I expected is:
Route: route4 started and consuming from:
Endpoint[file://C:/applications/data?fileName=20150918/dataFile.csv]
Any idea? cheers
--
View this message in context: http://camel.465427.n5.nabble.com/smb-endpoint-modified-files-not-copied-to-destination-tp5770499p5771676.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: smb endpoint - modified files not copied to destination
Posted by hayden74 <hd...@gmail.com>.
Thanks Claus. Yes the file name will always be the same (eg:
inputFile.csv). Only the directory name is dynamic. I will give it a go
tomorrow at work. Thanks for you help.
On Mon, Aug 10, 2015 at 7:46 PM, Claus Ibsen-2 [via Camel] <
ml-node+s465427n5770522h79@n5.nabble.com> wrote:
> Hi
>
> Yeah you can filter by fileName actually. Though this would require
> the file to be named inputFile.csv
>
> On Mon, Aug 10, 2015 at 10:45 AM, hayden74 <[hidden email]
> <http:///user/SendEmail.jtp?type=node&node=5770522&i=0>> wrote:
>
> > So basically change the route to:
> >
> >
> from("smb://networkDriveOne/"?noop=true&fileName=${date:now:ddMMyyyy}/inputFile.csv&recursive=true")
>
> > .to("smb://networkDriveTwo?fileExist=Override")
> >
> >
> >
> > --
> > View this message in context:
> http://camel.465427.n5.nabble.com/smb-endpoint-modified-files-not-copied-to-destination-tp5770499p5770521.html
> > Sent from the Camel - Users mailing list archive at Nabble.com.
>
>
>
> --
> Claus Ibsen
> -----------------
> http://davsclaus.com @davsclaus
> Camel in Action 2nd edition: http://www.manning.com/ibsen2
>
>
> ------------------------------
> If you reply to this email, your message will be added to the discussion
> below:
>
> http://camel.465427.n5.nabble.com/smb-endpoint-modified-files-not-copied-to-destination-tp5770499p5770522.html
> To unsubscribe from smb endpoint - modified files not copied to
> destination, click here
> <http://camel.465427.n5.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=5770499&code=aGRyZDc0QGdtYWlsLmNvbXw1NzcwNDk5fC0xODg5Nzg4OTI4>
> .
> NAML
> <http://camel.465427.n5.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>
--
View this message in context: http://camel.465427.n5.nabble.com/smb-endpoint-modified-files-not-copied-to-destination-tp5770499p5770523.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: smb endpoint - modified files not copied to destination
Posted by Claus Ibsen <cl...@gmail.com>.
Hi
Yeah you can filter by fileName actually. Though this would require
the file to be named inputFile.csv
On Mon, Aug 10, 2015 at 10:45 AM, hayden74 <hd...@gmail.com> wrote:
> So basically change the route to:
>
> from("smb://networkDriveOne/"?noop=true&fileName=${date:now:ddMMyyyy}/inputFile.csv&recursive=true")
> .to("smb://networkDriveTwo?fileExist=Override")
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/smb-endpoint-modified-files-not-copied-to-destination-tp5770499p5770521.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
--
Claus Ibsen
-----------------
http://davsclaus.com @davsclaus
Camel in Action 2nd edition: http://www.manning.com/ibsen2
Re: smb endpoint - modified files not copied to destination
Posted by hayden74 <hd...@gmail.com>.
So basically change the route to:
from("smb://networkDriveOne/"?noop=true&fileName=${date:now:ddMMyyyy}/inputFile.csv&recursive=true")
.to("smb://networkDriveTwo?fileExist=Override")
--
View this message in context: http://camel.465427.n5.nabble.com/smb-endpoint-modified-files-not-copied-to-destination-tp5770499p5770521.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: smb endpoint - modified files not copied to destination
Posted by Claus Ibsen <cl...@gmail.com>.
No you cannot do dynamic inputs. However you can input from the root
folder where those dirs are, and use the filter option to filter to
only include the directory of todays date. Then you need to use the
recursive=true option. And you can use maxDeep to limit going down too
far.
For the filer option you would need to write a bit of java code to
decide whether to include the file / directory or not.
On Mon, Aug 10, 2015 at 9:49 AM, hayden74 <hd...@gmail.com> wrote:
> Many thanks Claus. I've tried to simply the scenario and ended up asking the
> wrong question. My bad.
>
> The input directory in my scenario changes daily based on the date. So the
> route looks like this:
>
> string todaysDateStr = "01-01-2015"; //this value changes based on the
> current date
> from("smb://networkDriveOne/"+ todaysDateStr
> +"?noop=true&fileName=inputFile.csv")
> .to("smb://networkDriveTwo?fileExist=Override")
>
> So everyday there will be a new input directory with new inputFile.csv in
> it. Camel, however, doesn't copy the file unless I restart it.
> Does Camel handle dynamic input directories?
>
> Moving/deleting files is not an option as they are used by other legacy
> systems.
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/smb-endpoint-modified-files-not-copied-to-destination-tp5770499p5770515.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
--
Claus Ibsen
-----------------
http://davsclaus.com @davsclaus
Camel in Action 2nd edition: http://www.manning.com/ibsen2
Re: smb endpoint - modified files not copied to destination
Posted by hayden74 <hd...@gmail.com>.
Many thanks Claus. I've tried to simply the scenario and ended up asking the
wrong question. My bad.
The input directory in my scenario changes daily based on the date. So the
route looks like this:
string todaysDateStr = "01-01-2015"; //this value changes based on the
current date
from("smb://networkDriveOne/"+ todaysDateStr
+"?noop=true&fileName=inputFile.csv")
.to("smb://networkDriveTwo?fileExist=Override")
So everyday there will be a new input directory with new inputFile.csv in
it. Camel, however, doesn't copy the file unless I restart it.
Does Camel handle dynamic input directories?
Moving/deleting files is not an option as they are used by other legacy
systems.
--
View this message in context: http://camel.465427.n5.nabble.com/smb-endpoint-modified-files-not-copied-to-destination-tp5770499p5770515.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: smb endpoint - modified files not copied to destination
Posted by Claus Ibsen <cl...@gmail.com>.
That is expected, noop=true means do not copy the file ever again. You
need to use an idempotent repository where you store the key with the
file name and timestamp so the repo can know if its the same file or a
changed file.
btw its often much better to either delete or move the files after you
have processed the file, instead of leaving them as-is.
On Sun, Aug 9, 2015 at 11:18 PM, hayden74 <hd...@gmail.com> wrote:
> Hi Camel experts.
> The issue I am facing is that Camel doesn't seem to copy a modified file
> after being copied the first time to its destination. The route used is:
>
> from("smb://networkDriveOne?noop=true&fileName=inputFile.csv")
> .to("smb://networkDriveTwo?fileExist=Override")
>
> When Camel is restarted, the inputFile.csv file is copied to its destination
> as expected. When inputFile.csv modified and saved in the input directory,
> however, Camel doesn't pick it up and override the one in the destination.
>
> Versions:
>
> Camel: 2.14.0
>
> ActiveMQ: 5.11.0
>
> Spring MVC: 4.1.6.RELEASE
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/smb-endpoint-modified-files-not-copied-to-destination-tp5770499.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
--
Claus Ibsen
-----------------
http://davsclaus.com @davsclaus
Camel in Action 2nd edition: http://www.manning.com/ibsen2