You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by mikaelfj <mi...@gmail.com> on 2010/12/16 11:05:08 UTC
Problem renaming existing file in FTP
Hi,
We are delivering files to an FTP server with the URI option
tempPrefix=.uploading and nothing set for the fileExists option, i.e. it
will default to override of an existing file.
However, if uploading, say, "file.txt" twice, the second upload will fail
with a rename error - as the renaming from ".uploadingfile.txt" to
"file.txt" fails as "file.txt" already exists on the FTP server.
My expectations would be, that the fileExists default to override should
apply in this case, and somehow force the ftp server to overwrite the
existing file in the renaming.
It looks like the FTP servers do not provide a common implementation of
rename - but then Camel could, e.g. delete the target file and retry the
rename if fileexist is override.
(We are running this on Camel 2.2)
Any ideas on how to overcome this?
/Mikael
--
View this message in context: http://camel.465427.n5.nabble.com/Problem-renaming-existing-file-in-FTP-tp3307670p3307670.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Problem renaming existing file in FTP
Posted by mikaelfj <mi...@gmail.com>.
Hi,
Hmmm - we found that this doesn't work when camel is runing on unix and the
ftp-server is windows:
If existing is aa/bb/cc\test.txt and OS (where camel runs) is windows then
normalizePath(existing) = aa\bb\cc\test.txt (i.e. fil separators are
normalized)
and stripPath(normalizePath(existing)) = test.txt (it takes substring after
last position of / or File.separator=\)
If existing is aa/bb/cc\test.txt and OS (where camel runs) is NOT windows
then
normalizePath(existing) = aa/bb/cc\test.txt (i.e. unchanged)
og stripPath(normalizePath(existing)) = Test\test.txt (it takes substring
after last position of / or File.separator=\)
Instead we are now using the following method added to the FtpOperations -
// Will return substring from last occurence of either / or \ - works as
long as / or \ are not part of the file name
public static String onlyFileName(String name){
char unixSep = '/';
char winSep = '\\';
if (name == null) {
return null;
}
int unixPos = name.lastIndexOf(unixSep);
int winPos = name.lastIndexOf(winSep);
if (unixPos > winPos) {
return name.substring(unixPos + 1);
}
if (winPos > unixPos) {
return name.substring(winPos + 1);
}
return name;
}
I have updated the jira-issue accordingly.
Regards
Mikael
--
View this message in context: http://camel.465427.n5.nabble.com/Problem-renaming-existing-file-in-FTP-tp3307670p3363358.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Problem renaming existing file in FTP
Posted by mikaelfj <mi...@gmail.com>.
Hi,
Got a chance to look into this again.
We still have the problem described, but just on a few destination servers
which all seems to be windows machines(?).
I have fetched the 2.4 branch of camel-ftp to find out why it does not work,
since the unit test you refer to in your previous post do excercise the
problem.
It appears that the cause of the problem is, that the
FtpOperations.existsFile fails to return true, which then causes the code
not to enter the part where it deletes the exisiting file.
The reason for the FtpOperations.existsFile to return false, is, that the
list of existing files has the directoryname prepended, e.g. I'm trying to
upload Test.REN to folder test/rename, and in the code
"if (existing.equals(onlyName))"
existing=test/rename\Test.REN and onlyName=Test.REN - which returns false -
but causes the later rename to fail as the target folder did contain the
Test.REN.
My guess is that it is server dependent whether the returned list of names
in the directory include the directory name or not?
To fix this I have added the following line of code before the if-statement
existing = FileUtil.stripPath(FileUtil.normalizePath(existing));
So it will strip the path part of the file name before comparing it to
onlyName.
I've added a jira for this
(https://issues.apache.org/jira/browse/CAMEL-3595), including a patch.
Regards
Mikael
--
View this message in context: http://camel.465427.n5.nabble.com/Problem-renaming-existing-file-in-FTP-tp3307670p3359761.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Problem renaming existing file in FTP
Posted by Claus Ibsen <cl...@gmail.com>.
I have created some unit tests and cannot reproduce this issue.
http://svn.apache.org/viewvc?view=revision&revision=1050041
Camel will delete any existing files.
There is even an option eagerDeleteTargetFile you can decide to do
this eager (= asap) or later (after writing the temporary file).
On Thu, Dec 16, 2010 at 12:49 PM, mikaelfj <mi...@gmail.com> wrote:
>
> Just tried with 2.4 and 2.5 - they both seem to have the same problem as 2.2.
>
> At the second upload of the file to the FTP server, they throw this
> exception:
>
> org.apache.camel.component.file.GenericFileOperationFailedException,
> message:Cannot rename file from: rename/.uploadingTest.REN to:
> rename/Test.REN
>
> I.e. having fileExists=override and tempPrefix=.uploading conflicts as the
> rename of the tempPrefix'ed file fails and thus do not deliver the
> "override" semantics - which should have provided that the file
> rename/Test.REN was replaced with the new Test.REN.
>
> Is this in the pipeline for 2.6?
>
> Regards
> Mikael
> --
> View this message in context: http://camel.465427.n5.nabble.com/Problem-renaming-existing-file-in-FTP-tp3307670p3307802.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
--
Claus Ibsen
-----------------
FuseSource
Email: cibsen@fusesource.com
Web: http://fusesource.com
Twitter: davsclaus
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/
Re: Problem renaming existing file in FTP
Posted by Claus Ibsen <cl...@gmail.com>.
On Thu, Dec 16, 2010 at 12:49 PM, mikaelfj <mi...@gmail.com> wrote:
>
> Just tried with 2.4 and 2.5 - they both seem to have the same problem as 2.2.
>
> At the second upload of the file to the FTP server, they throw this
> exception:
>
> org.apache.camel.component.file.GenericFileOperationFailedException,
> message:Cannot rename file from: rename/.uploadingTest.REN to:
> rename/Test.REN
>
> I.e. having fileExists=override and tempPrefix=.uploading conflicts as the
> rename of the tempPrefix'ed file fails and thus do not deliver the
> "override" semantics - which should have provided that the file
> rename/Test.REN was replaced with the new Test.REN.
>
> Is this in the pipeline for 2.6?
>
Could you create a ticket in JIRA. And link to this discussion using a
link from eg. nabble.
Also you are welcome to dig in and see if you can provide a patch.
> Regards
> Mikael
> --
> View this message in context: http://camel.465427.n5.nabble.com/Problem-renaming-existing-file-in-FTP-tp3307670p3307802.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
--
Claus Ibsen
-----------------
FuseSource
Email: cibsen@fusesource.com
Web: http://fusesource.com
Twitter: davsclaus
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/
Re: Problem renaming existing file in FTP
Posted by mikaelfj <mi...@gmail.com>.
Just tried with 2.4 and 2.5 - they both seem to have the same problem as 2.2.
At the second upload of the file to the FTP server, they throw this
exception:
org.apache.camel.component.file.GenericFileOperationFailedException,
message:Cannot rename file from: rename/.uploadingTest.REN to:
rename/Test.REN
I.e. having fileExists=override and tempPrefix=.uploading conflicts as the
rename of the tempPrefix'ed file fails and thus do not deliver the
"override" semantics - which should have provided that the file
rename/Test.REN was replaced with the new Test.REN.
Is this in the pipeline for 2.6?
Regards
Mikael
--
View this message in context: http://camel.465427.n5.nabble.com/Problem-renaming-existing-file-in-FTP-tp3307670p3307802.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Problem renaming existing file in FTP
Posted by mikaelfj <mi...@gmail.com>.
Hi - I looked through the release notes but couldn't see this mentioned
specifically - but I will give a later release a go - thanks
/Mikael
--
View this message in context: http://camel.465427.n5.nabble.com/Problem-renaming-existing-file-in-FTP-tp3307670p3307746.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Problem renaming existing file in FTP
Posted by Claus Ibsen <cl...@gmail.com>.
Hi
Try with a later release as the FTP component have been improved on
the times. I think we have fixes something similar like this in a
later release.
On Thu, Dec 16, 2010 at 11:05 AM, mikaelfj <mi...@gmail.com> wrote:
>
> Hi,
>
> We are delivering files to an FTP server with the URI option
> tempPrefix=.uploading and nothing set for the fileExists option, i.e. it
> will default to override of an existing file.
>
> However, if uploading, say, "file.txt" twice, the second upload will fail
> with a rename error - as the renaming from ".uploadingfile.txt" to
> "file.txt" fails as "file.txt" already exists on the FTP server.
>
> My expectations would be, that the fileExists default to override should
> apply in this case, and somehow force the ftp server to overwrite the
> existing file in the renaming.
>
> It looks like the FTP servers do not provide a common implementation of
> rename - but then Camel could, e.g. delete the target file and retry the
> rename if fileexist is override.
>
> (We are running this on Camel 2.2)
>
> Any ideas on how to overcome this?
>
> /Mikael
> --
> View this message in context: http://camel.465427.n5.nabble.com/Problem-renaming-existing-file-in-FTP-tp3307670p3307670.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
--
Claus Ibsen
-----------------
FuseSource
Email: cibsen@fusesource.com
Web: http://fusesource.com
Twitter: davsclaus
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/