You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by Lorrin Nelson <lh...@nerdylorrin.net> on 2010/10/01 22:45:57 UTC
Re: How to change directory while using sftp component
Hi Claus. Ack, sorry about that. I'm pulling through github rather than SVN directly and then running via a Maven project pointing at my locally built Camel, so it's sometimes confusing how up-to-date I am.
So I think I have current code, including checkin "CAMEL-3174: Changing dir with ftp must do one dir at a time." and I think SFTP polling is now broken for both absolute and relative URLs. See debug walk-throughs and stack traces below.
Absolute URI with double leading /:
sftp://user@host//home/user/subdir?idempotent=true&idempotentRepository=#myCustomIR&include=.*.log&noop=true&password=******
SftpOperations.changeCurrentDirectory("home/user/subdir") //note there's no leading slash on the supposedly absolute path!
//splits into chunks successfully, starts looping through chunks
doChangeDirectory("home")
ChannelSftp.cd("home")
//path=remoteAbsolutePath(path);
//yields "/home/user/home". Doh!
Relative URI with single leading /:
sftp://user@host/subdir?idempotent=true&idempotentRepository=#myCustomIR&include=.*.log&noop=true&password=******
SftpOperations.changeCurrentDirectory("subdir")
//splits into single chunk, starts looping through chunks
doChangeDirectory("subdir")
ChannelSftp.cd("home")
//path=remoteAbsolutePath(path);
//yields "/home/user/subdir". yay!
..but then, back in SftpOperations.retrieveFileToStreamInBody, it hits the "change back to current directory code"
changeCurrentDirectory("/home/tomcat"); //now we have a leading slash, unlike before
//splits info chunks "", "home", and "tomcat". Doh!
doChangeDirectory("")
ChannelSftp.cd("")
//path=remoteAbsolutePath(path) throws a StringIndexOutOfBoundsException when it receives empty string.
Stack trace when attempting to supply absolute path:
org.apache.camel.component.file.GenericFileOperationFailedException: Cannot change directory to: home
at org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:372)
at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:361)
at org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:428)
at org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:407)
at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299)
at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:637)
Caused by: 2: No such file
at com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2289)
at com.jcraft.jsch.ChannelSftp._realpath(ChannelSftp.java:1822)
at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:268)
at org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:370)
... 16 more
Stack trace when relative URL tries to restore current working directory:
SftpConsumer 2010-10-01 13:41:58,529 -- ERROR -- Caused by: [org.apache.camel.component.file.GenericFileOperationFailedException - Cannot change directory to: ]
org.apache.camel.component.file.GenericFileOperationFailedException: Cannot change directory to:
at org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:372)
at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:361)
at org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:435)
at org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:407)
at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299)
at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:637)
Caused by: 4:
at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:285)
at org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:370)
... 16 more
Caused by: java.lang.StringIndexOutOfBoundsException: String index out of range: 0
at java.lang.String.charAt(String.java:686)
at com.jcraft.jsch.ChannelSftp.remoteAbsolutePath(ChannelSftp.java:2359)
at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:264)
... 17 more
Cheers
-Lorrin
On Sep 29, 2010, at 9:37 PM, Claus Ibsen wrote:
> Hi
>
> You have not used the latest source code. The stacktrace points to
> empty lines etc.
> Make sure you build camel-core and camel-ftp using latest source code.
>
> Revision: 1002946
> Node Kind: directory
> Schedule: normal
> Last Changed Author: davsclaus
> Last Changed Rev: 1002821
> Last Changed Date: 2010-09-29 21:31:23 +0200 (Wed, 29 Sep 2010)
>
>
> On Wed, Sep 29, 2010 at 11:55 PM, Lorrin Nelson
> <lh...@nerdylorrin.net> wrote:
>> I'm pretty sure I was already at revision 1002541 before. Now I definitely am. Same Exception. I think the problem is the lack of leading /.
>>
>> SftpOperations.retrieveFileToStreamInBody computes path as "tmp/mult", calls changeCurrentDirectory(path), which in turn calls channel.cd(path), which (now we're in com.jcraft.jsch.ChannelSftp) calls path=remoteAbsolutePath(path); and computes path: "/home/test/tmp/mult", which does not exist.
>>
>> SftpConsumer 2010-09-29 14:51:50,487 -- ERROR -- Caused by: [org.apache.camel.component.file.GenericFileOperationFailedException - Cannot change current directory to: tmp/mult]
>> org.apache.camel.component.file.GenericFileOperationFailedException: Cannot change current directory to: tmp/mult
>> at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:352)
>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:408)
>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:387)
>> at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299)
>> at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
>> at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
>> at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>> at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>> at java.lang.Thread.run(Thread.java:637)
>> Caused by: 2: No such file
>> at com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2289)
>> at com.jcraft.jsch.ChannelSftp._realpath(ChannelSftp.java:1822)
>> at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:268)
>> at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:350)
>> ... 15 more
>>
>>
>> Cheers!
>> -Lorrin
>>
>> On Sep 29, 2010, at 12:32 PM, Claus Ibsen wrote:
>>
>>> Hi
>>>
>>> Can you try again with latest source code from trunk. I have changed
>>> the logic to change dir one folder at a time.
>>>
>>>
>>> On Wed, Sep 29, 2010 at 7:47 PM, Lorrin Nelson
>>> <lh...@nerdylorrin.net> wrote:
>>>> Thanks for the quick response. I'm seeing a different failure now ("Cannot change current directory to: tmp/mult" rather than "Cannot retrieve file: tmp/mult/dummy-5.log"). Is it suspicious that in either case there is no leading /?
>>>>
>>>> I tried uris
>>>> sftp://test@host//tmp/mult?include=.*\.log
>>>> and
>>>> sftp://test@host///tmp/mult?include=.*\.log
>>>>
>>>> SftpConsumer 2010-09-29 10:31:48,401 -- ERROR -- Caused by: [org.apache.camel.component.file.GenericFileOperationFailedException - Cannot change current directory to: tmp/mult]
>>>> org.apache.camel.component.file.GenericFileOperationFailedException: Cannot change current directory to: tmp/mult
>>>> at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:352)
>>>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:408)
>>>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:387)
>>>> at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299)
>>>> at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
>>>> at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
>>>> at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
>>>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>>>> at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>> at java.lang.Thread.run(Thread.java:637)
>>>> Caused by: 2: No such file
>>>> at com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2289)
>>>> at com.jcraft.jsch.ChannelSftp._realpath(ChannelSftp.java:1822)
>>>> at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:268)
>>>> at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:350)
>>>> ... 15 more
>>>>
>>>> On a tangentially related note, there's some file name processing during the move step that throws a NullPointerException when the URI contains a trailing slash:
>>>>
>>>> sftp://test@host/tmp/mult/?include=.*\.log
>>>>
>>>> The problem is that GenericFile.changeFileName calls
>>>> newFileName = ObjectHelper.after(newFileName, endpointPath + getFileSeparator());
>>>> ...without first checking if endPointPath contains a trailing separator.
>>>>
>>>> E.g.
>>>> newFileName: "tmp/mult/.done/dummy-2.log"
>>>> endPointPath: "tmp/mult/"
>>>>
>>>> ObjectHelper.after():
>>>> text: "tmp/mult/.done/dummy-2.log"
>>>> after: "tmp/mult//"
>>>>
>>>> text doesn't contain after (because of double //), returns null, changeFileName() doesn't expect null, throws NPE.
>>>>
>>>> GenericFileOnCompletion 2010-09-29 10:37:37,888 -- ERROR -- Caused by: [java.lang.NullPointerException - null]
>>>> java.lang.NullPointerException
>>>> at java.io.File.<init>(File.java:222)
>>>> at org.apache.camel.component.file.GenericFile.changeFileName(GenericFile.java:169)
>>>> at org.apache.camel.component.file.strategy.GenericFileExpressionRenamer.renameFile(GenericFileExpressionRenamer.java:41)
>>>> at org.apache.camel.component.file.strategy.GenericFileRenameProcessStrategy.commit(GenericFileRenameProcessStrategy.java:82)
>>>> at org.apache.camel.component.file.GenericFileOnCompletion.processStrategyCommit(GenericFileOnCompletion.java:121)
>>>> at org.apache.camel.component.file.GenericFileOnCompletion.onCompletion(GenericFileOnCompletion.java:83)
>>>> at org.apache.camel.component.file.GenericFileOnCompletion.onComplete(GenericFileOnCompletion.java:52)
>>>> at org.apache.camel.util.UnitOfWorkHelper.doneSynchronizations(UnitOfWorkHelper.java:55)
>>>> at org.apache.camel.impl.DefaultUnitOfWork.done(DefaultUnitOfWork.java:173)
>>>> at org.apache.camel.processor.UnitOfWorkProcessor.doneUow(UnitOfWorkProcessor.java:121)
>>>> at org.apache.camel.processor.UnitOfWorkProcessor.access$000(UnitOfWorkProcessor.java:36)
>>>> at org.apache.camel.processor.UnitOfWorkProcessor$1.done(UnitOfWorkProcessor.java:106)
>>>> at org.apache.camel.processor.Pipeline.process(Pipeline.java:130)
>>>> at org.apache.camel.processor.UnitOfWorkProcessor.process(UnitOfWorkProcessor.java:99)
>>>> at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:70)
>>>> at org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:98)
>>>> at org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:89)
>>>> at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:68)
>>>> at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:322)
>>>> at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
>>>> at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
>>>> at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
>>>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>>>> at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>> at java.lang.Thread.run(Thread.java:637)
>>>>
>>>>
>>>>
>>>>
>>>> On Sep 29, 2010, at 2:46 AM, Claus Ibsen wrote:
>>>>
>>>>> Hi
>>>>>
>>>>> I have committed a fix to the SFTP component. Can you try with latest
>>>>> source code from trunk?
>>>>>
>>>>>
>>>>> On Tue, Sep 28, 2010 at 7:36 PM, Lorrin <lh...@nerdylorrin.net> wrote:
>>>>>>
>>>>>> I just bumped into the same problem. I'm running 2.5-SNAPSHOT.
>>>>>>
>>>>>> SftpConsumer 2010-09-28 10:31:28,094 -- ERROR -- Caused by:
>>>>>> [org.apache.camel.component.file.GenericFileOperationFailedException -
>>>>>> Cannot retrieve file: tmp/mult/dummy-5.log]
>>>>>> org.apache.camel.component.file.GenericFileOperationFailedException: Cannot
>>>>>> retrieve file: tmp/mult/dummy-5.log
>>>>>> at
>>>>>> org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:403)
>>>>>> at
>>>>>> org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:387)
>>>>>> at
>>>>>> org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299)
>>>>>> at
>>>>>> org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
>>>>>> at
>>>>>> org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
>>>>>> at
>>>>>> org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
>>>>>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>>>>>> at
>>>>>> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>>>>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>>>>>> at
>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>>>>>> at
>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
>>>>>> at
>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
>>>>>> at
>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>>>> at
>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>>>> at java.lang.Thread.run(Thread.java:637)
>>>>>> Caused by: 2: No such file
>>>>>> at com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2289)
>>>>>> at com.jcraft.jsch.ChannelSftp._get(ChannelSftp.java:901)
>>>>>> at com.jcraft.jsch.ChannelSftp.get(ChannelSftp.java:870)
>>>>>> at com.jcraft.jsch.ChannelSftp.get(ChannelSftp.java:849)
>>>>>> at
>>>>>> org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:400)
>>>>>> ... 14 more
>>>>>>
>>>>>> --
>>>>>> View this message in context: http://camel.465427.n5.nabble.com/How-to-change-directory-while-using-sftp-component-tp2806817p2857187.html
>>>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Claus Ibsen
>>>>> Apache Camel Committer
>>>>>
>>>>> Author of Camel in Action: http://www.manning.com/ibsen/
>>>>> Open Source Integration: http://fusesource.com
>>>>> Blog: http://davsclaus.blogspot.com/
>>>>> Twitter: http://twitter.com/davsclaus
>>>>>
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Claus Ibsen
>>> Apache Camel Committer
>>>
>>> Author of Camel in Action: http://www.manning.com/ibsen/
>>> Open Source Integration: http://fusesource.com
>>> Blog: http://davsclaus.blogspot.com/
>>> Twitter: http://twitter.com/davsclaus
>>>
>>
>>
>
>
>
> --
> Claus Ibsen
> Apache Camel Committer
>
> Author of Camel in Action: http://www.manning.com/ibsen/
> Open Source Integration: http://fusesource.com
> Blog: http://davsclaus.blogspot.com/
> Twitter: http://twitter.com/davsclaus
>
Re: How to change directory while using sftp component
Posted by Claus Ibsen <cl...@gmail.com>.
Hi
I will try to reproduce the issue with the help of Apache SSHD as
simulating the SFTP server. It's not like a real server but it may
help and in the future catch regression bugs.
I got the first simple test up and running by uploading a file.
On Sat, Oct 2, 2010 at 4:50 PM, Claus Ibsen <cl...@gmail.com> wrote:
> On Sat, Oct 2, 2010 at 9:14 AM, Claus Ibsen <cl...@gmail.com> wrote:
>> Hi Loririn
>>
>> I think you/we have to step back. We have other users in the past
>> running with SFTP with no problems at all. Also fetching files
>> recursively.
>> So I wonder if we should revert back and keep the old logic!
>
> Btw this didn't come out right as I wanted to say. We should find a
> solution that works for everybody.
> file based data exchange is actually harder than it looks. There are
> so many OS and subtle quirks that can cause hickups.
> And apparently also SFTP servers which deny you to download files
> using paths in the file name :)
>
>
>
>>
>> 1)
>> Is this the first time you use camel-ftp?
>>
>> 2)
>> What OS and FTP server are you using?
>>
>> 3)
>> Can you use regular FTP instead?
>>
>> 4)
>> Use relative urls over absolute. They where the first supported.
>>
>> 5)
>> Can you enable TRACE logging on org.apache.camel.component.file and
>> setup a simple example so it makes it easier to follow and understand
>> what its supposed to do, and what happens in reality so we can see and
>> understand the difference.
>>
>>
>>
>>
>> On Fri, Oct 1, 2010 at 10:45 PM, Lorrin Nelson
>> <lh...@nerdylorrin.net> wrote:
>>> Hi Claus. Ack, sorry about that. I'm pulling through github rather than SVN directly and then running via a Maven project pointing at my locally built Camel, so it's sometimes confusing how up-to-date I am.
>>>
>>> So I think I have current code, including checkin "CAMEL-3174: Changing dir with ftp must do one dir at a time." and I think SFTP polling is now broken for both absolute and relative URLs. See debug walk-throughs and stack traces below.
>>>
>>> Absolute URI with double leading /:
>>> sftp://user@host//home/user/subdir?idempotent=true&idempotentRepository=#myCustomIR&include=.*.log&noop=true&password=******
>>>
>>> SftpOperations.changeCurrentDirectory("home/user/subdir") //note there's no leading slash on the supposedly absolute path!
>>> //splits into chunks successfully, starts looping through chunks
>>> doChangeDirectory("home")
>>> ChannelSftp.cd("home")
>>> //path=remoteAbsolutePath(path);
>>> //yields "/home/user/home". Doh!
>>>
>>>
>>> Relative URI with single leading /:
>>> sftp://user@host/subdir?idempotent=true&idempotentRepository=#myCustomIR&include=.*.log&noop=true&password=******
>>> SftpOperations.changeCurrentDirectory("subdir")
>>> //splits into single chunk, starts looping through chunks
>>> doChangeDirectory("subdir")
>>> ChannelSftp.cd("home")
>>> //path=remoteAbsolutePath(path);
>>> //yields "/home/user/subdir". yay!
>>>
>>> ..but then, back in SftpOperations.retrieveFileToStreamInBody, it hits the "change back to current directory code"
>>> changeCurrentDirectory("/home/tomcat"); //now we have a leading slash, unlike before
>>> //splits info chunks "", "home", and "tomcat". Doh!
>>> doChangeDirectory("")
>>> ChannelSftp.cd("")
>>> //path=remoteAbsolutePath(path) throws a StringIndexOutOfBoundsException when it receives empty string.
>>>
>>>
>>> Stack trace when attempting to supply absolute path:
>>> org.apache.camel.component.file.GenericFileOperationFailedException: Cannot change directory to: home
>>> at org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:372)
>>> at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:361)
>>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:428)
>>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:407)
>>> at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299)
>>> at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
>>> at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
>>> at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
>>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>>> at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>> at java.lang.Thread.run(Thread.java:637)
>>> Caused by: 2: No such file
>>> at com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2289)
>>> at com.jcraft.jsch.ChannelSftp._realpath(ChannelSftp.java:1822)
>>> at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:268)
>>> at org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:370)
>>> ... 16 more
>>>
>>>
>>> Stack trace when relative URL tries to restore current working directory:
>>> SftpConsumer 2010-10-01 13:41:58,529 -- ERROR -- Caused by: [org.apache.camel.component.file.GenericFileOperationFailedException - Cannot change directory to: ]
>>> org.apache.camel.component.file.GenericFileOperationFailedException: Cannot change directory to:
>>> at org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:372)
>>> at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:361)
>>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:435)
>>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:407)
>>> at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299)
>>> at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
>>> at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
>>> at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
>>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>>> at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>> at java.lang.Thread.run(Thread.java:637)
>>> Caused by: 4:
>>> at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:285)
>>> at org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:370)
>>> ... 16 more
>>> Caused by: java.lang.StringIndexOutOfBoundsException: String index out of range: 0
>>> at java.lang.String.charAt(String.java:686)
>>> at com.jcraft.jsch.ChannelSftp.remoteAbsolutePath(ChannelSftp.java:2359)
>>> at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:264)
>>> ... 17 more
>>>
>>> Cheers
>>> -Lorrin
>>>
>>> On Sep 29, 2010, at 9:37 PM, Claus Ibsen wrote:
>>>
>>>> Hi
>>>>
>>>> You have not used the latest source code. The stacktrace points to
>>>> empty lines etc.
>>>> Make sure you build camel-core and camel-ftp using latest source code.
>>>>
>>>> Revision: 1002946
>>>> Node Kind: directory
>>>> Schedule: normal
>>>> Last Changed Author: davsclaus
>>>> Last Changed Rev: 1002821
>>>> Last Changed Date: 2010-09-29 21:31:23 +0200 (Wed, 29 Sep 2010)
>>>>
>>>>
>>>> On Wed, Sep 29, 2010 at 11:55 PM, Lorrin Nelson
>>>> <lh...@nerdylorrin.net> wrote:
>>>>> I'm pretty sure I was already at revision 1002541 before. Now I definitely am. Same Exception. I think the problem is the lack of leading /.
>>>>>
>>>>> SftpOperations.retrieveFileToStreamInBody computes path as "tmp/mult", calls changeCurrentDirectory(path), which in turn calls channel.cd(path), which (now we're in com.jcraft.jsch.ChannelSftp) calls path=remoteAbsolutePath(path); and computes path: "/home/test/tmp/mult", which does not exist.
>>>>>
>>>>> SftpConsumer 2010-09-29 14:51:50,487 -- ERROR -- Caused by: [org.apache.camel.component.file.GenericFileOperationFailedException - Cannot change current directory to: tmp/mult]
>>>>> org.apache.camel.component.file.GenericFileOperationFailedException: Cannot change current directory to: tmp/mult
>>>>> at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:352)
>>>>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:408)
>>>>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:387)
>>>>> at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299)
>>>>> at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
>>>>> at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
>>>>> at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
>>>>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>>>>> at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>>>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>>> at java.lang.Thread.run(Thread.java:637)
>>>>> Caused by: 2: No such file
>>>>> at com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2289)
>>>>> at com.jcraft.jsch.ChannelSftp._realpath(ChannelSftp.java:1822)
>>>>> at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:268)
>>>>> at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:350)
>>>>> ... 15 more
>>>>>
>>>>>
>>>>> Cheers!
>>>>> -Lorrin
>>>>>
>>>>> On Sep 29, 2010, at 12:32 PM, Claus Ibsen wrote:
>>>>>
>>>>>> Hi
>>>>>>
>>>>>> Can you try again with latest source code from trunk. I have changed
>>>>>> the logic to change dir one folder at a time.
>>>>>>
>>>>>>
>>>>>> On Wed, Sep 29, 2010 at 7:47 PM, Lorrin Nelson
>>>>>> <lh...@nerdylorrin.net> wrote:
>>>>>>> Thanks for the quick response. I'm seeing a different failure now ("Cannot change current directory to: tmp/mult" rather than "Cannot retrieve file: tmp/mult/dummy-5.log"). Is it suspicious that in either case there is no leading /?
>>>>>>>
>>>>>>> I tried uris
>>>>>>> sftp://test@host//tmp/mult?include=.*\.log
>>>>>>> and
>>>>>>> sftp://test@host///tmp/mult?include=.*\.log
>>>>>>>
>>>>>>> SftpConsumer 2010-09-29 10:31:48,401 -- ERROR -- Caused by: [org.apache.camel.component.file.GenericFileOperationFailedException - Cannot change current directory to: tmp/mult]
>>>>>>> org.apache.camel.component.file.GenericFileOperationFailedException: Cannot change current directory to: tmp/mult
>>>>>>> at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:352)
>>>>>>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:408)
>>>>>>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:387)
>>>>>>> at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299)
>>>>>>> at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
>>>>>>> at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
>>>>>>> at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
>>>>>>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>>>>>>> at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>>>>>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>>>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>>>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
>>>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
>>>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>>>>> at java.lang.Thread.run(Thread.java:637)
>>>>>>> Caused by: 2: No such file
>>>>>>> at com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2289)
>>>>>>> at com.jcraft.jsch.ChannelSftp._realpath(ChannelSftp.java:1822)
>>>>>>> at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:268)
>>>>>>> at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:350)
>>>>>>> ... 15 more
>>>>>>>
>>>>>>> On a tangentially related note, there's some file name processing during the move step that throws a NullPointerException when the URI contains a trailing slash:
>>>>>>>
>>>>>>> sftp://test@host/tmp/mult/?include=.*\.log
>>>>>>>
>>>>>>> The problem is that GenericFile.changeFileName calls
>>>>>>> newFileName = ObjectHelper.after(newFileName, endpointPath + getFileSeparator());
>>>>>>> ...without first checking if endPointPath contains a trailing separator.
>>>>>>>
>>>>>>> E.g.
>>>>>>> newFileName: "tmp/mult/.done/dummy-2.log"
>>>>>>> endPointPath: "tmp/mult/"
>>>>>>>
>>>>>>> ObjectHelper.after():
>>>>>>> text: "tmp/mult/.done/dummy-2.log"
>>>>>>> after: "tmp/mult//"
>>>>>>>
>>>>>>> text doesn't contain after (because of double //), returns null, changeFileName() doesn't expect null, throws NPE.
>>>>>>>
>>>>>>> GenericFileOnCompletion 2010-09-29 10:37:37,888 -- ERROR -- Caused by: [java.lang.NullPointerException - null]
>>>>>>> java.lang.NullPointerException
>>>>>>> at java.io.File.<init>(File.java:222)
>>>>>>> at org.apache.camel.component.file.GenericFile.changeFileName(GenericFile.java:169)
>>>>>>> at org.apache.camel.component.file.strategy.GenericFileExpressionRenamer.renameFile(GenericFileExpressionRenamer.java:41)
>>>>>>> at org.apache.camel.component.file.strategy.GenericFileRenameProcessStrategy.commit(GenericFileRenameProcessStrategy.java:82)
>>>>>>> at org.apache.camel.component.file.GenericFileOnCompletion.processStrategyCommit(GenericFileOnCompletion.java:121)
>>>>>>> at org.apache.camel.component.file.GenericFileOnCompletion.onCompletion(GenericFileOnCompletion.java:83)
>>>>>>> at org.apache.camel.component.file.GenericFileOnCompletion.onComplete(GenericFileOnCompletion.java:52)
>>>>>>> at org.apache.camel.util.UnitOfWorkHelper.doneSynchronizations(UnitOfWorkHelper.java:55)
>>>>>>> at org.apache.camel.impl.DefaultUnitOfWork.done(DefaultUnitOfWork.java:173)
>>>>>>> at org.apache.camel.processor.UnitOfWorkProcessor.doneUow(UnitOfWorkProcessor.java:121)
>>>>>>> at org.apache.camel.processor.UnitOfWorkProcessor.access$000(UnitOfWorkProcessor.java:36)
>>>>>>> at org.apache.camel.processor.UnitOfWorkProcessor$1.done(UnitOfWorkProcessor.java:106)
>>>>>>> at org.apache.camel.processor.Pipeline.process(Pipeline.java:130)
>>>>>>> at org.apache.camel.processor.UnitOfWorkProcessor.process(UnitOfWorkProcessor.java:99)
>>>>>>> at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:70)
>>>>>>> at org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:98)
>>>>>>> at org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:89)
>>>>>>> at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:68)
>>>>>>> at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:322)
>>>>>>> at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
>>>>>>> at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
>>>>>>> at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
>>>>>>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>>>>>>> at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>>>>>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>>>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>>>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
>>>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
>>>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>>>>> at java.lang.Thread.run(Thread.java:637)
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Sep 29, 2010, at 2:46 AM, Claus Ibsen wrote:
>>>>>>>
>>>>>>>> Hi
>>>>>>>>
>>>>>>>> I have committed a fix to the SFTP component. Can you try with latest
>>>>>>>> source code from trunk?
>>>>>>>>
>>>>>>>>
>>>>>>>> On Tue, Sep 28, 2010 at 7:36 PM, Lorrin <lh...@nerdylorrin.net> wrote:
>>>>>>>>>
>>>>>>>>> I just bumped into the same problem. I'm running 2.5-SNAPSHOT.
>>>>>>>>>
>>>>>>>>> SftpConsumer 2010-09-28 10:31:28,094 -- ERROR -- Caused by:
>>>>>>>>> [org.apache.camel.component.file.GenericFileOperationFailedException -
>>>>>>>>> Cannot retrieve file: tmp/mult/dummy-5.log]
>>>>>>>>> org.apache.camel.component.file.GenericFileOperationFailedException: Cannot
>>>>>>>>> retrieve file: tmp/mult/dummy-5.log
>>>>>>>>> at
>>>>>>>>> org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:403)
>>>>>>>>> at
>>>>>>>>> org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:387)
>>>>>>>>> at
>>>>>>>>> org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299)
>>>>>>>>> at
>>>>>>>>> org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
>>>>>>>>> at
>>>>>>>>> org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
>>>>>>>>> at
>>>>>>>>> org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
>>>>>>>>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>>>>>>>>> at
>>>>>>>>> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>>>>>>>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>>>>>>>>> at
>>>>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>>>>>>>>> at
>>>>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
>>>>>>>>> at
>>>>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
>>>>>>>>> at
>>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>>>>>>> at
>>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>>>>>>> at java.lang.Thread.run(Thread.java:637)
>>>>>>>>> Caused by: 2: No such file
>>>>>>>>> at com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2289)
>>>>>>>>> at com.jcraft.jsch.ChannelSftp._get(ChannelSftp.java:901)
>>>>>>>>> at com.jcraft.jsch.ChannelSftp.get(ChannelSftp.java:870)
>>>>>>>>> at com.jcraft.jsch.ChannelSftp.get(ChannelSftp.java:849)
>>>>>>>>> at
>>>>>>>>> org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:400)
>>>>>>>>> ... 14 more
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> View this message in context: http://camel.465427.n5.nabble.com/How-to-change-directory-while-using-sftp-component-tp2806817p2857187.html
>>>>>>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Claus Ibsen
>>>>>>>> Apache Camel Committer
>>>>>>>>
>>>>>>>> Author of Camel in Action: http://www.manning.com/ibsen/
>>>>>>>> Open Source Integration: http://fusesource.com
>>>>>>>> Blog: http://davsclaus.blogspot.com/
>>>>>>>> Twitter: http://twitter.com/davsclaus
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Claus Ibsen
>>>>>> Apache Camel Committer
>>>>>>
>>>>>> Author of Camel in Action: http://www.manning.com/ibsen/
>>>>>> Open Source Integration: http://fusesource.com
>>>>>> Blog: http://davsclaus.blogspot.com/
>>>>>> Twitter: http://twitter.com/davsclaus
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Claus Ibsen
>>>> Apache Camel Committer
>>>>
>>>> Author of Camel in Action: http://www.manning.com/ibsen/
>>>> Open Source Integration: http://fusesource.com
>>>> Blog: http://davsclaus.blogspot.com/
>>>> Twitter: http://twitter.com/davsclaus
>>>>
>>>
>>>
>>
>>
>>
>> --
>> Claus Ibsen
>> Apache Camel Committer
>>
>> Author of Camel in Action: http://www.manning.com/ibsen/
>> Open Source Integration: http://fusesource.com
>> Blog: http://davsclaus.blogspot.com/
>> Twitter: http://twitter.com/davsclaus
>>
>
>
>
> --
> Claus Ibsen
> Apache Camel Committer
>
> Author of Camel in Action: http://www.manning.com/ibsen/
> Open Source Integration: http://fusesource.com
> Blog: http://davsclaus.blogspot.com/
> Twitter: http://twitter.com/davsclaus
>
--
Claus Ibsen
Apache Camel Committer
Author of Camel in Action: http://www.manning.com/ibsen/
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus
Re: How to change directory while using sftp component
Posted by Claus Ibsen <cl...@gmail.com>.
On Sat, Oct 2, 2010 at 9:14 AM, Claus Ibsen <cl...@gmail.com> wrote:
> Hi Loririn
>
> I think you/we have to step back. We have other users in the past
> running with SFTP with no problems at all. Also fetching files
> recursively.
> So I wonder if we should revert back and keep the old logic!
Btw this didn't come out right as I wanted to say. We should find a
solution that works for everybody.
file based data exchange is actually harder than it looks. There are
so many OS and subtle quirks that can cause hickups.
And apparently also SFTP servers which deny you to download files
using paths in the file name :)
>
> 1)
> Is this the first time you use camel-ftp?
>
> 2)
> What OS and FTP server are you using?
>
> 3)
> Can you use regular FTP instead?
>
> 4)
> Use relative urls over absolute. They where the first supported.
>
> 5)
> Can you enable TRACE logging on org.apache.camel.component.file and
> setup a simple example so it makes it easier to follow and understand
> what its supposed to do, and what happens in reality so we can see and
> understand the difference.
>
>
>
>
> On Fri, Oct 1, 2010 at 10:45 PM, Lorrin Nelson
> <lh...@nerdylorrin.net> wrote:
>> Hi Claus. Ack, sorry about that. I'm pulling through github rather than SVN directly and then running via a Maven project pointing at my locally built Camel, so it's sometimes confusing how up-to-date I am.
>>
>> So I think I have current code, including checkin "CAMEL-3174: Changing dir with ftp must do one dir at a time." and I think SFTP polling is now broken for both absolute and relative URLs. See debug walk-throughs and stack traces below.
>>
>> Absolute URI with double leading /:
>> sftp://user@host//home/user/subdir?idempotent=true&idempotentRepository=#myCustomIR&include=.*.log&noop=true&password=******
>>
>> SftpOperations.changeCurrentDirectory("home/user/subdir") //note there's no leading slash on the supposedly absolute path!
>> //splits into chunks successfully, starts looping through chunks
>> doChangeDirectory("home")
>> ChannelSftp.cd("home")
>> //path=remoteAbsolutePath(path);
>> //yields "/home/user/home". Doh!
>>
>>
>> Relative URI with single leading /:
>> sftp://user@host/subdir?idempotent=true&idempotentRepository=#myCustomIR&include=.*.log&noop=true&password=******
>> SftpOperations.changeCurrentDirectory("subdir")
>> //splits into single chunk, starts looping through chunks
>> doChangeDirectory("subdir")
>> ChannelSftp.cd("home")
>> //path=remoteAbsolutePath(path);
>> //yields "/home/user/subdir". yay!
>>
>> ..but then, back in SftpOperations.retrieveFileToStreamInBody, it hits the "change back to current directory code"
>> changeCurrentDirectory("/home/tomcat"); //now we have a leading slash, unlike before
>> //splits info chunks "", "home", and "tomcat". Doh!
>> doChangeDirectory("")
>> ChannelSftp.cd("")
>> //path=remoteAbsolutePath(path) throws a StringIndexOutOfBoundsException when it receives empty string.
>>
>>
>> Stack trace when attempting to supply absolute path:
>> org.apache.camel.component.file.GenericFileOperationFailedException: Cannot change directory to: home
>> at org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:372)
>> at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:361)
>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:428)
>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:407)
>> at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299)
>> at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
>> at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
>> at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>> at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>> at java.lang.Thread.run(Thread.java:637)
>> Caused by: 2: No such file
>> at com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2289)
>> at com.jcraft.jsch.ChannelSftp._realpath(ChannelSftp.java:1822)
>> at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:268)
>> at org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:370)
>> ... 16 more
>>
>>
>> Stack trace when relative URL tries to restore current working directory:
>> SftpConsumer 2010-10-01 13:41:58,529 -- ERROR -- Caused by: [org.apache.camel.component.file.GenericFileOperationFailedException - Cannot change directory to: ]
>> org.apache.camel.component.file.GenericFileOperationFailedException: Cannot change directory to:
>> at org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:372)
>> at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:361)
>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:435)
>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:407)
>> at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299)
>> at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
>> at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
>> at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>> at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>> at java.lang.Thread.run(Thread.java:637)
>> Caused by: 4:
>> at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:285)
>> at org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:370)
>> ... 16 more
>> Caused by: java.lang.StringIndexOutOfBoundsException: String index out of range: 0
>> at java.lang.String.charAt(String.java:686)
>> at com.jcraft.jsch.ChannelSftp.remoteAbsolutePath(ChannelSftp.java:2359)
>> at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:264)
>> ... 17 more
>>
>> Cheers
>> -Lorrin
>>
>> On Sep 29, 2010, at 9:37 PM, Claus Ibsen wrote:
>>
>>> Hi
>>>
>>> You have not used the latest source code. The stacktrace points to
>>> empty lines etc.
>>> Make sure you build camel-core and camel-ftp using latest source code.
>>>
>>> Revision: 1002946
>>> Node Kind: directory
>>> Schedule: normal
>>> Last Changed Author: davsclaus
>>> Last Changed Rev: 1002821
>>> Last Changed Date: 2010-09-29 21:31:23 +0200 (Wed, 29 Sep 2010)
>>>
>>>
>>> On Wed, Sep 29, 2010 at 11:55 PM, Lorrin Nelson
>>> <lh...@nerdylorrin.net> wrote:
>>>> I'm pretty sure I was already at revision 1002541 before. Now I definitely am. Same Exception. I think the problem is the lack of leading /.
>>>>
>>>> SftpOperations.retrieveFileToStreamInBody computes path as "tmp/mult", calls changeCurrentDirectory(path), which in turn calls channel.cd(path), which (now we're in com.jcraft.jsch.ChannelSftp) calls path=remoteAbsolutePath(path); and computes path: "/home/test/tmp/mult", which does not exist.
>>>>
>>>> SftpConsumer 2010-09-29 14:51:50,487 -- ERROR -- Caused by: [org.apache.camel.component.file.GenericFileOperationFailedException - Cannot change current directory to: tmp/mult]
>>>> org.apache.camel.component.file.GenericFileOperationFailedException: Cannot change current directory to: tmp/mult
>>>> at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:352)
>>>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:408)
>>>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:387)
>>>> at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299)
>>>> at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
>>>> at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
>>>> at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
>>>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>>>> at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>> at java.lang.Thread.run(Thread.java:637)
>>>> Caused by: 2: No such file
>>>> at com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2289)
>>>> at com.jcraft.jsch.ChannelSftp._realpath(ChannelSftp.java:1822)
>>>> at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:268)
>>>> at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:350)
>>>> ... 15 more
>>>>
>>>>
>>>> Cheers!
>>>> -Lorrin
>>>>
>>>> On Sep 29, 2010, at 12:32 PM, Claus Ibsen wrote:
>>>>
>>>>> Hi
>>>>>
>>>>> Can you try again with latest source code from trunk. I have changed
>>>>> the logic to change dir one folder at a time.
>>>>>
>>>>>
>>>>> On Wed, Sep 29, 2010 at 7:47 PM, Lorrin Nelson
>>>>> <lh...@nerdylorrin.net> wrote:
>>>>>> Thanks for the quick response. I'm seeing a different failure now ("Cannot change current directory to: tmp/mult" rather than "Cannot retrieve file: tmp/mult/dummy-5.log"). Is it suspicious that in either case there is no leading /?
>>>>>>
>>>>>> I tried uris
>>>>>> sftp://test@host//tmp/mult?include=.*\.log
>>>>>> and
>>>>>> sftp://test@host///tmp/mult?include=.*\.log
>>>>>>
>>>>>> SftpConsumer 2010-09-29 10:31:48,401 -- ERROR -- Caused by: [org.apache.camel.component.file.GenericFileOperationFailedException - Cannot change current directory to: tmp/mult]
>>>>>> org.apache.camel.component.file.GenericFileOperationFailedException: Cannot change current directory to: tmp/mult
>>>>>> at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:352)
>>>>>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:408)
>>>>>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:387)
>>>>>> at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299)
>>>>>> at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
>>>>>> at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
>>>>>> at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
>>>>>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>>>>>> at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>>>>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
>>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
>>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>>>> at java.lang.Thread.run(Thread.java:637)
>>>>>> Caused by: 2: No such file
>>>>>> at com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2289)
>>>>>> at com.jcraft.jsch.ChannelSftp._realpath(ChannelSftp.java:1822)
>>>>>> at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:268)
>>>>>> at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:350)
>>>>>> ... 15 more
>>>>>>
>>>>>> On a tangentially related note, there's some file name processing during the move step that throws a NullPointerException when the URI contains a trailing slash:
>>>>>>
>>>>>> sftp://test@host/tmp/mult/?include=.*\.log
>>>>>>
>>>>>> The problem is that GenericFile.changeFileName calls
>>>>>> newFileName = ObjectHelper.after(newFileName, endpointPath + getFileSeparator());
>>>>>> ...without first checking if endPointPath contains a trailing separator.
>>>>>>
>>>>>> E.g.
>>>>>> newFileName: "tmp/mult/.done/dummy-2.log"
>>>>>> endPointPath: "tmp/mult/"
>>>>>>
>>>>>> ObjectHelper.after():
>>>>>> text: "tmp/mult/.done/dummy-2.log"
>>>>>> after: "tmp/mult//"
>>>>>>
>>>>>> text doesn't contain after (because of double //), returns null, changeFileName() doesn't expect null, throws NPE.
>>>>>>
>>>>>> GenericFileOnCompletion 2010-09-29 10:37:37,888 -- ERROR -- Caused by: [java.lang.NullPointerException - null]
>>>>>> java.lang.NullPointerException
>>>>>> at java.io.File.<init>(File.java:222)
>>>>>> at org.apache.camel.component.file.GenericFile.changeFileName(GenericFile.java:169)
>>>>>> at org.apache.camel.component.file.strategy.GenericFileExpressionRenamer.renameFile(GenericFileExpressionRenamer.java:41)
>>>>>> at org.apache.camel.component.file.strategy.GenericFileRenameProcessStrategy.commit(GenericFileRenameProcessStrategy.java:82)
>>>>>> at org.apache.camel.component.file.GenericFileOnCompletion.processStrategyCommit(GenericFileOnCompletion.java:121)
>>>>>> at org.apache.camel.component.file.GenericFileOnCompletion.onCompletion(GenericFileOnCompletion.java:83)
>>>>>> at org.apache.camel.component.file.GenericFileOnCompletion.onComplete(GenericFileOnCompletion.java:52)
>>>>>> at org.apache.camel.util.UnitOfWorkHelper.doneSynchronizations(UnitOfWorkHelper.java:55)
>>>>>> at org.apache.camel.impl.DefaultUnitOfWork.done(DefaultUnitOfWork.java:173)
>>>>>> at org.apache.camel.processor.UnitOfWorkProcessor.doneUow(UnitOfWorkProcessor.java:121)
>>>>>> at org.apache.camel.processor.UnitOfWorkProcessor.access$000(UnitOfWorkProcessor.java:36)
>>>>>> at org.apache.camel.processor.UnitOfWorkProcessor$1.done(UnitOfWorkProcessor.java:106)
>>>>>> at org.apache.camel.processor.Pipeline.process(Pipeline.java:130)
>>>>>> at org.apache.camel.processor.UnitOfWorkProcessor.process(UnitOfWorkProcessor.java:99)
>>>>>> at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:70)
>>>>>> at org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:98)
>>>>>> at org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:89)
>>>>>> at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:68)
>>>>>> at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:322)
>>>>>> at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
>>>>>> at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
>>>>>> at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
>>>>>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>>>>>> at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>>>>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
>>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
>>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>>>> at java.lang.Thread.run(Thread.java:637)
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Sep 29, 2010, at 2:46 AM, Claus Ibsen wrote:
>>>>>>
>>>>>>> Hi
>>>>>>>
>>>>>>> I have committed a fix to the SFTP component. Can you try with latest
>>>>>>> source code from trunk?
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Sep 28, 2010 at 7:36 PM, Lorrin <lh...@nerdylorrin.net> wrote:
>>>>>>>>
>>>>>>>> I just bumped into the same problem. I'm running 2.5-SNAPSHOT.
>>>>>>>>
>>>>>>>> SftpConsumer 2010-09-28 10:31:28,094 -- ERROR -- Caused by:
>>>>>>>> [org.apache.camel.component.file.GenericFileOperationFailedException -
>>>>>>>> Cannot retrieve file: tmp/mult/dummy-5.log]
>>>>>>>> org.apache.camel.component.file.GenericFileOperationFailedException: Cannot
>>>>>>>> retrieve file: tmp/mult/dummy-5.log
>>>>>>>> at
>>>>>>>> org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:403)
>>>>>>>> at
>>>>>>>> org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:387)
>>>>>>>> at
>>>>>>>> org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299)
>>>>>>>> at
>>>>>>>> org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
>>>>>>>> at
>>>>>>>> org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
>>>>>>>> at
>>>>>>>> org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
>>>>>>>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>>>>>>>> at
>>>>>>>> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>>>>>>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>>>>>>>> at
>>>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>>>>>>>> at
>>>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
>>>>>>>> at
>>>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
>>>>>>>> at
>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>>>>>> at
>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>>>>>> at java.lang.Thread.run(Thread.java:637)
>>>>>>>> Caused by: 2: No such file
>>>>>>>> at com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2289)
>>>>>>>> at com.jcraft.jsch.ChannelSftp._get(ChannelSftp.java:901)
>>>>>>>> at com.jcraft.jsch.ChannelSftp.get(ChannelSftp.java:870)
>>>>>>>> at com.jcraft.jsch.ChannelSftp.get(ChannelSftp.java:849)
>>>>>>>> at
>>>>>>>> org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:400)
>>>>>>>> ... 14 more
>>>>>>>>
>>>>>>>> --
>>>>>>>> View this message in context: http://camel.465427.n5.nabble.com/How-to-change-directory-while-using-sftp-component-tp2806817p2857187.html
>>>>>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Claus Ibsen
>>>>>>> Apache Camel Committer
>>>>>>>
>>>>>>> Author of Camel in Action: http://www.manning.com/ibsen/
>>>>>>> Open Source Integration: http://fusesource.com
>>>>>>> Blog: http://davsclaus.blogspot.com/
>>>>>>> Twitter: http://twitter.com/davsclaus
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Claus Ibsen
>>>>> Apache Camel Committer
>>>>>
>>>>> Author of Camel in Action: http://www.manning.com/ibsen/
>>>>> Open Source Integration: http://fusesource.com
>>>>> Blog: http://davsclaus.blogspot.com/
>>>>> Twitter: http://twitter.com/davsclaus
>>>>>
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Claus Ibsen
>>> Apache Camel Committer
>>>
>>> Author of Camel in Action: http://www.manning.com/ibsen/
>>> Open Source Integration: http://fusesource.com
>>> Blog: http://davsclaus.blogspot.com/
>>> Twitter: http://twitter.com/davsclaus
>>>
>>
>>
>
>
>
> --
> Claus Ibsen
> Apache Camel Committer
>
> Author of Camel in Action: http://www.manning.com/ibsen/
> Open Source Integration: http://fusesource.com
> Blog: http://davsclaus.blogspot.com/
> Twitter: http://twitter.com/davsclaus
>
--
Claus Ibsen
Apache Camel Committer
Author of Camel in Action: http://www.manning.com/ibsen/
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus
Re: How to change directory while using sftp component
Posted by Claus Ibsen <cl...@gmail.com>.
Hi Lorin
Thanks for digging into it. I will take a look at it in the morning.
Hopefully I have a fix in SVN which you can try.
On Tue, Oct 5, 2010 at 8:09 PM, Lorrin Nelson
<lh...@nerdylorrin.net> wrote:
> Hi, I've confirmed this is a Camel bug and not an SFTP library limitation. See below for trace log of a successful retrieval from an absolute path.
>
> I did this in a way that's not suitable for general use: I just unconditionally hacked a "/" on the front of the path in SftpOperations.retrieveFileToStreamInBody:
>
> // change directory to path where the file is to be retrieved
> // (must do this as some FTP servers cannot retrieve using absolute path)
> String path = FileUtil.onlyPath(name);
> if (path != null) {
> changeCurrentDirectory("/" + path);
> }
>
> Cheers
> -Lorrin
>
>
> ScheduledPollConsumer 2010-10-05 11:05:48,540 -- TRACE -- Starting to poll: Endpoint[sftp://tomcat@host//tmp/pitch_activity_logs/?delay=15000&idempotent=true&idempotentRepository=%23apacheLogFilenameRepositoryAcceptOlder&include=.*.log.%5B-%2F%3A0-9%5D*&initialDelay=3000&move=.done&password=******]
> SftpConsumer 2010-10-05 11:05:48,547 -- TRACE -- doPollDirectory from absolutePath: /tmp/pitch_activity_logs, dirName: null
> SftpOperations 2010-10-05 11:05:48,548 -- TRACE -- Changing directory: /
> SftpOperations 2010-10-05 11:05:48,632 -- TRACE -- Changing directory: tmp
> ScheduledPollConsumer 2010-10-05 11:05:48,681 -- TRACE -- Starting to poll: Endpoint[sftp://tomcat@host//tmp/pitch_activity_logs/?delay=15000&idempotent=true&idempotentRepository=%23apacheLogFilenameRepositoryAcceptNewest&include=.*.log.%5B-%2F%3A0-9%5D*&initialDelay=3000&noop=true&password=******]
> SftpConsumer 2010-10-05 11:05:48,682 -- TRACE -- doPollDirectory from absolutePath: /tmp/pitch_activity_logs, dirName: null
> SftpOperations 2010-10-05 11:05:48,682 -- TRACE -- Changing directory: /
> SftpOperations 2010-10-05 11:05:48,713 -- TRACE -- Changing directory: pitch_activity_logs
> SftpOperations 2010-10-05 11:05:48,768 -- TRACE -- Changing directory: tmp
> SftpConsumer 2010-10-05 11:05:48,797 -- TRACE -- Polling directory: /tmp/pitch_activity_logs
> SftpOperations 2010-10-05 11:05:48,856 -- TRACE -- Changing directory: pitch_activity_logs
> SftpConsumer 2010-10-05 11:05:48,947 -- TRACE -- Polling directory: /tmp/pitch_activity_logs
> SftpConsumer 2010-10-05 11:05:49,013 -- TRACE -- Found 4 in directory: /tmp/pitch_activity_logs
> SftpConsumer 2010-10-05 11:05:49,013 -- TRACE -- This consumer is idempotent and the file has been consumed before. Will skip this file: GenericFile[pitch_activity_log.1285909200]
> SftpOperations 2010-10-05 11:05:49,013 -- TRACE -- Changing directory: /
> SftpOperations 2010-10-05 11:05:49,105 -- TRACE -- Changing directory: home
> SftpConsumer 2010-10-05 11:05:49,169 -- TRACE -- Found 4 in directory: /tmp/pitch_activity_logs
> SftpOperations 2010-10-05 11:05:49,170 -- TRACE -- Changing directory: /
> SftpOperations 2010-10-05 11:05:49,196 -- TRACE -- Changing directory: tomcat
> SftpOperations 2010-10-05 11:05:49,258 -- TRACE -- Changing directory: home
> SftpConsumer 2010-10-05 11:05:49,284 -- DEBUG -- Took 0.737 seconds to poll: /tmp/pitch_activity_logs/
> ScheduledPollConsumer 2010-10-05 11:05:49,284 -- TRACE -- Finished polling: Endpoint[sftp://tomcat@host//tmp/pitch_activity_logs/?delay=15000&idempotent=true&idempotentRepository=%23apacheLogFilenameRepositoryAcceptOlder&include=.*.log.%5B-%2F%3A0-9%5D*&initialDelay=3000&move=.done&password=******]
> SftpOperations 2010-10-05 11:05:49,347 -- TRACE -- Changing directory: tomcat
> SftpConsumer 2010-10-05 11:05:49,435 -- DEBUG -- Took 0.753 seconds to poll: /tmp/pitch_activity_logs/
> SftpConsumer 2010-10-05 11:05:49,436 -- DEBUG -- Total 1 files to consume
> SftpConsumer 2010-10-05 11:05:49,436 -- TRACE -- Processing file: GenericFile[pitch_activity_log.1285909200]
> SftpConsumer 2010-10-05 11:05:49,436 -- TRACE -- Retrieving file: tmp/pitch_activity_logs/pitch_activity_log.1285909200 from: Endpoint[sftp://tomcat@host//tmp/pitch_activity_logs/?delay=15000&idempotent=true&idempotentRepository=%23apacheLogFilenameRepositoryAcceptNewest&include=.*.log.%5B-%2F%3A0-9%5D*&initialDelay=3000&noop=true&password=******]
> SftpOperations 2010-10-05 11:05:49,436 -- TRACE -- Changing directory: /
> SftpOperations 2010-10-05 11:05:49,526 -- TRACE -- Changing directory: tmp
> SftpOperations 2010-10-05 11:05:49,608 -- TRACE -- Changing directory: pitch_activity_logs
> SftpOperations 2010-10-05 11:05:49,928 -- TRACE -- Changing directory: /
> SftpOperations 2010-10-05 11:05:50,026 -- TRACE -- Changing directory: home
> SftpOperations 2010-10-05 11:05:50,117 -- TRACE -- Changing directory: tomcat
> SftpConsumer 2010-10-05 11:05:50,204 -- TRACE -- Retrieved file: tmp/pitch_activity_logs/pitch_activity_log.1285909200 from: Endpoint[sftp://tomcat@host//tmp/pitch_activity_logs/?delay=15000&idempotent=true&idempotentRepository=%23apacheLogFilenameRepositoryAcceptNewest&include=.*.log.%5B-%2F%3A0-9%5D*&initialDelay=3000&noop=true&password=******]
> SftpConsumer 2010-10-05 11:05:50,204 -- DEBUG -- About to process file: GenericFile[pitch_activity_log.1285909200] using exchange: Exchange[pitch_activity_log.1285909200]
>
>
>
>
>
>
> On Oct 5, 2010, at 10:21 AM, Lorrin Nelson wrote:
>
>> Hmm. How come when polling the absolute path it says this before listing the files in the directory (which works)?
>>>> SftpOperations 2010-10-04 15:45:07,494 -- TRACE -- Changing directory: /
>>>> SftpOperations 2010-10-04 15:45:07,608 -- TRACE -- Changing directory: tmp
>>>> SftpOperations 2010-10-04 15:45:07,697 -- TRACE -- Changing directory: activity_logs
>>
>> ...but then when it tries to retrieve the file (which fails) it just says:
>>>> SftpOperations 2010-10-04 15:45:08,277 -- TRACE -- Changing directory: tmp
>>>> SftpOperations 2010-10-04 15:45:08,367 -- TRACE -- Changing directory: activity_logs
>>
>>
>> That looks awfully like Camel isn't even asking the SFTP library to go to / first when it comes time to retrieve the file.
>>
>> Cheers!
>> -Lorrin
>>
>>
>> On Oct 5, 2010, at 12:02 AM, Claus Ibsen wrote:
>>
>>> Hi
>>>
>>> Great that the relative works. This is also what I expected.
>>>
>>> In terms of absolute then Camel changes the directory to / (since its
>>> absolute) and the SFTP library returns OK for this.
>>> But in reality it does not go to the root path, but most likely just
>>> goes to the home directory of the user.
>>> And hence why absolute does not work for you.
>>>
>>> I don't see any API or way to tell SFTP library it should change to
>>> root path and not home folder.
>>>
>>> A solution to be able to poll files from an absolute folder is to
>>> create a symlink in the user home directory which can be polled by the
>>> SFTP library.
>>>
>>>> nlike the polling stage where it says "doPollDirectory from absolutePath: /tmp/activity_logs".
>>> The reason why you see this is because Camel logs the path configured
>>> from the endpoint, when the poller starts. Hence you see the leading
>>> /.
>>>
>>>
>>>
>>> On Tue, Oct 5, 2010 at 12:58 AM, Lorrin Nelson
>>> <lh...@nerdylorrin.net> wrote:
>>>> Hi Claus, thanks for the continued support. With the latest code (revision 1003927) relative URIs are working again for me.
>>>>
>>>> This the first time I'm using camel-ftp. The server I'm talking to is a Fedora 11 box running OpenSSH_5.2p1. Regular FTP is an unlikely option for us. Relative URLs are an acceptable work-around.
>>>>
>>>> Below is a trace log of an absolute URL.
>>>>
>>>> On the first attempt, note that ~/tmp does exist. Therefore when it tries to go to *relative* path tmp (which is the wrong thing to do), it succeeds. Then when it tries to go one level deeper into pitch_activity_logs it fails. The second attempt is really interesting. Note there that the current working directory has remained at /home/tomcat/tmp. So when it fails to go to *relative* path tmp, it fails right away. BTW, note that it says "Retrieving file: tmp/activity_logs/activity_log.1285909200" with no leading slash, unlike the polling stage where it says "doPollDirectory from absolutePath: /tmp/activity_logs".
>>>>
>>>> Cheers
>>>> -Lorrin
>>>>
>>>>
>>>> SftpConsumer 2010-10-04 15:45:07,445 -- INFO -- Connected and logged in to: sftp://tomcat@host:22
>>>> SftpConsumer 2010-10-04 15:45:07,494 -- TRACE -- doPollDirectory from absolutePath: /tmp/activity_logs, dirName: null
>>>> SftpOperations 2010-10-04 15:45:07,494 -- TRACE -- Changing directory: /
>>>> SftpOperations 2010-10-04 15:45:07,608 -- TRACE -- Changing directory: tmp
>>>> SftpOperations 2010-10-04 15:45:07,697 -- TRACE -- Changing directory: activity_logs
>>>> SftpConsumer 2010-10-04 15:45:07,788 -- TRACE -- Polling directory: /tmp/activity_logs
>>>> SftpConsumer 2010-10-04 15:45:08,011 -- TRACE -- Found 4 in directory: /tmp/activity_logs
>>>> SftpOperations 2010-10-04 15:45:08,012 -- TRACE -- Changing directory: /
>>>> SftpOperations 2010-10-04 15:45:08,099 -- TRACE -- Changing directory: home
>>>> SftpOperations 2010-10-04 15:45:08,189 -- TRACE -- Changing directory: tomcat
>>>> SftpConsumer 2010-10-04 15:45:08,274 -- DEBUG -- Took 0.829 seconds to poll: /tmp/activity_logs/
>>>> SftpConsumer 2010-10-04 15:45:08,275 -- DEBUG -- Total 1 files to consume
>>>> SftpConsumer 2010-10-04 15:45:08,276 -- TRACE -- Processing file: GenericFile[activity_log.1285909200]
>>>> SftpConsumer 2010-10-04 15:45:08,276 -- TRACE -- Retrieving file: tmp/activity_logs/activity_log.1285909200 from: Endpoint[sftp://tomcat@host//tmp/activity_logs/?delay=15000&idempotent=true&idempotentRepository=%23apacheLogFilenameRepositoryAcceptNewest&include=.*.log.%5B-%2F%3A0-9%5D*&initialDelay=3000&noop=true&password=******]
>>>> SftpOperations 2010-10-04 15:45:08,277 -- TRACE -- Changing directory: tmp
>>>> SftpOperations 2010-10-04 15:45:08,367 -- TRACE -- Changing directory: activity_logs
>>>> SftpConsumer 2010-10-04 15:45:08,414 -- ERROR -- Caused by: [org.apache.camel.component.file.GenericFileOperationFailedException - Cannot change directory to: activity_logs]
>>>> org.apache.camel.component.file.GenericFileOperationFailedException: Cannot change directory to: activity_logs
>>>> at org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:383)
>>>> at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:368)
>>>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:451)
>>>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:430)
>>>> at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299)
>>>> at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
>>>> at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
>>>> at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
>>>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>>>> at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>> at java.lang.Thread.run(Thread.java:637)
>>>> Caused by: 2: No such file
>>>> at com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2289)
>>>> at com.jcraft.jsch.ChannelSftp._realpath(ChannelSftp.java:1822)
>>>> at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:268)
>>>> at org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:381)
>>>> ... 16 more
>>>> ScheduledPollConsumer 2010-10-04 15:45:08,426 -- TRACE -- Finished polling: Endpoint[sftp://tomcat@host//tmp/activity_logs/?delay=15000&idempotent=true&idempotentRepository=%23apacheLogFilenameRepositoryAcceptNewest&include=.*.log.%5B-%2F%3A0-9%5D*&initialDelay=3000&noop=true&password=******]
>>>>
>>>>
>>>>
>>>> ScheduledPollConsumer 2010-10-04 15:45:21,413 -- TRACE -- Starting to poll: Endpoint[sftp://tomcat@host//tmp/activity_logs/?delay=15000&idempotent=true&idempotentRepository=%23apacheLogFilenameRepositoryAcceptNewest&include=.*.log.%5B-%2F%3A0-9%5D*&initialDelay=3000&noop=true&password=******]
>>>> SftpConsumer 2010-10-04 15:45:21,413 -- TRACE -- doPollDirectory from absolutePath: /tmp/activity_logs, dirName: null
>>>> SftpOperations 2010-10-04 15:45:21,413 -- TRACE -- Changing directory: /
>>>> SftpOperations 2010-10-04 15:45:21,496 -- TRACE -- Changing directory: tmp
>>>> SftpOperations 2010-10-04 15:45:21,585 -- TRACE -- Changing directory: activity_logs
>>>> SftpConsumer 2010-10-04 15:45:21,675 -- TRACE -- Polling directory: /tmp/activity_logs
>>>> SftpConsumer 2010-10-04 15:45:21,907 -- TRACE -- Found 4 in directory: /tmp/activity_logs
>>>> SftpOperations 2010-10-04 15:45:21,907 -- TRACE -- Changing directory: /
>>>> SftpOperations 2010-10-04 15:45:21,997 -- TRACE -- Changing directory: home
>>>> SftpOperations 2010-10-04 15:45:22,084 -- TRACE -- Changing directory: tomcat
>>>> SftpOperations 2010-10-04 15:45:22,177 -- TRACE -- Changing directory: tmp
>>>> SftpConsumer 2010-10-04 15:45:22,261 -- DEBUG -- Took 0.848 seconds to poll: /tmp/activity_logs/
>>>> SftpConsumer 2010-10-04 15:45:22,261 -- DEBUG -- Total 1 files to consume
>>>> SftpConsumer 2010-10-04 15:45:22,261 -- TRACE -- Processing file: GenericFile[activity_log.1285909200]
>>>> SftpConsumer 2010-10-04 15:45:22,261 -- TRACE -- Retrieving file: tmp/activity_logs/activity_log.1285909200 from: Endpoint[sftp://tomcat@host//tmp/activity_logs/?delay=15000&idempotent=true&idempotentRepository=%23apacheLogFilenameRepositoryAcceptNewest&include=.*.log.%5B-%2F%3A0-9%5D*&initialDelay=3000&noop=true&password=******]
>>>> SftpOperations 2010-10-04 15:45:22,262 -- TRACE -- Changing directory: tmp
>>>> SftpConsumer 2010-10-04 15:45:22,305 -- ERROR -- Caused by: [org.apache.camel.component.file.GenericFileOperationFailedException - Cannot change directory to: tmp]
>>>> org.apache.camel.component.file.GenericFileOperationFailedException: Cannot change directory to: tmp
>>>> at org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:383)
>>>> at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:368)
>>>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:451)
>>>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:430)
>>>> at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299)
>>>> at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
>>>> at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
>>>> at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
>>>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>>>> at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>> at java.lang.Thread.run(Thread.java:637)
>>>> Caused by: 2: No such file
>>>> at com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2289)
>>>> at com.jcraft.jsch.ChannelSftp._realpath(ChannelSftp.java:1822)
>>>> at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:268)
>>>> at org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:381)
>>>> ... 16 more
>>>> ScheduledPollConsumer 2010-10-04 15:45:22,311 -- TRACE -- Finished polling: Endpoint[sftp://tomcat@host//tmp/activity_logs/?delay=15000&idempotent=true&idempotentRepository=%23apacheLogFilenameRepositoryAcceptNewest&include=.*.log.%5B-%2F%3A0-9%5D*&initialDelay=3000&noop=true&password=******]
>>>>
>>>>
>>>>
>>>>
>>>> On Oct 2, 2010, at 12:14 AM, Claus Ibsen wrote:
>>>>
>>>>> Hi Loririn
>>>>>
>>>>> I think you/we have to step back. We have other users in the past
>>>>> running with SFTP with no problems at all. Also fetching files
>>>>> recursively.
>>>>> So I wonder if we should revert back and keep the old logic!
>>>>>
>>>>> 1)
>>>>> Is this the first time you use camel-ftp?
>>>>>
>>>>> 2)
>>>>> What OS and FTP server are you using?
>>>>>
>>>>> 3)
>>>>> Can you use regular FTP instead?
>>>>>
>>>>> 4)
>>>>> Use relative urls over absolute. They where the first supported.
>>>>>
>>>>> 5)
>>>>> Can you enable TRACE logging on org.apache.camel.component.file and
>>>>> setup a simple example so it makes it easier to follow and understand
>>>>> what its supposed to do, and what happens in reality so we can see and
>>>>> understand the difference.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Fri, Oct 1, 2010 at 10:45 PM, Lorrin Nelson
>>>>> <lh...@nerdylorrin.net> wrote:
>>>>>> Hi Claus. Ack, sorry about that. I'm pulling through github rather than SVN directly and then running via a Maven project pointing at my locally built Camel, so it's sometimes confusing how up-to-date I am.
>>>>>>
>>>>>> So I think I have current code, including checkin "CAMEL-3174: Changing dir with ftp must do one dir at a time." and I think SFTP polling is now broken for both absolute and relative URLs. See debug walk-throughs and stack traces below.
>>>>>>
>>>>>> Absolute URI with double leading /:
>>>>>> sftp://user@host//home/user/subdir?idempotent=true&idempotentRepository=#myCustomIR&include=.*.log&noop=true&password=******
>>>>>>
>>>>>> SftpOperations.changeCurrentDirectory("home/user/subdir") //note there's no leading slash on the supposedly absolute path!
>>>>>> //splits into chunks successfully, starts looping through chunks
>>>>>> doChangeDirectory("home")
>>>>>> ChannelSftp.cd("home")
>>>>>> //path=remoteAbsolutePath(path);
>>>>>> //yields "/home/user/home". Doh!
>>>>>>
>>>>>>
>>>>>> Relative URI with single leading /:
>>>>>> sftp://user@host/subdir?idempotent=true&idempotentRepository=#myCustomIR&include=.*.log&noop=true&password=******
>>>>>> SftpOperations.changeCurrentDirectory("subdir")
>>>>>> //splits into single chunk, starts looping through chunks
>>>>>> doChangeDirectory("subdir")
>>>>>> ChannelSftp.cd("home")
>>>>>> //path=remoteAbsolutePath(path);
>>>>>> //yields "/home/user/subdir". yay!
>>>>>>
>>>>>> ..but then, back in SftpOperations.retrieveFileToStreamInBody, it hits the "change back to current directory code"
>>>>>> changeCurrentDirectory("/home/tomcat"); //now we have a leading slash, unlike before
>>>>>> //splits info chunks "", "home", and "tomcat". Doh!
>>>>>> doChangeDirectory("")
>>>>>> ChannelSftp.cd("")
>>>>>> //path=remoteAbsolutePath(path) throws a StringIndexOutOfBoundsException when it receives empty string.
>>>>>>
>>>>>>
>>>>>> Stack trace when attempting to supply absolute path:
>>>>>> org.apache.camel.component.file.GenericFileOperationFailedException: Cannot change directory to: home
>>>>>> at org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:372)
>>>>>> at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:361)
>>>>>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:428)
>>>>>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:407)
>>>>>> at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299)
>>>>>> at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
>>>>>> at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
>>>>>> at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
>>>>>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>>>>>> at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>>>>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
>>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
>>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>>>> at java.lang.Thread.run(Thread.java:637)
>>>>>> Caused by: 2: No such file
>>>>>> at com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2289)
>>>>>> at com.jcraft.jsch.ChannelSftp._realpath(ChannelSftp.java:1822)
>>>>>> at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:268)
>>>>>> at org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:370)
>>>>>> ... 16 more
>>>>>>
>>>>>>
>>>>>> Stack trace when relative URL tries to restore current working directory:
>>>>>> SftpConsumer 2010-10-01 13:41:58,529 -- ERROR -- Caused by: [org.apache.camel.component.file.GenericFileOperationFailedException - Cannot change directory to: ]
>>>>>> org.apache.camel.component.file.GenericFileOperationFailedException: Cannot change directory to:
>>>>>> at org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:372)
>>>>>> at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:361)
>>>>>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:435)
>>>>>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:407)
>>>>>> at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299)
>>>>>> at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
>>>>>> at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
>>>>>> at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
>>>>>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>>>>>> at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>>>>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
>>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
>>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>>>> at java.lang.Thread.run(Thread.java:637)
>>>>>> Caused by: 4:
>>>>>> at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:285)
>>>>>> at org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:370)
>>>>>> ... 16 more
>>>>>> Caused by: java.lang.StringIndexOutOfBoundsException: String index out of range: 0
>>>>>> at java.lang.String.charAt(String.java:686)
>>>>>> at com.jcraft.jsch.ChannelSftp.remoteAbsolutePath(ChannelSftp.java:2359)
>>>>>> at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:264)
>>>>>> ... 17 more
>>>>>>
>>>>>> Cheers
>>>>>> -Lorrin
>>>>>>
>>>>>> On Sep 29, 2010, at 9:37 PM, Claus Ibsen wrote:
>>>>>>
>>>>>>> Hi
>>>>>>>
>>>>>>> You have not used the latest source code. The stacktrace points to
>>>>>>> empty lines etc.
>>>>>>> Make sure you build camel-core and camel-ftp using latest source code.
>>>>>>>
>>>>>>> Revision: 1002946
>>>>>>> Node Kind: directory
>>>>>>> Schedule: normal
>>>>>>> Last Changed Author: davsclaus
>>>>>>> Last Changed Rev: 1002821
>>>>>>> Last Changed Date: 2010-09-29 21:31:23 +0200 (Wed, 29 Sep 2010)
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Sep 29, 2010 at 11:55 PM, Lorrin Nelson
>>>>>>> <lh...@nerdylorrin.net> wrote:
>>>>>>>> I'm pretty sure I was already at revision 1002541 before. Now I definitely am. Same Exception. I think the problem is the lack of leading /.
>>>>>>>>
>>>>>>>> SftpOperations.retrieveFileToStreamInBody computes path as "tmp/mult", calls changeCurrentDirectory(path), which in turn calls channel.cd(path), which (now we're in com.jcraft.jsch.ChannelSftp) calls path=remoteAbsolutePath(path); and computes path: "/home/test/tmp/mult", which does not exist.
>>>>>>>>
>>>>>>>> SftpConsumer 2010-09-29 14:51:50,487 -- ERROR -- Caused by: [org.apache.camel.component.file.GenericFileOperationFailedException - Cannot change current directory to: tmp/mult]
>>>>>>>> org.apache.camel.component.file.GenericFileOperationFailedException: Cannot change current directory to: tmp/mult
>>>>>>>> at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:352)
>>>>>>>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:408)
>>>>>>>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:387)
>>>>>>>> at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299)
>>>>>>>> at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
>>>>>>>> at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
>>>>>>>> at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
>>>>>>>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>>>>>>>> at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>>>>>>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>>>>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>>>>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
>>>>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
>>>>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>>>>>> at java.lang.Thread.run(Thread.java:637)
>>>>>>>> Caused by: 2: No such file
>>>>>>>> at com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2289)
>>>>>>>> at com.jcraft.jsch.ChannelSftp._realpath(ChannelSftp.java:1822)
>>>>>>>> at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:268)
>>>>>>>> at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:350)
>>>>>>>> ... 15 more
>>>>>>>>
>>>>>>>>
>>>>>>>> Cheers!
>>>>>>>> -Lorrin
>>>>>>>>
>>>>>>>> On Sep 29, 2010, at 12:32 PM, Claus Ibsen wrote:
>>>>>>>>
>>>>>>>>> Hi
>>>>>>>>>
>>>>>>>>> Can you try again with latest source code from trunk. I have changed
>>>>>>>>> the logic to change dir one folder at a time.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Wed, Sep 29, 2010 at 7:47 PM, Lorrin Nelson
>>>>>>>>> <lh...@nerdylorrin.net> wrote:
>>>>>>>>>> Thanks for the quick response. I'm seeing a different failure now ("Cannot change current directory to: tmp/mult" rather than "Cannot retrieve file: tmp/mult/dummy-5.log"). Is it suspicious that in either case there is no leading /?
>>>>>>>>>>
>>>>>>>>>> I tried uris
>>>>>>>>>> sftp://test@host//tmp/mult?include=.*\.log
>>>>>>>>>> and
>>>>>>>>>> sftp://test@host///tmp/mult?include=.*\.log
>>>>>>>>>>
>>>>>>>>>> SftpConsumer 2010-09-29 10:31:48,401 -- ERROR -- Caused by: [org.apache.camel.component.file.GenericFileOperationFailedException - Cannot change current directory to: tmp/mult]
>>>>>>>>>> org.apache.camel.component.file.GenericFileOperationFailedException: Cannot change current directory to: tmp/mult
>>>>>>>>>> at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:352)
>>>>>>>>>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:408)
>>>>>>>>>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:387)
>>>>>>>>>> at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299)
>>>>>>>>>> at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
>>>>>>>>>> at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
>>>>>>>>>> at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
>>>>>>>>>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>>>>>>>>>> at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>>>>>>>>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>>>>>>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>>>>>>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
>>>>>>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
>>>>>>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>>>>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>>>>>>>> at java.lang.Thread.run(Thread.java:637)
>>>>>>>>>> Caused by: 2: No such file
>>>>>>>>>> at com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2289)
>>>>>>>>>> at com.jcraft.jsch.ChannelSftp._realpath(ChannelSftp.java:1822)
>>>>>>>>>> at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:268)
>>>>>>>>>> at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:350)
>>>>>>>>>> ... 15 more
>>>>>>>>>>
>>>>>>>>>> On a tangentially related note, there's some file name processing during the move step that throws a NullPointerException when the URI contains a trailing slash:
>>>>>>>>>>
>>>>>>>>>> sftp://test@host/tmp/mult/?include=.*\.log
>>>>>>>>>>
>>>>>>>>>> The problem is that GenericFile.changeFileName calls
>>>>>>>>>> newFileName = ObjectHelper.after(newFileName, endpointPath + getFileSeparator());
>>>>>>>>>> ...without first checking if endPointPath contains a trailing separator.
>>>>>>>>>>
>>>>>>>>>> E.g.
>>>>>>>>>> newFileName: "tmp/mult/.done/dummy-2.log"
>>>>>>>>>> endPointPath: "tmp/mult/"
>>>>>>>>>>
>>>>>>>>>> ObjectHelper.after():
>>>>>>>>>> text: "tmp/mult/.done/dummy-2.log"
>>>>>>>>>> after: "tmp/mult//"
>>>>>>>>>>
>>>>>>>>>> text doesn't contain after (because of double //), returns null, changeFileName() doesn't expect null, throws NPE.
>>>>>>>>>>
>>>>>>>>>> GenericFileOnCompletion 2010-09-29 10:37:37,888 -- ERROR -- Caused by: [java.lang.NullPointerException - null]
>>>>>>>>>> java.lang.NullPointerException
>>>>>>>>>> at java.io.File.<init>(File.java:222)
>>>>>>>>>> at org.apache.camel.component.file.GenericFile.changeFileName(GenericFile.java:169)
>>>>>>>>>> at org.apache.camel.component.file.strategy.GenericFileExpressionRenamer.renameFile(GenericFileExpressionRenamer.java:41)
>>>>>>>>>> at org.apache.camel.component.file.strategy.GenericFileRenameProcessStrategy.commit(GenericFileRenameProcessStrategy.java:82)
>>>>>>>>>> at org.apache.camel.component.file.GenericFileOnCompletion.processStrategyCommit(GenericFileOnCompletion.java:121)
>>>>>>>>>> at org.apache.camel.component.file.GenericFileOnCompletion.onCompletion(GenericFileOnCompletion.java:83)
>>>>>>>>>> at org.apache.camel.component.file.GenericFileOnCompletion.onComplete(GenericFileOnCompletion.java:52)
>>>>>>>>>> at org.apache.camel.util.UnitOfWorkHelper.doneSynchronizations(UnitOfWorkHelper.java:55)
>>>>>>>>>> at org.apache.camel.impl.DefaultUnitOfWork.done(DefaultUnitOfWork.java:173)
>>>>>>>>>> at org.apache.camel.processor.UnitOfWorkProcessor.doneUow(UnitOfWorkProcessor.java:121)
>>>>>>>>>> at org.apache.camel.processor.UnitOfWorkProcessor.access$000(UnitOfWorkProcessor.java:36)
>>>>>>>>>> at org.apache.camel.processor.UnitOfWorkProcessor$1.done(UnitOfWorkProcessor.java:106)
>>>>>>>>>> at org.apache.camel.processor.Pipeline.process(Pipeline.java:130)
>>>>>>>>>> at org.apache.camel.processor.UnitOfWorkProcessor.process(UnitOfWorkProcessor.java:99)
>>>>>>>>>> at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:70)
>>>>>>>>>> at org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:98)
>>>>>>>>>> at org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:89)
>>>>>>>>>> at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:68)
>>>>>>>>>> at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:322)
>>>>>>>>>> at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
>>>>>>>>>> at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
>>>>>>>>>> at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
>>>>>>>>>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>>>>>>>>>> at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>>>>>>>>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>>>>>>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>>>>>>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
>>>>>>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
>>>>>>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>>>>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>>>>>>>> at java.lang.Thread.run(Thread.java:637)
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Sep 29, 2010, at 2:46 AM, Claus Ibsen wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi
>>>>>>>>>>>
>>>>>>>>>>> I have committed a fix to the SFTP component. Can you try with latest
>>>>>>>>>>> source code from trunk?
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Tue, Sep 28, 2010 at 7:36 PM, Lorrin <lh...@nerdylorrin.net> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> I just bumped into the same problem. I'm running 2.5-SNAPSHOT.
>>>>>>>>>>>>
>>>>>>>>>>>> SftpConsumer 2010-09-28 10:31:28,094 -- ERROR -- Caused by:
>>>>>>>>>>>> [org.apache.camel.component.file.GenericFileOperationFailedException -
>>>>>>>>>>>> Cannot retrieve file: tmp/mult/dummy-5.log]
>>>>>>>>>>>> org.apache.camel.component.file.GenericFileOperationFailedException: Cannot
>>>>>>>>>>>> retrieve file: tmp/mult/dummy-5.log
>>>>>>>>>>>> at
>>>>>>>>>>>> org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:403)
>>>>>>>>>>>> at
>>>>>>>>>>>> org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:387)
>>>>>>>>>>>> at
>>>>>>>>>>>> org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299)
>>>>>>>>>>>> at
>>>>>>>>>>>> org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
>>>>>>>>>>>> at
>>>>>>>>>>>> org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
>>>>>>>>>>>> at
>>>>>>>>>>>> org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
>>>>>>>>>>>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>>>>>>>>>>>> at
>>>>>>>>>>>> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>>>>>>>>>>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>>>>>>>>>>>> at
>>>>>>>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>>>>>>>>>>>> at
>>>>>>>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
>>>>>>>>>>>> at
>>>>>>>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
>>>>>>>>>>>> at
>>>>>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>>>>>>>>>> at
>>>>>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>>>>>>>>>> at java.lang.Thread.run(Thread.java:637)
>>>>>>>>>>>> Caused by: 2: No such file
>>>>>>>>>>>> at com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2289)
>>>>>>>>>>>> at com.jcraft.jsch.ChannelSftp._get(ChannelSftp.java:901)
>>>>>>>>>>>> at com.jcraft.jsch.ChannelSftp.get(ChannelSftp.java:870)
>>>>>>>>>>>> at com.jcraft.jsch.ChannelSftp.get(ChannelSftp.java:849)
>>>>>>>>>>>> at
>>>>>>>>>>>> org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:400)
>>>>>>>>>>>> ... 14 more
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> View this message in context: http://camel.465427.n5.nabble.com/How-to-change-directory-while-using-sftp-component-tp2806817p2857187.html
>>>>>>>>>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Claus Ibsen
>>>>>>>>>>> Apache Camel Committer
>>>>>>>>>>>
>>>>>>>>>>> Author of Camel in Action: http://www.manning.com/ibsen/
>>>>>>>>>>> Open Source Integration: http://fusesource.com
>>>>>>>>>>> Blog: http://davsclaus.blogspot.com/
>>>>>>>>>>> Twitter: http://twitter.com/davsclaus
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Claus Ibsen
>>>>>>>>> Apache Camel Committer
>>>>>>>>>
>>>>>>>>> Author of Camel in Action: http://www.manning.com/ibsen/
>>>>>>>>> Open Source Integration: http://fusesource.com
>>>>>>>>> Blog: http://davsclaus.blogspot.com/
>>>>>>>>> Twitter: http://twitter.com/davsclaus
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Claus Ibsen
>>>>>>> Apache Camel Committer
>>>>>>>
>>>>>>> Author of Camel in Action: http://www.manning.com/ibsen/
>>>>>>> Open Source Integration: http://fusesource.com
>>>>>>> Blog: http://davsclaus.blogspot.com/
>>>>>>> Twitter: http://twitter.com/davsclaus
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Claus Ibsen
>>>>> Apache Camel Committer
>>>>>
>>>>> Author of Camel in Action: http://www.manning.com/ibsen/
>>>>> Open Source Integration: http://fusesource.com
>>>>> Blog: http://davsclaus.blogspot.com/
>>>>> Twitter: http://twitter.com/davsclaus
>>>>>
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Claus Ibsen
>>> Apache Camel Committer
>>>
>>> Author of Camel in Action: http://www.manning.com/ibsen/
>>> Open Source Integration: http://fusesource.com
>>> Blog: http://davsclaus.blogspot.com/
>>> Twitter: http://twitter.com/davsclaus
>>>
>>
>>
>
>
--
Claus Ibsen
Apache Camel Committer
Author of Camel in Action: http://www.manning.com/ibsen/
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus
Re: How to change directory while using sftp component
Posted by Lorrin Nelson <lh...@nerdylorrin.net>.
Hi, I've confirmed this is a Camel bug and not an SFTP library limitation. See below for trace log of a successful retrieval from an absolute path.
I did this in a way that's not suitable for general use: I just unconditionally hacked a "/" on the front of the path in SftpOperations.retrieveFileToStreamInBody:
// change directory to path where the file is to be retrieved
// (must do this as some FTP servers cannot retrieve using absolute path)
String path = FileUtil.onlyPath(name);
if (path != null) {
changeCurrentDirectory("/" + path);
}
Cheers
-Lorrin
ScheduledPollConsumer 2010-10-05 11:05:48,540 -- TRACE -- Starting to poll: Endpoint[sftp://tomcat@host//tmp/pitch_activity_logs/?delay=15000&idempotent=true&idempotentRepository=%23apacheLogFilenameRepositoryAcceptOlder&include=.*.log.%5B-%2F%3A0-9%5D*&initialDelay=3000&move=.done&password=******]
SftpConsumer 2010-10-05 11:05:48,547 -- TRACE -- doPollDirectory from absolutePath: /tmp/pitch_activity_logs, dirName: null
SftpOperations 2010-10-05 11:05:48,548 -- TRACE -- Changing directory: /
SftpOperations 2010-10-05 11:05:48,632 -- TRACE -- Changing directory: tmp
ScheduledPollConsumer 2010-10-05 11:05:48,681 -- TRACE -- Starting to poll: Endpoint[sftp://tomcat@host//tmp/pitch_activity_logs/?delay=15000&idempotent=true&idempotentRepository=%23apacheLogFilenameRepositoryAcceptNewest&include=.*.log.%5B-%2F%3A0-9%5D*&initialDelay=3000&noop=true&password=******]
SftpConsumer 2010-10-05 11:05:48,682 -- TRACE -- doPollDirectory from absolutePath: /tmp/pitch_activity_logs, dirName: null
SftpOperations 2010-10-05 11:05:48,682 -- TRACE -- Changing directory: /
SftpOperations 2010-10-05 11:05:48,713 -- TRACE -- Changing directory: pitch_activity_logs
SftpOperations 2010-10-05 11:05:48,768 -- TRACE -- Changing directory: tmp
SftpConsumer 2010-10-05 11:05:48,797 -- TRACE -- Polling directory: /tmp/pitch_activity_logs
SftpOperations 2010-10-05 11:05:48,856 -- TRACE -- Changing directory: pitch_activity_logs
SftpConsumer 2010-10-05 11:05:48,947 -- TRACE -- Polling directory: /tmp/pitch_activity_logs
SftpConsumer 2010-10-05 11:05:49,013 -- TRACE -- Found 4 in directory: /tmp/pitch_activity_logs
SftpConsumer 2010-10-05 11:05:49,013 -- TRACE -- This consumer is idempotent and the file has been consumed before. Will skip this file: GenericFile[pitch_activity_log.1285909200]
SftpOperations 2010-10-05 11:05:49,013 -- TRACE -- Changing directory: /
SftpOperations 2010-10-05 11:05:49,105 -- TRACE -- Changing directory: home
SftpConsumer 2010-10-05 11:05:49,169 -- TRACE -- Found 4 in directory: /tmp/pitch_activity_logs
SftpOperations 2010-10-05 11:05:49,170 -- TRACE -- Changing directory: /
SftpOperations 2010-10-05 11:05:49,196 -- TRACE -- Changing directory: tomcat
SftpOperations 2010-10-05 11:05:49,258 -- TRACE -- Changing directory: home
SftpConsumer 2010-10-05 11:05:49,284 -- DEBUG -- Took 0.737 seconds to poll: /tmp/pitch_activity_logs/
ScheduledPollConsumer 2010-10-05 11:05:49,284 -- TRACE -- Finished polling: Endpoint[sftp://tomcat@host//tmp/pitch_activity_logs/?delay=15000&idempotent=true&idempotentRepository=%23apacheLogFilenameRepositoryAcceptOlder&include=.*.log.%5B-%2F%3A0-9%5D*&initialDelay=3000&move=.done&password=******]
SftpOperations 2010-10-05 11:05:49,347 -- TRACE -- Changing directory: tomcat
SftpConsumer 2010-10-05 11:05:49,435 -- DEBUG -- Took 0.753 seconds to poll: /tmp/pitch_activity_logs/
SftpConsumer 2010-10-05 11:05:49,436 -- DEBUG -- Total 1 files to consume
SftpConsumer 2010-10-05 11:05:49,436 -- TRACE -- Processing file: GenericFile[pitch_activity_log.1285909200]
SftpConsumer 2010-10-05 11:05:49,436 -- TRACE -- Retrieving file: tmp/pitch_activity_logs/pitch_activity_log.1285909200 from: Endpoint[sftp://tomcat@host//tmp/pitch_activity_logs/?delay=15000&idempotent=true&idempotentRepository=%23apacheLogFilenameRepositoryAcceptNewest&include=.*.log.%5B-%2F%3A0-9%5D*&initialDelay=3000&noop=true&password=******]
SftpOperations 2010-10-05 11:05:49,436 -- TRACE -- Changing directory: /
SftpOperations 2010-10-05 11:05:49,526 -- TRACE -- Changing directory: tmp
SftpOperations 2010-10-05 11:05:49,608 -- TRACE -- Changing directory: pitch_activity_logs
SftpOperations 2010-10-05 11:05:49,928 -- TRACE -- Changing directory: /
SftpOperations 2010-10-05 11:05:50,026 -- TRACE -- Changing directory: home
SftpOperations 2010-10-05 11:05:50,117 -- TRACE -- Changing directory: tomcat
SftpConsumer 2010-10-05 11:05:50,204 -- TRACE -- Retrieved file: tmp/pitch_activity_logs/pitch_activity_log.1285909200 from: Endpoint[sftp://tomcat@host//tmp/pitch_activity_logs/?delay=15000&idempotent=true&idempotentRepository=%23apacheLogFilenameRepositoryAcceptNewest&include=.*.log.%5B-%2F%3A0-9%5D*&initialDelay=3000&noop=true&password=******]
SftpConsumer 2010-10-05 11:05:50,204 -- DEBUG -- About to process file: GenericFile[pitch_activity_log.1285909200] using exchange: Exchange[pitch_activity_log.1285909200]
On Oct 5, 2010, at 10:21 AM, Lorrin Nelson wrote:
> Hmm. How come when polling the absolute path it says this before listing the files in the directory (which works)?
>>> SftpOperations 2010-10-04 15:45:07,494 -- TRACE -- Changing directory: /
>>> SftpOperations 2010-10-04 15:45:07,608 -- TRACE -- Changing directory: tmp
>>> SftpOperations 2010-10-04 15:45:07,697 -- TRACE -- Changing directory: activity_logs
>
> ...but then when it tries to retrieve the file (which fails) it just says:
>>> SftpOperations 2010-10-04 15:45:08,277 -- TRACE -- Changing directory: tmp
>>> SftpOperations 2010-10-04 15:45:08,367 -- TRACE -- Changing directory: activity_logs
>
>
> That looks awfully like Camel isn't even asking the SFTP library to go to / first when it comes time to retrieve the file.
>
> Cheers!
> -Lorrin
>
>
> On Oct 5, 2010, at 12:02 AM, Claus Ibsen wrote:
>
>> Hi
>>
>> Great that the relative works. This is also what I expected.
>>
>> In terms of absolute then Camel changes the directory to / (since its
>> absolute) and the SFTP library returns OK for this.
>> But in reality it does not go to the root path, but most likely just
>> goes to the home directory of the user.
>> And hence why absolute does not work for you.
>>
>> I don't see any API or way to tell SFTP library it should change to
>> root path and not home folder.
>>
>> A solution to be able to poll files from an absolute folder is to
>> create a symlink in the user home directory which can be polled by the
>> SFTP library.
>>
>>> nlike the polling stage where it says "doPollDirectory from absolutePath: /tmp/activity_logs".
>> The reason why you see this is because Camel logs the path configured
>> from the endpoint, when the poller starts. Hence you see the leading
>> /.
>>
>>
>>
>> On Tue, Oct 5, 2010 at 12:58 AM, Lorrin Nelson
>> <lh...@nerdylorrin.net> wrote:
>>> Hi Claus, thanks for the continued support. With the latest code (revision 1003927) relative URIs are working again for me.
>>>
>>> This the first time I'm using camel-ftp. The server I'm talking to is a Fedora 11 box running OpenSSH_5.2p1. Regular FTP is an unlikely option for us. Relative URLs are an acceptable work-around.
>>>
>>> Below is a trace log of an absolute URL.
>>>
>>> On the first attempt, note that ~/tmp does exist. Therefore when it tries to go to *relative* path tmp (which is the wrong thing to do), it succeeds. Then when it tries to go one level deeper into pitch_activity_logs it fails. The second attempt is really interesting. Note there that the current working directory has remained at /home/tomcat/tmp. So when it fails to go to *relative* path tmp, it fails right away. BTW, note that it says "Retrieving file: tmp/activity_logs/activity_log.1285909200" with no leading slash, unlike the polling stage where it says "doPollDirectory from absolutePath: /tmp/activity_logs".
>>>
>>> Cheers
>>> -Lorrin
>>>
>>>
>>> SftpConsumer 2010-10-04 15:45:07,445 -- INFO -- Connected and logged in to: sftp://tomcat@host:22
>>> SftpConsumer 2010-10-04 15:45:07,494 -- TRACE -- doPollDirectory from absolutePath: /tmp/activity_logs, dirName: null
>>> SftpOperations 2010-10-04 15:45:07,494 -- TRACE -- Changing directory: /
>>> SftpOperations 2010-10-04 15:45:07,608 -- TRACE -- Changing directory: tmp
>>> SftpOperations 2010-10-04 15:45:07,697 -- TRACE -- Changing directory: activity_logs
>>> SftpConsumer 2010-10-04 15:45:07,788 -- TRACE -- Polling directory: /tmp/activity_logs
>>> SftpConsumer 2010-10-04 15:45:08,011 -- TRACE -- Found 4 in directory: /tmp/activity_logs
>>> SftpOperations 2010-10-04 15:45:08,012 -- TRACE -- Changing directory: /
>>> SftpOperations 2010-10-04 15:45:08,099 -- TRACE -- Changing directory: home
>>> SftpOperations 2010-10-04 15:45:08,189 -- TRACE -- Changing directory: tomcat
>>> SftpConsumer 2010-10-04 15:45:08,274 -- DEBUG -- Took 0.829 seconds to poll: /tmp/activity_logs/
>>> SftpConsumer 2010-10-04 15:45:08,275 -- DEBUG -- Total 1 files to consume
>>> SftpConsumer 2010-10-04 15:45:08,276 -- TRACE -- Processing file: GenericFile[activity_log.1285909200]
>>> SftpConsumer 2010-10-04 15:45:08,276 -- TRACE -- Retrieving file: tmp/activity_logs/activity_log.1285909200 from: Endpoint[sftp://tomcat@host//tmp/activity_logs/?delay=15000&idempotent=true&idempotentRepository=%23apacheLogFilenameRepositoryAcceptNewest&include=.*.log.%5B-%2F%3A0-9%5D*&initialDelay=3000&noop=true&password=******]
>>> SftpOperations 2010-10-04 15:45:08,277 -- TRACE -- Changing directory: tmp
>>> SftpOperations 2010-10-04 15:45:08,367 -- TRACE -- Changing directory: activity_logs
>>> SftpConsumer 2010-10-04 15:45:08,414 -- ERROR -- Caused by: [org.apache.camel.component.file.GenericFileOperationFailedException - Cannot change directory to: activity_logs]
>>> org.apache.camel.component.file.GenericFileOperationFailedException: Cannot change directory to: activity_logs
>>> at org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:383)
>>> at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:368)
>>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:451)
>>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:430)
>>> at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299)
>>> at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
>>> at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
>>> at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
>>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>>> at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>> at java.lang.Thread.run(Thread.java:637)
>>> Caused by: 2: No such file
>>> at com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2289)
>>> at com.jcraft.jsch.ChannelSftp._realpath(ChannelSftp.java:1822)
>>> at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:268)
>>> at org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:381)
>>> ... 16 more
>>> ScheduledPollConsumer 2010-10-04 15:45:08,426 -- TRACE -- Finished polling: Endpoint[sftp://tomcat@host//tmp/activity_logs/?delay=15000&idempotent=true&idempotentRepository=%23apacheLogFilenameRepositoryAcceptNewest&include=.*.log.%5B-%2F%3A0-9%5D*&initialDelay=3000&noop=true&password=******]
>>>
>>>
>>>
>>> ScheduledPollConsumer 2010-10-04 15:45:21,413 -- TRACE -- Starting to poll: Endpoint[sftp://tomcat@host//tmp/activity_logs/?delay=15000&idempotent=true&idempotentRepository=%23apacheLogFilenameRepositoryAcceptNewest&include=.*.log.%5B-%2F%3A0-9%5D*&initialDelay=3000&noop=true&password=******]
>>> SftpConsumer 2010-10-04 15:45:21,413 -- TRACE -- doPollDirectory from absolutePath: /tmp/activity_logs, dirName: null
>>> SftpOperations 2010-10-04 15:45:21,413 -- TRACE -- Changing directory: /
>>> SftpOperations 2010-10-04 15:45:21,496 -- TRACE -- Changing directory: tmp
>>> SftpOperations 2010-10-04 15:45:21,585 -- TRACE -- Changing directory: activity_logs
>>> SftpConsumer 2010-10-04 15:45:21,675 -- TRACE -- Polling directory: /tmp/activity_logs
>>> SftpConsumer 2010-10-04 15:45:21,907 -- TRACE -- Found 4 in directory: /tmp/activity_logs
>>> SftpOperations 2010-10-04 15:45:21,907 -- TRACE -- Changing directory: /
>>> SftpOperations 2010-10-04 15:45:21,997 -- TRACE -- Changing directory: home
>>> SftpOperations 2010-10-04 15:45:22,084 -- TRACE -- Changing directory: tomcat
>>> SftpOperations 2010-10-04 15:45:22,177 -- TRACE -- Changing directory: tmp
>>> SftpConsumer 2010-10-04 15:45:22,261 -- DEBUG -- Took 0.848 seconds to poll: /tmp/activity_logs/
>>> SftpConsumer 2010-10-04 15:45:22,261 -- DEBUG -- Total 1 files to consume
>>> SftpConsumer 2010-10-04 15:45:22,261 -- TRACE -- Processing file: GenericFile[activity_log.1285909200]
>>> SftpConsumer 2010-10-04 15:45:22,261 -- TRACE -- Retrieving file: tmp/activity_logs/activity_log.1285909200 from: Endpoint[sftp://tomcat@host//tmp/activity_logs/?delay=15000&idempotent=true&idempotentRepository=%23apacheLogFilenameRepositoryAcceptNewest&include=.*.log.%5B-%2F%3A0-9%5D*&initialDelay=3000&noop=true&password=******]
>>> SftpOperations 2010-10-04 15:45:22,262 -- TRACE -- Changing directory: tmp
>>> SftpConsumer 2010-10-04 15:45:22,305 -- ERROR -- Caused by: [org.apache.camel.component.file.GenericFileOperationFailedException - Cannot change directory to: tmp]
>>> org.apache.camel.component.file.GenericFileOperationFailedException: Cannot change directory to: tmp
>>> at org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:383)
>>> at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:368)
>>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:451)
>>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:430)
>>> at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299)
>>> at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
>>> at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
>>> at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
>>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>>> at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>> at java.lang.Thread.run(Thread.java:637)
>>> Caused by: 2: No such file
>>> at com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2289)
>>> at com.jcraft.jsch.ChannelSftp._realpath(ChannelSftp.java:1822)
>>> at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:268)
>>> at org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:381)
>>> ... 16 more
>>> ScheduledPollConsumer 2010-10-04 15:45:22,311 -- TRACE -- Finished polling: Endpoint[sftp://tomcat@host//tmp/activity_logs/?delay=15000&idempotent=true&idempotentRepository=%23apacheLogFilenameRepositoryAcceptNewest&include=.*.log.%5B-%2F%3A0-9%5D*&initialDelay=3000&noop=true&password=******]
>>>
>>>
>>>
>>>
>>> On Oct 2, 2010, at 12:14 AM, Claus Ibsen wrote:
>>>
>>>> Hi Loririn
>>>>
>>>> I think you/we have to step back. We have other users in the past
>>>> running with SFTP with no problems at all. Also fetching files
>>>> recursively.
>>>> So I wonder if we should revert back and keep the old logic!
>>>>
>>>> 1)
>>>> Is this the first time you use camel-ftp?
>>>>
>>>> 2)
>>>> What OS and FTP server are you using?
>>>>
>>>> 3)
>>>> Can you use regular FTP instead?
>>>>
>>>> 4)
>>>> Use relative urls over absolute. They where the first supported.
>>>>
>>>> 5)
>>>> Can you enable TRACE logging on org.apache.camel.component.file and
>>>> setup a simple example so it makes it easier to follow and understand
>>>> what its supposed to do, and what happens in reality so we can see and
>>>> understand the difference.
>>>>
>>>>
>>>>
>>>>
>>>> On Fri, Oct 1, 2010 at 10:45 PM, Lorrin Nelson
>>>> <lh...@nerdylorrin.net> wrote:
>>>>> Hi Claus. Ack, sorry about that. I'm pulling through github rather than SVN directly and then running via a Maven project pointing at my locally built Camel, so it's sometimes confusing how up-to-date I am.
>>>>>
>>>>> So I think I have current code, including checkin "CAMEL-3174: Changing dir with ftp must do one dir at a time." and I think SFTP polling is now broken for both absolute and relative URLs. See debug walk-throughs and stack traces below.
>>>>>
>>>>> Absolute URI with double leading /:
>>>>> sftp://user@host//home/user/subdir?idempotent=true&idempotentRepository=#myCustomIR&include=.*.log&noop=true&password=******
>>>>>
>>>>> SftpOperations.changeCurrentDirectory("home/user/subdir") //note there's no leading slash on the supposedly absolute path!
>>>>> //splits into chunks successfully, starts looping through chunks
>>>>> doChangeDirectory("home")
>>>>> ChannelSftp.cd("home")
>>>>> //path=remoteAbsolutePath(path);
>>>>> //yields "/home/user/home". Doh!
>>>>>
>>>>>
>>>>> Relative URI with single leading /:
>>>>> sftp://user@host/subdir?idempotent=true&idempotentRepository=#myCustomIR&include=.*.log&noop=true&password=******
>>>>> SftpOperations.changeCurrentDirectory("subdir")
>>>>> //splits into single chunk, starts looping through chunks
>>>>> doChangeDirectory("subdir")
>>>>> ChannelSftp.cd("home")
>>>>> //path=remoteAbsolutePath(path);
>>>>> //yields "/home/user/subdir". yay!
>>>>>
>>>>> ..but then, back in SftpOperations.retrieveFileToStreamInBody, it hits the "change back to current directory code"
>>>>> changeCurrentDirectory("/home/tomcat"); //now we have a leading slash, unlike before
>>>>> //splits info chunks "", "home", and "tomcat". Doh!
>>>>> doChangeDirectory("")
>>>>> ChannelSftp.cd("")
>>>>> //path=remoteAbsolutePath(path) throws a StringIndexOutOfBoundsException when it receives empty string.
>>>>>
>>>>>
>>>>> Stack trace when attempting to supply absolute path:
>>>>> org.apache.camel.component.file.GenericFileOperationFailedException: Cannot change directory to: home
>>>>> at org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:372)
>>>>> at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:361)
>>>>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:428)
>>>>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:407)
>>>>> at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299)
>>>>> at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
>>>>> at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
>>>>> at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
>>>>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>>>>> at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>>>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>>> at java.lang.Thread.run(Thread.java:637)
>>>>> Caused by: 2: No such file
>>>>> at com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2289)
>>>>> at com.jcraft.jsch.ChannelSftp._realpath(ChannelSftp.java:1822)
>>>>> at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:268)
>>>>> at org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:370)
>>>>> ... 16 more
>>>>>
>>>>>
>>>>> Stack trace when relative URL tries to restore current working directory:
>>>>> SftpConsumer 2010-10-01 13:41:58,529 -- ERROR -- Caused by: [org.apache.camel.component.file.GenericFileOperationFailedException - Cannot change directory to: ]
>>>>> org.apache.camel.component.file.GenericFileOperationFailedException: Cannot change directory to:
>>>>> at org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:372)
>>>>> at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:361)
>>>>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:435)
>>>>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:407)
>>>>> at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299)
>>>>> at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
>>>>> at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
>>>>> at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
>>>>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>>>>> at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>>>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>>> at java.lang.Thread.run(Thread.java:637)
>>>>> Caused by: 4:
>>>>> at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:285)
>>>>> at org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:370)
>>>>> ... 16 more
>>>>> Caused by: java.lang.StringIndexOutOfBoundsException: String index out of range: 0
>>>>> at java.lang.String.charAt(String.java:686)
>>>>> at com.jcraft.jsch.ChannelSftp.remoteAbsolutePath(ChannelSftp.java:2359)
>>>>> at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:264)
>>>>> ... 17 more
>>>>>
>>>>> Cheers
>>>>> -Lorrin
>>>>>
>>>>> On Sep 29, 2010, at 9:37 PM, Claus Ibsen wrote:
>>>>>
>>>>>> Hi
>>>>>>
>>>>>> You have not used the latest source code. The stacktrace points to
>>>>>> empty lines etc.
>>>>>> Make sure you build camel-core and camel-ftp using latest source code.
>>>>>>
>>>>>> Revision: 1002946
>>>>>> Node Kind: directory
>>>>>> Schedule: normal
>>>>>> Last Changed Author: davsclaus
>>>>>> Last Changed Rev: 1002821
>>>>>> Last Changed Date: 2010-09-29 21:31:23 +0200 (Wed, 29 Sep 2010)
>>>>>>
>>>>>>
>>>>>> On Wed, Sep 29, 2010 at 11:55 PM, Lorrin Nelson
>>>>>> <lh...@nerdylorrin.net> wrote:
>>>>>>> I'm pretty sure I was already at revision 1002541 before. Now I definitely am. Same Exception. I think the problem is the lack of leading /.
>>>>>>>
>>>>>>> SftpOperations.retrieveFileToStreamInBody computes path as "tmp/mult", calls changeCurrentDirectory(path), which in turn calls channel.cd(path), which (now we're in com.jcraft.jsch.ChannelSftp) calls path=remoteAbsolutePath(path); and computes path: "/home/test/tmp/mult", which does not exist.
>>>>>>>
>>>>>>> SftpConsumer 2010-09-29 14:51:50,487 -- ERROR -- Caused by: [org.apache.camel.component.file.GenericFileOperationFailedException - Cannot change current directory to: tmp/mult]
>>>>>>> org.apache.camel.component.file.GenericFileOperationFailedException: Cannot change current directory to: tmp/mult
>>>>>>> at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:352)
>>>>>>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:408)
>>>>>>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:387)
>>>>>>> at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299)
>>>>>>> at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
>>>>>>> at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
>>>>>>> at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
>>>>>>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>>>>>>> at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>>>>>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>>>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>>>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
>>>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
>>>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>>>>> at java.lang.Thread.run(Thread.java:637)
>>>>>>> Caused by: 2: No such file
>>>>>>> at com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2289)
>>>>>>> at com.jcraft.jsch.ChannelSftp._realpath(ChannelSftp.java:1822)
>>>>>>> at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:268)
>>>>>>> at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:350)
>>>>>>> ... 15 more
>>>>>>>
>>>>>>>
>>>>>>> Cheers!
>>>>>>> -Lorrin
>>>>>>>
>>>>>>> On Sep 29, 2010, at 12:32 PM, Claus Ibsen wrote:
>>>>>>>
>>>>>>>> Hi
>>>>>>>>
>>>>>>>> Can you try again with latest source code from trunk. I have changed
>>>>>>>> the logic to change dir one folder at a time.
>>>>>>>>
>>>>>>>>
>>>>>>>> On Wed, Sep 29, 2010 at 7:47 PM, Lorrin Nelson
>>>>>>>> <lh...@nerdylorrin.net> wrote:
>>>>>>>>> Thanks for the quick response. I'm seeing a different failure now ("Cannot change current directory to: tmp/mult" rather than "Cannot retrieve file: tmp/mult/dummy-5.log"). Is it suspicious that in either case there is no leading /?
>>>>>>>>>
>>>>>>>>> I tried uris
>>>>>>>>> sftp://test@host//tmp/mult?include=.*\.log
>>>>>>>>> and
>>>>>>>>> sftp://test@host///tmp/mult?include=.*\.log
>>>>>>>>>
>>>>>>>>> SftpConsumer 2010-09-29 10:31:48,401 -- ERROR -- Caused by: [org.apache.camel.component.file.GenericFileOperationFailedException - Cannot change current directory to: tmp/mult]
>>>>>>>>> org.apache.camel.component.file.GenericFileOperationFailedException: Cannot change current directory to: tmp/mult
>>>>>>>>> at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:352)
>>>>>>>>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:408)
>>>>>>>>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:387)
>>>>>>>>> at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299)
>>>>>>>>> at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
>>>>>>>>> at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
>>>>>>>>> at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
>>>>>>>>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>>>>>>>>> at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>>>>>>>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>>>>>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>>>>>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
>>>>>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
>>>>>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>>>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>>>>>>> at java.lang.Thread.run(Thread.java:637)
>>>>>>>>> Caused by: 2: No such file
>>>>>>>>> at com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2289)
>>>>>>>>> at com.jcraft.jsch.ChannelSftp._realpath(ChannelSftp.java:1822)
>>>>>>>>> at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:268)
>>>>>>>>> at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:350)
>>>>>>>>> ... 15 more
>>>>>>>>>
>>>>>>>>> On a tangentially related note, there's some file name processing during the move step that throws a NullPointerException when the URI contains a trailing slash:
>>>>>>>>>
>>>>>>>>> sftp://test@host/tmp/mult/?include=.*\.log
>>>>>>>>>
>>>>>>>>> The problem is that GenericFile.changeFileName calls
>>>>>>>>> newFileName = ObjectHelper.after(newFileName, endpointPath + getFileSeparator());
>>>>>>>>> ...without first checking if endPointPath contains a trailing separator.
>>>>>>>>>
>>>>>>>>> E.g.
>>>>>>>>> newFileName: "tmp/mult/.done/dummy-2.log"
>>>>>>>>> endPointPath: "tmp/mult/"
>>>>>>>>>
>>>>>>>>> ObjectHelper.after():
>>>>>>>>> text: "tmp/mult/.done/dummy-2.log"
>>>>>>>>> after: "tmp/mult//"
>>>>>>>>>
>>>>>>>>> text doesn't contain after (because of double //), returns null, changeFileName() doesn't expect null, throws NPE.
>>>>>>>>>
>>>>>>>>> GenericFileOnCompletion 2010-09-29 10:37:37,888 -- ERROR -- Caused by: [java.lang.NullPointerException - null]
>>>>>>>>> java.lang.NullPointerException
>>>>>>>>> at java.io.File.<init>(File.java:222)
>>>>>>>>> at org.apache.camel.component.file.GenericFile.changeFileName(GenericFile.java:169)
>>>>>>>>> at org.apache.camel.component.file.strategy.GenericFileExpressionRenamer.renameFile(GenericFileExpressionRenamer.java:41)
>>>>>>>>> at org.apache.camel.component.file.strategy.GenericFileRenameProcessStrategy.commit(GenericFileRenameProcessStrategy.java:82)
>>>>>>>>> at org.apache.camel.component.file.GenericFileOnCompletion.processStrategyCommit(GenericFileOnCompletion.java:121)
>>>>>>>>> at org.apache.camel.component.file.GenericFileOnCompletion.onCompletion(GenericFileOnCompletion.java:83)
>>>>>>>>> at org.apache.camel.component.file.GenericFileOnCompletion.onComplete(GenericFileOnCompletion.java:52)
>>>>>>>>> at org.apache.camel.util.UnitOfWorkHelper.doneSynchronizations(UnitOfWorkHelper.java:55)
>>>>>>>>> at org.apache.camel.impl.DefaultUnitOfWork.done(DefaultUnitOfWork.java:173)
>>>>>>>>> at org.apache.camel.processor.UnitOfWorkProcessor.doneUow(UnitOfWorkProcessor.java:121)
>>>>>>>>> at org.apache.camel.processor.UnitOfWorkProcessor.access$000(UnitOfWorkProcessor.java:36)
>>>>>>>>> at org.apache.camel.processor.UnitOfWorkProcessor$1.done(UnitOfWorkProcessor.java:106)
>>>>>>>>> at org.apache.camel.processor.Pipeline.process(Pipeline.java:130)
>>>>>>>>> at org.apache.camel.processor.UnitOfWorkProcessor.process(UnitOfWorkProcessor.java:99)
>>>>>>>>> at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:70)
>>>>>>>>> at org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:98)
>>>>>>>>> at org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:89)
>>>>>>>>> at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:68)
>>>>>>>>> at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:322)
>>>>>>>>> at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
>>>>>>>>> at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
>>>>>>>>> at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
>>>>>>>>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>>>>>>>>> at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>>>>>>>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>>>>>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>>>>>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
>>>>>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
>>>>>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>>>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>>>>>>> at java.lang.Thread.run(Thread.java:637)
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Sep 29, 2010, at 2:46 AM, Claus Ibsen wrote:
>>>>>>>>>
>>>>>>>>>> Hi
>>>>>>>>>>
>>>>>>>>>> I have committed a fix to the SFTP component. Can you try with latest
>>>>>>>>>> source code from trunk?
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Tue, Sep 28, 2010 at 7:36 PM, Lorrin <lh...@nerdylorrin.net> wrote:
>>>>>>>>>>>
>>>>>>>>>>> I just bumped into the same problem. I'm running 2.5-SNAPSHOT.
>>>>>>>>>>>
>>>>>>>>>>> SftpConsumer 2010-09-28 10:31:28,094 -- ERROR -- Caused by:
>>>>>>>>>>> [org.apache.camel.component.file.GenericFileOperationFailedException -
>>>>>>>>>>> Cannot retrieve file: tmp/mult/dummy-5.log]
>>>>>>>>>>> org.apache.camel.component.file.GenericFileOperationFailedException: Cannot
>>>>>>>>>>> retrieve file: tmp/mult/dummy-5.log
>>>>>>>>>>> at
>>>>>>>>>>> org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:403)
>>>>>>>>>>> at
>>>>>>>>>>> org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:387)
>>>>>>>>>>> at
>>>>>>>>>>> org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299)
>>>>>>>>>>> at
>>>>>>>>>>> org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
>>>>>>>>>>> at
>>>>>>>>>>> org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
>>>>>>>>>>> at
>>>>>>>>>>> org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
>>>>>>>>>>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>>>>>>>>>>> at
>>>>>>>>>>> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>>>>>>>>>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>>>>>>>>>>> at
>>>>>>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>>>>>>>>>>> at
>>>>>>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
>>>>>>>>>>> at
>>>>>>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
>>>>>>>>>>> at
>>>>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>>>>>>>>> at
>>>>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>>>>>>>>> at java.lang.Thread.run(Thread.java:637)
>>>>>>>>>>> Caused by: 2: No such file
>>>>>>>>>>> at com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2289)
>>>>>>>>>>> at com.jcraft.jsch.ChannelSftp._get(ChannelSftp.java:901)
>>>>>>>>>>> at com.jcraft.jsch.ChannelSftp.get(ChannelSftp.java:870)
>>>>>>>>>>> at com.jcraft.jsch.ChannelSftp.get(ChannelSftp.java:849)
>>>>>>>>>>> at
>>>>>>>>>>> org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:400)
>>>>>>>>>>> ... 14 more
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> View this message in context: http://camel.465427.n5.nabble.com/How-to-change-directory-while-using-sftp-component-tp2806817p2857187.html
>>>>>>>>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Claus Ibsen
>>>>>>>>>> Apache Camel Committer
>>>>>>>>>>
>>>>>>>>>> Author of Camel in Action: http://www.manning.com/ibsen/
>>>>>>>>>> Open Source Integration: http://fusesource.com
>>>>>>>>>> Blog: http://davsclaus.blogspot.com/
>>>>>>>>>> Twitter: http://twitter.com/davsclaus
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Claus Ibsen
>>>>>>>> Apache Camel Committer
>>>>>>>>
>>>>>>>> Author of Camel in Action: http://www.manning.com/ibsen/
>>>>>>>> Open Source Integration: http://fusesource.com
>>>>>>>> Blog: http://davsclaus.blogspot.com/
>>>>>>>> Twitter: http://twitter.com/davsclaus
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Claus Ibsen
>>>>>> Apache Camel Committer
>>>>>>
>>>>>> Author of Camel in Action: http://www.manning.com/ibsen/
>>>>>> Open Source Integration: http://fusesource.com
>>>>>> Blog: http://davsclaus.blogspot.com/
>>>>>> Twitter: http://twitter.com/davsclaus
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Claus Ibsen
>>>> Apache Camel Committer
>>>>
>>>> Author of Camel in Action: http://www.manning.com/ibsen/
>>>> Open Source Integration: http://fusesource.com
>>>> Blog: http://davsclaus.blogspot.com/
>>>> Twitter: http://twitter.com/davsclaus
>>>>
>>>
>>>
>>
>>
>>
>> --
>> Claus Ibsen
>> Apache Camel Committer
>>
>> Author of Camel in Action: http://www.manning.com/ibsen/
>> Open Source Integration: http://fusesource.com
>> Blog: http://davsclaus.blogspot.com/
>> Twitter: http://twitter.com/davsclaus
>>
>
>
Re: How to change directory while using sftp component
Posted by Lorrin Nelson <lh...@nerdylorrin.net>.
Hmm. How come when polling the absolute path it says this before listing the files in the directory (which works)?
>> SftpOperations 2010-10-04 15:45:07,494 -- TRACE -- Changing directory: /
>> SftpOperations 2010-10-04 15:45:07,608 -- TRACE -- Changing directory: tmp
>> SftpOperations 2010-10-04 15:45:07,697 -- TRACE -- Changing directory: activity_logs
...but then when it tries to retrieve the file (which fails) it just says:
>> SftpOperations 2010-10-04 15:45:08,277 -- TRACE -- Changing directory: tmp
>> SftpOperations 2010-10-04 15:45:08,367 -- TRACE -- Changing directory: activity_logs
That looks awfully like Camel isn't even asking the SFTP library to go to / first when it comes time to retrieve the file.
Cheers!
-Lorrin
On Oct 5, 2010, at 12:02 AM, Claus Ibsen wrote:
> Hi
>
> Great that the relative works. This is also what I expected.
>
> In terms of absolute then Camel changes the directory to / (since its
> absolute) and the SFTP library returns OK for this.
> But in reality it does not go to the root path, but most likely just
> goes to the home directory of the user.
> And hence why absolute does not work for you.
>
> I don't see any API or way to tell SFTP library it should change to
> root path and not home folder.
>
> A solution to be able to poll files from an absolute folder is to
> create a symlink in the user home directory which can be polled by the
> SFTP library.
>
>> nlike the polling stage where it says "doPollDirectory from absolutePath: /tmp/activity_logs".
> The reason why you see this is because Camel logs the path configured
> from the endpoint, when the poller starts. Hence you see the leading
> /.
>
>
>
> On Tue, Oct 5, 2010 at 12:58 AM, Lorrin Nelson
> <lh...@nerdylorrin.net> wrote:
>> Hi Claus, thanks for the continued support. With the latest code (revision 1003927) relative URIs are working again for me.
>>
>> This the first time I'm using camel-ftp. The server I'm talking to is a Fedora 11 box running OpenSSH_5.2p1. Regular FTP is an unlikely option for us. Relative URLs are an acceptable work-around.
>>
>> Below is a trace log of an absolute URL.
>>
>> On the first attempt, note that ~/tmp does exist. Therefore when it tries to go to *relative* path tmp (which is the wrong thing to do), it succeeds. Then when it tries to go one level deeper into pitch_activity_logs it fails. The second attempt is really interesting. Note there that the current working directory has remained at /home/tomcat/tmp. So when it fails to go to *relative* path tmp, it fails right away. BTW, note that it says "Retrieving file: tmp/activity_logs/activity_log.1285909200" with no leading slash, unlike the polling stage where it says "doPollDirectory from absolutePath: /tmp/activity_logs".
>>
>> Cheers
>> -Lorrin
>>
>>
>> SftpConsumer 2010-10-04 15:45:07,445 -- INFO -- Connected and logged in to: sftp://tomcat@host:22
>> SftpConsumer 2010-10-04 15:45:07,494 -- TRACE -- doPollDirectory from absolutePath: /tmp/activity_logs, dirName: null
>> SftpOperations 2010-10-04 15:45:07,494 -- TRACE -- Changing directory: /
>> SftpOperations 2010-10-04 15:45:07,608 -- TRACE -- Changing directory: tmp
>> SftpOperations 2010-10-04 15:45:07,697 -- TRACE -- Changing directory: activity_logs
>> SftpConsumer 2010-10-04 15:45:07,788 -- TRACE -- Polling directory: /tmp/activity_logs
>> SftpConsumer 2010-10-04 15:45:08,011 -- TRACE -- Found 4 in directory: /tmp/activity_logs
>> SftpOperations 2010-10-04 15:45:08,012 -- TRACE -- Changing directory: /
>> SftpOperations 2010-10-04 15:45:08,099 -- TRACE -- Changing directory: home
>> SftpOperations 2010-10-04 15:45:08,189 -- TRACE -- Changing directory: tomcat
>> SftpConsumer 2010-10-04 15:45:08,274 -- DEBUG -- Took 0.829 seconds to poll: /tmp/activity_logs/
>> SftpConsumer 2010-10-04 15:45:08,275 -- DEBUG -- Total 1 files to consume
>> SftpConsumer 2010-10-04 15:45:08,276 -- TRACE -- Processing file: GenericFile[activity_log.1285909200]
>> SftpConsumer 2010-10-04 15:45:08,276 -- TRACE -- Retrieving file: tmp/activity_logs/activity_log.1285909200 from: Endpoint[sftp://tomcat@host//tmp/activity_logs/?delay=15000&idempotent=true&idempotentRepository=%23apacheLogFilenameRepositoryAcceptNewest&include=.*.log.%5B-%2F%3A0-9%5D*&initialDelay=3000&noop=true&password=******]
>> SftpOperations 2010-10-04 15:45:08,277 -- TRACE -- Changing directory: tmp
>> SftpOperations 2010-10-04 15:45:08,367 -- TRACE -- Changing directory: activity_logs
>> SftpConsumer 2010-10-04 15:45:08,414 -- ERROR -- Caused by: [org.apache.camel.component.file.GenericFileOperationFailedException - Cannot change directory to: activity_logs]
>> org.apache.camel.component.file.GenericFileOperationFailedException: Cannot change directory to: activity_logs
>> at org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:383)
>> at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:368)
>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:451)
>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:430)
>> at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299)
>> at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
>> at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
>> at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>> at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>> at java.lang.Thread.run(Thread.java:637)
>> Caused by: 2: No such file
>> at com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2289)
>> at com.jcraft.jsch.ChannelSftp._realpath(ChannelSftp.java:1822)
>> at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:268)
>> at org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:381)
>> ... 16 more
>> ScheduledPollConsumer 2010-10-04 15:45:08,426 -- TRACE -- Finished polling: Endpoint[sftp://tomcat@host//tmp/activity_logs/?delay=15000&idempotent=true&idempotentRepository=%23apacheLogFilenameRepositoryAcceptNewest&include=.*.log.%5B-%2F%3A0-9%5D*&initialDelay=3000&noop=true&password=******]
>>
>>
>>
>> ScheduledPollConsumer 2010-10-04 15:45:21,413 -- TRACE -- Starting to poll: Endpoint[sftp://tomcat@host//tmp/activity_logs/?delay=15000&idempotent=true&idempotentRepository=%23apacheLogFilenameRepositoryAcceptNewest&include=.*.log.%5B-%2F%3A0-9%5D*&initialDelay=3000&noop=true&password=******]
>> SftpConsumer 2010-10-04 15:45:21,413 -- TRACE -- doPollDirectory from absolutePath: /tmp/activity_logs, dirName: null
>> SftpOperations 2010-10-04 15:45:21,413 -- TRACE -- Changing directory: /
>> SftpOperations 2010-10-04 15:45:21,496 -- TRACE -- Changing directory: tmp
>> SftpOperations 2010-10-04 15:45:21,585 -- TRACE -- Changing directory: activity_logs
>> SftpConsumer 2010-10-04 15:45:21,675 -- TRACE -- Polling directory: /tmp/activity_logs
>> SftpConsumer 2010-10-04 15:45:21,907 -- TRACE -- Found 4 in directory: /tmp/activity_logs
>> SftpOperations 2010-10-04 15:45:21,907 -- TRACE -- Changing directory: /
>> SftpOperations 2010-10-04 15:45:21,997 -- TRACE -- Changing directory: home
>> SftpOperations 2010-10-04 15:45:22,084 -- TRACE -- Changing directory: tomcat
>> SftpOperations 2010-10-04 15:45:22,177 -- TRACE -- Changing directory: tmp
>> SftpConsumer 2010-10-04 15:45:22,261 -- DEBUG -- Took 0.848 seconds to poll: /tmp/activity_logs/
>> SftpConsumer 2010-10-04 15:45:22,261 -- DEBUG -- Total 1 files to consume
>> SftpConsumer 2010-10-04 15:45:22,261 -- TRACE -- Processing file: GenericFile[activity_log.1285909200]
>> SftpConsumer 2010-10-04 15:45:22,261 -- TRACE -- Retrieving file: tmp/activity_logs/activity_log.1285909200 from: Endpoint[sftp://tomcat@host//tmp/activity_logs/?delay=15000&idempotent=true&idempotentRepository=%23apacheLogFilenameRepositoryAcceptNewest&include=.*.log.%5B-%2F%3A0-9%5D*&initialDelay=3000&noop=true&password=******]
>> SftpOperations 2010-10-04 15:45:22,262 -- TRACE -- Changing directory: tmp
>> SftpConsumer 2010-10-04 15:45:22,305 -- ERROR -- Caused by: [org.apache.camel.component.file.GenericFileOperationFailedException - Cannot change directory to: tmp]
>> org.apache.camel.component.file.GenericFileOperationFailedException: Cannot change directory to: tmp
>> at org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:383)
>> at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:368)
>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:451)
>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:430)
>> at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299)
>> at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
>> at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
>> at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>> at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>> at java.lang.Thread.run(Thread.java:637)
>> Caused by: 2: No such file
>> at com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2289)
>> at com.jcraft.jsch.ChannelSftp._realpath(ChannelSftp.java:1822)
>> at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:268)
>> at org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:381)
>> ... 16 more
>> ScheduledPollConsumer 2010-10-04 15:45:22,311 -- TRACE -- Finished polling: Endpoint[sftp://tomcat@host//tmp/activity_logs/?delay=15000&idempotent=true&idempotentRepository=%23apacheLogFilenameRepositoryAcceptNewest&include=.*.log.%5B-%2F%3A0-9%5D*&initialDelay=3000&noop=true&password=******]
>>
>>
>>
>>
>> On Oct 2, 2010, at 12:14 AM, Claus Ibsen wrote:
>>
>>> Hi Loririn
>>>
>>> I think you/we have to step back. We have other users in the past
>>> running with SFTP with no problems at all. Also fetching files
>>> recursively.
>>> So I wonder if we should revert back and keep the old logic!
>>>
>>> 1)
>>> Is this the first time you use camel-ftp?
>>>
>>> 2)
>>> What OS and FTP server are you using?
>>>
>>> 3)
>>> Can you use regular FTP instead?
>>>
>>> 4)
>>> Use relative urls over absolute. They where the first supported.
>>>
>>> 5)
>>> Can you enable TRACE logging on org.apache.camel.component.file and
>>> setup a simple example so it makes it easier to follow and understand
>>> what its supposed to do, and what happens in reality so we can see and
>>> understand the difference.
>>>
>>>
>>>
>>>
>>> On Fri, Oct 1, 2010 at 10:45 PM, Lorrin Nelson
>>> <lh...@nerdylorrin.net> wrote:
>>>> Hi Claus. Ack, sorry about that. I'm pulling through github rather than SVN directly and then running via a Maven project pointing at my locally built Camel, so it's sometimes confusing how up-to-date I am.
>>>>
>>>> So I think I have current code, including checkin "CAMEL-3174: Changing dir with ftp must do one dir at a time." and I think SFTP polling is now broken for both absolute and relative URLs. See debug walk-throughs and stack traces below.
>>>>
>>>> Absolute URI with double leading /:
>>>> sftp://user@host//home/user/subdir?idempotent=true&idempotentRepository=#myCustomIR&include=.*.log&noop=true&password=******
>>>>
>>>> SftpOperations.changeCurrentDirectory("home/user/subdir") //note there's no leading slash on the supposedly absolute path!
>>>> //splits into chunks successfully, starts looping through chunks
>>>> doChangeDirectory("home")
>>>> ChannelSftp.cd("home")
>>>> //path=remoteAbsolutePath(path);
>>>> //yields "/home/user/home". Doh!
>>>>
>>>>
>>>> Relative URI with single leading /:
>>>> sftp://user@host/subdir?idempotent=true&idempotentRepository=#myCustomIR&include=.*.log&noop=true&password=******
>>>> SftpOperations.changeCurrentDirectory("subdir")
>>>> //splits into single chunk, starts looping through chunks
>>>> doChangeDirectory("subdir")
>>>> ChannelSftp.cd("home")
>>>> //path=remoteAbsolutePath(path);
>>>> //yields "/home/user/subdir". yay!
>>>>
>>>> ..but then, back in SftpOperations.retrieveFileToStreamInBody, it hits the "change back to current directory code"
>>>> changeCurrentDirectory("/home/tomcat"); //now we have a leading slash, unlike before
>>>> //splits info chunks "", "home", and "tomcat". Doh!
>>>> doChangeDirectory("")
>>>> ChannelSftp.cd("")
>>>> //path=remoteAbsolutePath(path) throws a StringIndexOutOfBoundsException when it receives empty string.
>>>>
>>>>
>>>> Stack trace when attempting to supply absolute path:
>>>> org.apache.camel.component.file.GenericFileOperationFailedException: Cannot change directory to: home
>>>> at org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:372)
>>>> at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:361)
>>>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:428)
>>>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:407)
>>>> at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299)
>>>> at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
>>>> at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
>>>> at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
>>>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>>>> at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>> at java.lang.Thread.run(Thread.java:637)
>>>> Caused by: 2: No such file
>>>> at com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2289)
>>>> at com.jcraft.jsch.ChannelSftp._realpath(ChannelSftp.java:1822)
>>>> at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:268)
>>>> at org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:370)
>>>> ... 16 more
>>>>
>>>>
>>>> Stack trace when relative URL tries to restore current working directory:
>>>> SftpConsumer 2010-10-01 13:41:58,529 -- ERROR -- Caused by: [org.apache.camel.component.file.GenericFileOperationFailedException - Cannot change directory to: ]
>>>> org.apache.camel.component.file.GenericFileOperationFailedException: Cannot change directory to:
>>>> at org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:372)
>>>> at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:361)
>>>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:435)
>>>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:407)
>>>> at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299)
>>>> at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
>>>> at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
>>>> at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
>>>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>>>> at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>> at java.lang.Thread.run(Thread.java:637)
>>>> Caused by: 4:
>>>> at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:285)
>>>> at org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:370)
>>>> ... 16 more
>>>> Caused by: java.lang.StringIndexOutOfBoundsException: String index out of range: 0
>>>> at java.lang.String.charAt(String.java:686)
>>>> at com.jcraft.jsch.ChannelSftp.remoteAbsolutePath(ChannelSftp.java:2359)
>>>> at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:264)
>>>> ... 17 more
>>>>
>>>> Cheers
>>>> -Lorrin
>>>>
>>>> On Sep 29, 2010, at 9:37 PM, Claus Ibsen wrote:
>>>>
>>>>> Hi
>>>>>
>>>>> You have not used the latest source code. The stacktrace points to
>>>>> empty lines etc.
>>>>> Make sure you build camel-core and camel-ftp using latest source code.
>>>>>
>>>>> Revision: 1002946
>>>>> Node Kind: directory
>>>>> Schedule: normal
>>>>> Last Changed Author: davsclaus
>>>>> Last Changed Rev: 1002821
>>>>> Last Changed Date: 2010-09-29 21:31:23 +0200 (Wed, 29 Sep 2010)
>>>>>
>>>>>
>>>>> On Wed, Sep 29, 2010 at 11:55 PM, Lorrin Nelson
>>>>> <lh...@nerdylorrin.net> wrote:
>>>>>> I'm pretty sure I was already at revision 1002541 before. Now I definitely am. Same Exception. I think the problem is the lack of leading /.
>>>>>>
>>>>>> SftpOperations.retrieveFileToStreamInBody computes path as "tmp/mult", calls changeCurrentDirectory(path), which in turn calls channel.cd(path), which (now we're in com.jcraft.jsch.ChannelSftp) calls path=remoteAbsolutePath(path); and computes path: "/home/test/tmp/mult", which does not exist.
>>>>>>
>>>>>> SftpConsumer 2010-09-29 14:51:50,487 -- ERROR -- Caused by: [org.apache.camel.component.file.GenericFileOperationFailedException - Cannot change current directory to: tmp/mult]
>>>>>> org.apache.camel.component.file.GenericFileOperationFailedException: Cannot change current directory to: tmp/mult
>>>>>> at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:352)
>>>>>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:408)
>>>>>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:387)
>>>>>> at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299)
>>>>>> at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
>>>>>> at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
>>>>>> at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
>>>>>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>>>>>> at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>>>>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
>>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
>>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>>>> at java.lang.Thread.run(Thread.java:637)
>>>>>> Caused by: 2: No such file
>>>>>> at com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2289)
>>>>>> at com.jcraft.jsch.ChannelSftp._realpath(ChannelSftp.java:1822)
>>>>>> at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:268)
>>>>>> at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:350)
>>>>>> ... 15 more
>>>>>>
>>>>>>
>>>>>> Cheers!
>>>>>> -Lorrin
>>>>>>
>>>>>> On Sep 29, 2010, at 12:32 PM, Claus Ibsen wrote:
>>>>>>
>>>>>>> Hi
>>>>>>>
>>>>>>> Can you try again with latest source code from trunk. I have changed
>>>>>>> the logic to change dir one folder at a time.
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Sep 29, 2010 at 7:47 PM, Lorrin Nelson
>>>>>>> <lh...@nerdylorrin.net> wrote:
>>>>>>>> Thanks for the quick response. I'm seeing a different failure now ("Cannot change current directory to: tmp/mult" rather than "Cannot retrieve file: tmp/mult/dummy-5.log"). Is it suspicious that in either case there is no leading /?
>>>>>>>>
>>>>>>>> I tried uris
>>>>>>>> sftp://test@host//tmp/mult?include=.*\.log
>>>>>>>> and
>>>>>>>> sftp://test@host///tmp/mult?include=.*\.log
>>>>>>>>
>>>>>>>> SftpConsumer 2010-09-29 10:31:48,401 -- ERROR -- Caused by: [org.apache.camel.component.file.GenericFileOperationFailedException - Cannot change current directory to: tmp/mult]
>>>>>>>> org.apache.camel.component.file.GenericFileOperationFailedException: Cannot change current directory to: tmp/mult
>>>>>>>> at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:352)
>>>>>>>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:408)
>>>>>>>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:387)
>>>>>>>> at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299)
>>>>>>>> at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
>>>>>>>> at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
>>>>>>>> at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
>>>>>>>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>>>>>>>> at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>>>>>>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>>>>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>>>>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
>>>>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
>>>>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>>>>>> at java.lang.Thread.run(Thread.java:637)
>>>>>>>> Caused by: 2: No such file
>>>>>>>> at com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2289)
>>>>>>>> at com.jcraft.jsch.ChannelSftp._realpath(ChannelSftp.java:1822)
>>>>>>>> at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:268)
>>>>>>>> at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:350)
>>>>>>>> ... 15 more
>>>>>>>>
>>>>>>>> On a tangentially related note, there's some file name processing during the move step that throws a NullPointerException when the URI contains a trailing slash:
>>>>>>>>
>>>>>>>> sftp://test@host/tmp/mult/?include=.*\.log
>>>>>>>>
>>>>>>>> The problem is that GenericFile.changeFileName calls
>>>>>>>> newFileName = ObjectHelper.after(newFileName, endpointPath + getFileSeparator());
>>>>>>>> ...without first checking if endPointPath contains a trailing separator.
>>>>>>>>
>>>>>>>> E.g.
>>>>>>>> newFileName: "tmp/mult/.done/dummy-2.log"
>>>>>>>> endPointPath: "tmp/mult/"
>>>>>>>>
>>>>>>>> ObjectHelper.after():
>>>>>>>> text: "tmp/mult/.done/dummy-2.log"
>>>>>>>> after: "tmp/mult//"
>>>>>>>>
>>>>>>>> text doesn't contain after (because of double //), returns null, changeFileName() doesn't expect null, throws NPE.
>>>>>>>>
>>>>>>>> GenericFileOnCompletion 2010-09-29 10:37:37,888 -- ERROR -- Caused by: [java.lang.NullPointerException - null]
>>>>>>>> java.lang.NullPointerException
>>>>>>>> at java.io.File.<init>(File.java:222)
>>>>>>>> at org.apache.camel.component.file.GenericFile.changeFileName(GenericFile.java:169)
>>>>>>>> at org.apache.camel.component.file.strategy.GenericFileExpressionRenamer.renameFile(GenericFileExpressionRenamer.java:41)
>>>>>>>> at org.apache.camel.component.file.strategy.GenericFileRenameProcessStrategy.commit(GenericFileRenameProcessStrategy.java:82)
>>>>>>>> at org.apache.camel.component.file.GenericFileOnCompletion.processStrategyCommit(GenericFileOnCompletion.java:121)
>>>>>>>> at org.apache.camel.component.file.GenericFileOnCompletion.onCompletion(GenericFileOnCompletion.java:83)
>>>>>>>> at org.apache.camel.component.file.GenericFileOnCompletion.onComplete(GenericFileOnCompletion.java:52)
>>>>>>>> at org.apache.camel.util.UnitOfWorkHelper.doneSynchronizations(UnitOfWorkHelper.java:55)
>>>>>>>> at org.apache.camel.impl.DefaultUnitOfWork.done(DefaultUnitOfWork.java:173)
>>>>>>>> at org.apache.camel.processor.UnitOfWorkProcessor.doneUow(UnitOfWorkProcessor.java:121)
>>>>>>>> at org.apache.camel.processor.UnitOfWorkProcessor.access$000(UnitOfWorkProcessor.java:36)
>>>>>>>> at org.apache.camel.processor.UnitOfWorkProcessor$1.done(UnitOfWorkProcessor.java:106)
>>>>>>>> at org.apache.camel.processor.Pipeline.process(Pipeline.java:130)
>>>>>>>> at org.apache.camel.processor.UnitOfWorkProcessor.process(UnitOfWorkProcessor.java:99)
>>>>>>>> at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:70)
>>>>>>>> at org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:98)
>>>>>>>> at org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:89)
>>>>>>>> at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:68)
>>>>>>>> at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:322)
>>>>>>>> at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
>>>>>>>> at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
>>>>>>>> at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
>>>>>>>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>>>>>>>> at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>>>>>>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>>>>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>>>>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
>>>>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
>>>>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>>>>>> at java.lang.Thread.run(Thread.java:637)
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Sep 29, 2010, at 2:46 AM, Claus Ibsen wrote:
>>>>>>>>
>>>>>>>>> Hi
>>>>>>>>>
>>>>>>>>> I have committed a fix to the SFTP component. Can you try with latest
>>>>>>>>> source code from trunk?
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Tue, Sep 28, 2010 at 7:36 PM, Lorrin <lh...@nerdylorrin.net> wrote:
>>>>>>>>>>
>>>>>>>>>> I just bumped into the same problem. I'm running 2.5-SNAPSHOT.
>>>>>>>>>>
>>>>>>>>>> SftpConsumer 2010-09-28 10:31:28,094 -- ERROR -- Caused by:
>>>>>>>>>> [org.apache.camel.component.file.GenericFileOperationFailedException -
>>>>>>>>>> Cannot retrieve file: tmp/mult/dummy-5.log]
>>>>>>>>>> org.apache.camel.component.file.GenericFileOperationFailedException: Cannot
>>>>>>>>>> retrieve file: tmp/mult/dummy-5.log
>>>>>>>>>> at
>>>>>>>>>> org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:403)
>>>>>>>>>> at
>>>>>>>>>> org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:387)
>>>>>>>>>> at
>>>>>>>>>> org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299)
>>>>>>>>>> at
>>>>>>>>>> org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
>>>>>>>>>> at
>>>>>>>>>> org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
>>>>>>>>>> at
>>>>>>>>>> org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
>>>>>>>>>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>>>>>>>>>> at
>>>>>>>>>> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>>>>>>>>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>>>>>>>>>> at
>>>>>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>>>>>>>>>> at
>>>>>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
>>>>>>>>>> at
>>>>>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
>>>>>>>>>> at
>>>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>>>>>>>> at
>>>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>>>>>>>> at java.lang.Thread.run(Thread.java:637)
>>>>>>>>>> Caused by: 2: No such file
>>>>>>>>>> at com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2289)
>>>>>>>>>> at com.jcraft.jsch.ChannelSftp._get(ChannelSftp.java:901)
>>>>>>>>>> at com.jcraft.jsch.ChannelSftp.get(ChannelSftp.java:870)
>>>>>>>>>> at com.jcraft.jsch.ChannelSftp.get(ChannelSftp.java:849)
>>>>>>>>>> at
>>>>>>>>>> org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:400)
>>>>>>>>>> ... 14 more
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> View this message in context: http://camel.465427.n5.nabble.com/How-to-change-directory-while-using-sftp-component-tp2806817p2857187.html
>>>>>>>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Claus Ibsen
>>>>>>>>> Apache Camel Committer
>>>>>>>>>
>>>>>>>>> Author of Camel in Action: http://www.manning.com/ibsen/
>>>>>>>>> Open Source Integration: http://fusesource.com
>>>>>>>>> Blog: http://davsclaus.blogspot.com/
>>>>>>>>> Twitter: http://twitter.com/davsclaus
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Claus Ibsen
>>>>>>> Apache Camel Committer
>>>>>>>
>>>>>>> Author of Camel in Action: http://www.manning.com/ibsen/
>>>>>>> Open Source Integration: http://fusesource.com
>>>>>>> Blog: http://davsclaus.blogspot.com/
>>>>>>> Twitter: http://twitter.com/davsclaus
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Claus Ibsen
>>>>> Apache Camel Committer
>>>>>
>>>>> Author of Camel in Action: http://www.manning.com/ibsen/
>>>>> Open Source Integration: http://fusesource.com
>>>>> Blog: http://davsclaus.blogspot.com/
>>>>> Twitter: http://twitter.com/davsclaus
>>>>>
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Claus Ibsen
>>> Apache Camel Committer
>>>
>>> Author of Camel in Action: http://www.manning.com/ibsen/
>>> Open Source Integration: http://fusesource.com
>>> Blog: http://davsclaus.blogspot.com/
>>> Twitter: http://twitter.com/davsclaus
>>>
>>
>>
>
>
>
> --
> Claus Ibsen
> Apache Camel Committer
>
> Author of Camel in Action: http://www.manning.com/ibsen/
> Open Source Integration: http://fusesource.com
> Blog: http://davsclaus.blogspot.com/
> Twitter: http://twitter.com/davsclaus
>
Re: How to change directory while using sftp component
Posted by Claus Ibsen <cl...@gmail.com>.
Hi
Great that the relative works. This is also what I expected.
In terms of absolute then Camel changes the directory to / (since its
absolute) and the SFTP library returns OK for this.
But in reality it does not go to the root path, but most likely just
goes to the home directory of the user.
And hence why absolute does not work for you.
I don't see any API or way to tell SFTP library it should change to
root path and not home folder.
A solution to be able to poll files from an absolute folder is to
create a symlink in the user home directory which can be polled by the
SFTP library.
> nlike the polling stage where it says "doPollDirectory from absolutePath: /tmp/activity_logs".
The reason why you see this is because Camel logs the path configured
from the endpoint, when the poller starts. Hence you see the leading
/.
On Tue, Oct 5, 2010 at 12:58 AM, Lorrin Nelson
<lh...@nerdylorrin.net> wrote:
> Hi Claus, thanks for the continued support. With the latest code (revision 1003927) relative URIs are working again for me.
>
> This the first time I'm using camel-ftp. The server I'm talking to is a Fedora 11 box running OpenSSH_5.2p1. Regular FTP is an unlikely option for us. Relative URLs are an acceptable work-around.
>
> Below is a trace log of an absolute URL.
>
> On the first attempt, note that ~/tmp does exist. Therefore when it tries to go to *relative* path tmp (which is the wrong thing to do), it succeeds. Then when it tries to go one level deeper into pitch_activity_logs it fails. The second attempt is really interesting. Note there that the current working directory has remained at /home/tomcat/tmp. So when it fails to go to *relative* path tmp, it fails right away. BTW, note that it says "Retrieving file: tmp/activity_logs/activity_log.1285909200" with no leading slash, unlike the polling stage where it says "doPollDirectory from absolutePath: /tmp/activity_logs".
>
> Cheers
> -Lorrin
>
>
> SftpConsumer 2010-10-04 15:45:07,445 -- INFO -- Connected and logged in to: sftp://tomcat@host:22
> SftpConsumer 2010-10-04 15:45:07,494 -- TRACE -- doPollDirectory from absolutePath: /tmp/activity_logs, dirName: null
> SftpOperations 2010-10-04 15:45:07,494 -- TRACE -- Changing directory: /
> SftpOperations 2010-10-04 15:45:07,608 -- TRACE -- Changing directory: tmp
> SftpOperations 2010-10-04 15:45:07,697 -- TRACE -- Changing directory: activity_logs
> SftpConsumer 2010-10-04 15:45:07,788 -- TRACE -- Polling directory: /tmp/activity_logs
> SftpConsumer 2010-10-04 15:45:08,011 -- TRACE -- Found 4 in directory: /tmp/activity_logs
> SftpOperations 2010-10-04 15:45:08,012 -- TRACE -- Changing directory: /
> SftpOperations 2010-10-04 15:45:08,099 -- TRACE -- Changing directory: home
> SftpOperations 2010-10-04 15:45:08,189 -- TRACE -- Changing directory: tomcat
> SftpConsumer 2010-10-04 15:45:08,274 -- DEBUG -- Took 0.829 seconds to poll: /tmp/activity_logs/
> SftpConsumer 2010-10-04 15:45:08,275 -- DEBUG -- Total 1 files to consume
> SftpConsumer 2010-10-04 15:45:08,276 -- TRACE -- Processing file: GenericFile[activity_log.1285909200]
> SftpConsumer 2010-10-04 15:45:08,276 -- TRACE -- Retrieving file: tmp/activity_logs/activity_log.1285909200 from: Endpoint[sftp://tomcat@host//tmp/activity_logs/?delay=15000&idempotent=true&idempotentRepository=%23apacheLogFilenameRepositoryAcceptNewest&include=.*.log.%5B-%2F%3A0-9%5D*&initialDelay=3000&noop=true&password=******]
> SftpOperations 2010-10-04 15:45:08,277 -- TRACE -- Changing directory: tmp
> SftpOperations 2010-10-04 15:45:08,367 -- TRACE -- Changing directory: activity_logs
> SftpConsumer 2010-10-04 15:45:08,414 -- ERROR -- Caused by: [org.apache.camel.component.file.GenericFileOperationFailedException - Cannot change directory to: activity_logs]
> org.apache.camel.component.file.GenericFileOperationFailedException: Cannot change directory to: activity_logs
> at org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:383)
> at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:368)
> at org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:451)
> at org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:430)
> at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299)
> at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
> at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
> at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
> at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:637)
> Caused by: 2: No such file
> at com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2289)
> at com.jcraft.jsch.ChannelSftp._realpath(ChannelSftp.java:1822)
> at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:268)
> at org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:381)
> ... 16 more
> ScheduledPollConsumer 2010-10-04 15:45:08,426 -- TRACE -- Finished polling: Endpoint[sftp://tomcat@host//tmp/activity_logs/?delay=15000&idempotent=true&idempotentRepository=%23apacheLogFilenameRepositoryAcceptNewest&include=.*.log.%5B-%2F%3A0-9%5D*&initialDelay=3000&noop=true&password=******]
>
>
>
> ScheduledPollConsumer 2010-10-04 15:45:21,413 -- TRACE -- Starting to poll: Endpoint[sftp://tomcat@host//tmp/activity_logs/?delay=15000&idempotent=true&idempotentRepository=%23apacheLogFilenameRepositoryAcceptNewest&include=.*.log.%5B-%2F%3A0-9%5D*&initialDelay=3000&noop=true&password=******]
> SftpConsumer 2010-10-04 15:45:21,413 -- TRACE -- doPollDirectory from absolutePath: /tmp/activity_logs, dirName: null
> SftpOperations 2010-10-04 15:45:21,413 -- TRACE -- Changing directory: /
> SftpOperations 2010-10-04 15:45:21,496 -- TRACE -- Changing directory: tmp
> SftpOperations 2010-10-04 15:45:21,585 -- TRACE -- Changing directory: activity_logs
> SftpConsumer 2010-10-04 15:45:21,675 -- TRACE -- Polling directory: /tmp/activity_logs
> SftpConsumer 2010-10-04 15:45:21,907 -- TRACE -- Found 4 in directory: /tmp/activity_logs
> SftpOperations 2010-10-04 15:45:21,907 -- TRACE -- Changing directory: /
> SftpOperations 2010-10-04 15:45:21,997 -- TRACE -- Changing directory: home
> SftpOperations 2010-10-04 15:45:22,084 -- TRACE -- Changing directory: tomcat
> SftpOperations 2010-10-04 15:45:22,177 -- TRACE -- Changing directory: tmp
> SftpConsumer 2010-10-04 15:45:22,261 -- DEBUG -- Took 0.848 seconds to poll: /tmp/activity_logs/
> SftpConsumer 2010-10-04 15:45:22,261 -- DEBUG -- Total 1 files to consume
> SftpConsumer 2010-10-04 15:45:22,261 -- TRACE -- Processing file: GenericFile[activity_log.1285909200]
> SftpConsumer 2010-10-04 15:45:22,261 -- TRACE -- Retrieving file: tmp/activity_logs/activity_log.1285909200 from: Endpoint[sftp://tomcat@host//tmp/activity_logs/?delay=15000&idempotent=true&idempotentRepository=%23apacheLogFilenameRepositoryAcceptNewest&include=.*.log.%5B-%2F%3A0-9%5D*&initialDelay=3000&noop=true&password=******]
> SftpOperations 2010-10-04 15:45:22,262 -- TRACE -- Changing directory: tmp
> SftpConsumer 2010-10-04 15:45:22,305 -- ERROR -- Caused by: [org.apache.camel.component.file.GenericFileOperationFailedException - Cannot change directory to: tmp]
> org.apache.camel.component.file.GenericFileOperationFailedException: Cannot change directory to: tmp
> at org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:383)
> at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:368)
> at org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:451)
> at org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:430)
> at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299)
> at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
> at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
> at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
> at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:637)
> Caused by: 2: No such file
> at com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2289)
> at com.jcraft.jsch.ChannelSftp._realpath(ChannelSftp.java:1822)
> at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:268)
> at org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:381)
> ... 16 more
> ScheduledPollConsumer 2010-10-04 15:45:22,311 -- TRACE -- Finished polling: Endpoint[sftp://tomcat@host//tmp/activity_logs/?delay=15000&idempotent=true&idempotentRepository=%23apacheLogFilenameRepositoryAcceptNewest&include=.*.log.%5B-%2F%3A0-9%5D*&initialDelay=3000&noop=true&password=******]
>
>
>
>
> On Oct 2, 2010, at 12:14 AM, Claus Ibsen wrote:
>
>> Hi Loririn
>>
>> I think you/we have to step back. We have other users in the past
>> running with SFTP with no problems at all. Also fetching files
>> recursively.
>> So I wonder if we should revert back and keep the old logic!
>>
>> 1)
>> Is this the first time you use camel-ftp?
>>
>> 2)
>> What OS and FTP server are you using?
>>
>> 3)
>> Can you use regular FTP instead?
>>
>> 4)
>> Use relative urls over absolute. They where the first supported.
>>
>> 5)
>> Can you enable TRACE logging on org.apache.camel.component.file and
>> setup a simple example so it makes it easier to follow and understand
>> what its supposed to do, and what happens in reality so we can see and
>> understand the difference.
>>
>>
>>
>>
>> On Fri, Oct 1, 2010 at 10:45 PM, Lorrin Nelson
>> <lh...@nerdylorrin.net> wrote:
>>> Hi Claus. Ack, sorry about that. I'm pulling through github rather than SVN directly and then running via a Maven project pointing at my locally built Camel, so it's sometimes confusing how up-to-date I am.
>>>
>>> So I think I have current code, including checkin "CAMEL-3174: Changing dir with ftp must do one dir at a time." and I think SFTP polling is now broken for both absolute and relative URLs. See debug walk-throughs and stack traces below.
>>>
>>> Absolute URI with double leading /:
>>> sftp://user@host//home/user/subdir?idempotent=true&idempotentRepository=#myCustomIR&include=.*.log&noop=true&password=******
>>>
>>> SftpOperations.changeCurrentDirectory("home/user/subdir") //note there's no leading slash on the supposedly absolute path!
>>> //splits into chunks successfully, starts looping through chunks
>>> doChangeDirectory("home")
>>> ChannelSftp.cd("home")
>>> //path=remoteAbsolutePath(path);
>>> //yields "/home/user/home". Doh!
>>>
>>>
>>> Relative URI with single leading /:
>>> sftp://user@host/subdir?idempotent=true&idempotentRepository=#myCustomIR&include=.*.log&noop=true&password=******
>>> SftpOperations.changeCurrentDirectory("subdir")
>>> //splits into single chunk, starts looping through chunks
>>> doChangeDirectory("subdir")
>>> ChannelSftp.cd("home")
>>> //path=remoteAbsolutePath(path);
>>> //yields "/home/user/subdir". yay!
>>>
>>> ..but then, back in SftpOperations.retrieveFileToStreamInBody, it hits the "change back to current directory code"
>>> changeCurrentDirectory("/home/tomcat"); //now we have a leading slash, unlike before
>>> //splits info chunks "", "home", and "tomcat". Doh!
>>> doChangeDirectory("")
>>> ChannelSftp.cd("")
>>> //path=remoteAbsolutePath(path) throws a StringIndexOutOfBoundsException when it receives empty string.
>>>
>>>
>>> Stack trace when attempting to supply absolute path:
>>> org.apache.camel.component.file.GenericFileOperationFailedException: Cannot change directory to: home
>>> at org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:372)
>>> at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:361)
>>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:428)
>>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:407)
>>> at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299)
>>> at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
>>> at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
>>> at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
>>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>>> at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>> at java.lang.Thread.run(Thread.java:637)
>>> Caused by: 2: No such file
>>> at com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2289)
>>> at com.jcraft.jsch.ChannelSftp._realpath(ChannelSftp.java:1822)
>>> at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:268)
>>> at org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:370)
>>> ... 16 more
>>>
>>>
>>> Stack trace when relative URL tries to restore current working directory:
>>> SftpConsumer 2010-10-01 13:41:58,529 -- ERROR -- Caused by: [org.apache.camel.component.file.GenericFileOperationFailedException - Cannot change directory to: ]
>>> org.apache.camel.component.file.GenericFileOperationFailedException: Cannot change directory to:
>>> at org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:372)
>>> at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:361)
>>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:435)
>>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:407)
>>> at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299)
>>> at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
>>> at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
>>> at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
>>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>>> at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>> at java.lang.Thread.run(Thread.java:637)
>>> Caused by: 4:
>>> at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:285)
>>> at org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:370)
>>> ... 16 more
>>> Caused by: java.lang.StringIndexOutOfBoundsException: String index out of range: 0
>>> at java.lang.String.charAt(String.java:686)
>>> at com.jcraft.jsch.ChannelSftp.remoteAbsolutePath(ChannelSftp.java:2359)
>>> at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:264)
>>> ... 17 more
>>>
>>> Cheers
>>> -Lorrin
>>>
>>> On Sep 29, 2010, at 9:37 PM, Claus Ibsen wrote:
>>>
>>>> Hi
>>>>
>>>> You have not used the latest source code. The stacktrace points to
>>>> empty lines etc.
>>>> Make sure you build camel-core and camel-ftp using latest source code.
>>>>
>>>> Revision: 1002946
>>>> Node Kind: directory
>>>> Schedule: normal
>>>> Last Changed Author: davsclaus
>>>> Last Changed Rev: 1002821
>>>> Last Changed Date: 2010-09-29 21:31:23 +0200 (Wed, 29 Sep 2010)
>>>>
>>>>
>>>> On Wed, Sep 29, 2010 at 11:55 PM, Lorrin Nelson
>>>> <lh...@nerdylorrin.net> wrote:
>>>>> I'm pretty sure I was already at revision 1002541 before. Now I definitely am. Same Exception. I think the problem is the lack of leading /.
>>>>>
>>>>> SftpOperations.retrieveFileToStreamInBody computes path as "tmp/mult", calls changeCurrentDirectory(path), which in turn calls channel.cd(path), which (now we're in com.jcraft.jsch.ChannelSftp) calls path=remoteAbsolutePath(path); and computes path: "/home/test/tmp/mult", which does not exist.
>>>>>
>>>>> SftpConsumer 2010-09-29 14:51:50,487 -- ERROR -- Caused by: [org.apache.camel.component.file.GenericFileOperationFailedException - Cannot change current directory to: tmp/mult]
>>>>> org.apache.camel.component.file.GenericFileOperationFailedException: Cannot change current directory to: tmp/mult
>>>>> at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:352)
>>>>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:408)
>>>>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:387)
>>>>> at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299)
>>>>> at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
>>>>> at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
>>>>> at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
>>>>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>>>>> at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>>>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>>> at java.lang.Thread.run(Thread.java:637)
>>>>> Caused by: 2: No such file
>>>>> at com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2289)
>>>>> at com.jcraft.jsch.ChannelSftp._realpath(ChannelSftp.java:1822)
>>>>> at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:268)
>>>>> at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:350)
>>>>> ... 15 more
>>>>>
>>>>>
>>>>> Cheers!
>>>>> -Lorrin
>>>>>
>>>>> On Sep 29, 2010, at 12:32 PM, Claus Ibsen wrote:
>>>>>
>>>>>> Hi
>>>>>>
>>>>>> Can you try again with latest source code from trunk. I have changed
>>>>>> the logic to change dir one folder at a time.
>>>>>>
>>>>>>
>>>>>> On Wed, Sep 29, 2010 at 7:47 PM, Lorrin Nelson
>>>>>> <lh...@nerdylorrin.net> wrote:
>>>>>>> Thanks for the quick response. I'm seeing a different failure now ("Cannot change current directory to: tmp/mult" rather than "Cannot retrieve file: tmp/mult/dummy-5.log"). Is it suspicious that in either case there is no leading /?
>>>>>>>
>>>>>>> I tried uris
>>>>>>> sftp://test@host//tmp/mult?include=.*\.log
>>>>>>> and
>>>>>>> sftp://test@host///tmp/mult?include=.*\.log
>>>>>>>
>>>>>>> SftpConsumer 2010-09-29 10:31:48,401 -- ERROR -- Caused by: [org.apache.camel.component.file.GenericFileOperationFailedException - Cannot change current directory to: tmp/mult]
>>>>>>> org.apache.camel.component.file.GenericFileOperationFailedException: Cannot change current directory to: tmp/mult
>>>>>>> at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:352)
>>>>>>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:408)
>>>>>>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:387)
>>>>>>> at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299)
>>>>>>> at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
>>>>>>> at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
>>>>>>> at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
>>>>>>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>>>>>>> at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>>>>>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>>>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>>>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
>>>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
>>>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>>>>> at java.lang.Thread.run(Thread.java:637)
>>>>>>> Caused by: 2: No such file
>>>>>>> at com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2289)
>>>>>>> at com.jcraft.jsch.ChannelSftp._realpath(ChannelSftp.java:1822)
>>>>>>> at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:268)
>>>>>>> at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:350)
>>>>>>> ... 15 more
>>>>>>>
>>>>>>> On a tangentially related note, there's some file name processing during the move step that throws a NullPointerException when the URI contains a trailing slash:
>>>>>>>
>>>>>>> sftp://test@host/tmp/mult/?include=.*\.log
>>>>>>>
>>>>>>> The problem is that GenericFile.changeFileName calls
>>>>>>> newFileName = ObjectHelper.after(newFileName, endpointPath + getFileSeparator());
>>>>>>> ...without first checking if endPointPath contains a trailing separator.
>>>>>>>
>>>>>>> E.g.
>>>>>>> newFileName: "tmp/mult/.done/dummy-2.log"
>>>>>>> endPointPath: "tmp/mult/"
>>>>>>>
>>>>>>> ObjectHelper.after():
>>>>>>> text: "tmp/mult/.done/dummy-2.log"
>>>>>>> after: "tmp/mult//"
>>>>>>>
>>>>>>> text doesn't contain after (because of double //), returns null, changeFileName() doesn't expect null, throws NPE.
>>>>>>>
>>>>>>> GenericFileOnCompletion 2010-09-29 10:37:37,888 -- ERROR -- Caused by: [java.lang.NullPointerException - null]
>>>>>>> java.lang.NullPointerException
>>>>>>> at java.io.File.<init>(File.java:222)
>>>>>>> at org.apache.camel.component.file.GenericFile.changeFileName(GenericFile.java:169)
>>>>>>> at org.apache.camel.component.file.strategy.GenericFileExpressionRenamer.renameFile(GenericFileExpressionRenamer.java:41)
>>>>>>> at org.apache.camel.component.file.strategy.GenericFileRenameProcessStrategy.commit(GenericFileRenameProcessStrategy.java:82)
>>>>>>> at org.apache.camel.component.file.GenericFileOnCompletion.processStrategyCommit(GenericFileOnCompletion.java:121)
>>>>>>> at org.apache.camel.component.file.GenericFileOnCompletion.onCompletion(GenericFileOnCompletion.java:83)
>>>>>>> at org.apache.camel.component.file.GenericFileOnCompletion.onComplete(GenericFileOnCompletion.java:52)
>>>>>>> at org.apache.camel.util.UnitOfWorkHelper.doneSynchronizations(UnitOfWorkHelper.java:55)
>>>>>>> at org.apache.camel.impl.DefaultUnitOfWork.done(DefaultUnitOfWork.java:173)
>>>>>>> at org.apache.camel.processor.UnitOfWorkProcessor.doneUow(UnitOfWorkProcessor.java:121)
>>>>>>> at org.apache.camel.processor.UnitOfWorkProcessor.access$000(UnitOfWorkProcessor.java:36)
>>>>>>> at org.apache.camel.processor.UnitOfWorkProcessor$1.done(UnitOfWorkProcessor.java:106)
>>>>>>> at org.apache.camel.processor.Pipeline.process(Pipeline.java:130)
>>>>>>> at org.apache.camel.processor.UnitOfWorkProcessor.process(UnitOfWorkProcessor.java:99)
>>>>>>> at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:70)
>>>>>>> at org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:98)
>>>>>>> at org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:89)
>>>>>>> at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:68)
>>>>>>> at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:322)
>>>>>>> at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
>>>>>>> at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
>>>>>>> at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
>>>>>>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>>>>>>> at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>>>>>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>>>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>>>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
>>>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
>>>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>>>>> at java.lang.Thread.run(Thread.java:637)
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Sep 29, 2010, at 2:46 AM, Claus Ibsen wrote:
>>>>>>>
>>>>>>>> Hi
>>>>>>>>
>>>>>>>> I have committed a fix to the SFTP component. Can you try with latest
>>>>>>>> source code from trunk?
>>>>>>>>
>>>>>>>>
>>>>>>>> On Tue, Sep 28, 2010 at 7:36 PM, Lorrin <lh...@nerdylorrin.net> wrote:
>>>>>>>>>
>>>>>>>>> I just bumped into the same problem. I'm running 2.5-SNAPSHOT.
>>>>>>>>>
>>>>>>>>> SftpConsumer 2010-09-28 10:31:28,094 -- ERROR -- Caused by:
>>>>>>>>> [org.apache.camel.component.file.GenericFileOperationFailedException -
>>>>>>>>> Cannot retrieve file: tmp/mult/dummy-5.log]
>>>>>>>>> org.apache.camel.component.file.GenericFileOperationFailedException: Cannot
>>>>>>>>> retrieve file: tmp/mult/dummy-5.log
>>>>>>>>> at
>>>>>>>>> org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:403)
>>>>>>>>> at
>>>>>>>>> org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:387)
>>>>>>>>> at
>>>>>>>>> org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299)
>>>>>>>>> at
>>>>>>>>> org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
>>>>>>>>> at
>>>>>>>>> org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
>>>>>>>>> at
>>>>>>>>> org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
>>>>>>>>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>>>>>>>>> at
>>>>>>>>> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>>>>>>>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>>>>>>>>> at
>>>>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>>>>>>>>> at
>>>>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
>>>>>>>>> at
>>>>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
>>>>>>>>> at
>>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>>>>>>> at
>>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>>>>>>> at java.lang.Thread.run(Thread.java:637)
>>>>>>>>> Caused by: 2: No such file
>>>>>>>>> at com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2289)
>>>>>>>>> at com.jcraft.jsch.ChannelSftp._get(ChannelSftp.java:901)
>>>>>>>>> at com.jcraft.jsch.ChannelSftp.get(ChannelSftp.java:870)
>>>>>>>>> at com.jcraft.jsch.ChannelSftp.get(ChannelSftp.java:849)
>>>>>>>>> at
>>>>>>>>> org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:400)
>>>>>>>>> ... 14 more
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> View this message in context: http://camel.465427.n5.nabble.com/How-to-change-directory-while-using-sftp-component-tp2806817p2857187.html
>>>>>>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Claus Ibsen
>>>>>>>> Apache Camel Committer
>>>>>>>>
>>>>>>>> Author of Camel in Action: http://www.manning.com/ibsen/
>>>>>>>> Open Source Integration: http://fusesource.com
>>>>>>>> Blog: http://davsclaus.blogspot.com/
>>>>>>>> Twitter: http://twitter.com/davsclaus
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Claus Ibsen
>>>>>> Apache Camel Committer
>>>>>>
>>>>>> Author of Camel in Action: http://www.manning.com/ibsen/
>>>>>> Open Source Integration: http://fusesource.com
>>>>>> Blog: http://davsclaus.blogspot.com/
>>>>>> Twitter: http://twitter.com/davsclaus
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Claus Ibsen
>>>> Apache Camel Committer
>>>>
>>>> Author of Camel in Action: http://www.manning.com/ibsen/
>>>> Open Source Integration: http://fusesource.com
>>>> Blog: http://davsclaus.blogspot.com/
>>>> Twitter: http://twitter.com/davsclaus
>>>>
>>>
>>>
>>
>>
>>
>> --
>> Claus Ibsen
>> Apache Camel Committer
>>
>> Author of Camel in Action: http://www.manning.com/ibsen/
>> Open Source Integration: http://fusesource.com
>> Blog: http://davsclaus.blogspot.com/
>> Twitter: http://twitter.com/davsclaus
>>
>
>
--
Claus Ibsen
Apache Camel Committer
Author of Camel in Action: http://www.manning.com/ibsen/
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus
Re: How to change directory while using sftp component
Posted by Lorrin Nelson <lh...@nerdylorrin.net>.
Hi Claus, thanks for the continued support. With the latest code (revision 1003927) relative URIs are working again for me.
This the first time I'm using camel-ftp. The server I'm talking to is a Fedora 11 box running OpenSSH_5.2p1. Regular FTP is an unlikely option for us. Relative URLs are an acceptable work-around.
Below is a trace log of an absolute URL.
On the first attempt, note that ~/tmp does exist. Therefore when it tries to go to *relative* path tmp (which is the wrong thing to do), it succeeds. Then when it tries to go one level deeper into pitch_activity_logs it fails. The second attempt is really interesting. Note there that the current working directory has remained at /home/tomcat/tmp. So when it fails to go to *relative* path tmp, it fails right away. BTW, note that it says "Retrieving file: tmp/activity_logs/activity_log.1285909200" with no leading slash, unlike the polling stage where it says "doPollDirectory from absolutePath: /tmp/activity_logs".
Cheers
-Lorrin
SftpConsumer 2010-10-04 15:45:07,445 -- INFO -- Connected and logged in to: sftp://tomcat@host:22
SftpConsumer 2010-10-04 15:45:07,494 -- TRACE -- doPollDirectory from absolutePath: /tmp/activity_logs, dirName: null
SftpOperations 2010-10-04 15:45:07,494 -- TRACE -- Changing directory: /
SftpOperations 2010-10-04 15:45:07,608 -- TRACE -- Changing directory: tmp
SftpOperations 2010-10-04 15:45:07,697 -- TRACE -- Changing directory: activity_logs
SftpConsumer 2010-10-04 15:45:07,788 -- TRACE -- Polling directory: /tmp/activity_logs
SftpConsumer 2010-10-04 15:45:08,011 -- TRACE -- Found 4 in directory: /tmp/activity_logs
SftpOperations 2010-10-04 15:45:08,012 -- TRACE -- Changing directory: /
SftpOperations 2010-10-04 15:45:08,099 -- TRACE -- Changing directory: home
SftpOperations 2010-10-04 15:45:08,189 -- TRACE -- Changing directory: tomcat
SftpConsumer 2010-10-04 15:45:08,274 -- DEBUG -- Took 0.829 seconds to poll: /tmp/activity_logs/
SftpConsumer 2010-10-04 15:45:08,275 -- DEBUG -- Total 1 files to consume
SftpConsumer 2010-10-04 15:45:08,276 -- TRACE -- Processing file: GenericFile[activity_log.1285909200]
SftpConsumer 2010-10-04 15:45:08,276 -- TRACE -- Retrieving file: tmp/activity_logs/activity_log.1285909200 from: Endpoint[sftp://tomcat@host//tmp/activity_logs/?delay=15000&idempotent=true&idempotentRepository=%23apacheLogFilenameRepositoryAcceptNewest&include=.*.log.%5B-%2F%3A0-9%5D*&initialDelay=3000&noop=true&password=******]
SftpOperations 2010-10-04 15:45:08,277 -- TRACE -- Changing directory: tmp
SftpOperations 2010-10-04 15:45:08,367 -- TRACE -- Changing directory: activity_logs
SftpConsumer 2010-10-04 15:45:08,414 -- ERROR -- Caused by: [org.apache.camel.component.file.GenericFileOperationFailedException - Cannot change directory to: activity_logs]
org.apache.camel.component.file.GenericFileOperationFailedException: Cannot change directory to: activity_logs
at org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:383)
at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:368)
at org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:451)
at org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:430)
at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299)
at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:637)
Caused by: 2: No such file
at com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2289)
at com.jcraft.jsch.ChannelSftp._realpath(ChannelSftp.java:1822)
at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:268)
at org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:381)
... 16 more
ScheduledPollConsumer 2010-10-04 15:45:08,426 -- TRACE -- Finished polling: Endpoint[sftp://tomcat@host//tmp/activity_logs/?delay=15000&idempotent=true&idempotentRepository=%23apacheLogFilenameRepositoryAcceptNewest&include=.*.log.%5B-%2F%3A0-9%5D*&initialDelay=3000&noop=true&password=******]
ScheduledPollConsumer 2010-10-04 15:45:21,413 -- TRACE -- Starting to poll: Endpoint[sftp://tomcat@host//tmp/activity_logs/?delay=15000&idempotent=true&idempotentRepository=%23apacheLogFilenameRepositoryAcceptNewest&include=.*.log.%5B-%2F%3A0-9%5D*&initialDelay=3000&noop=true&password=******]
SftpConsumer 2010-10-04 15:45:21,413 -- TRACE -- doPollDirectory from absolutePath: /tmp/activity_logs, dirName: null
SftpOperations 2010-10-04 15:45:21,413 -- TRACE -- Changing directory: /
SftpOperations 2010-10-04 15:45:21,496 -- TRACE -- Changing directory: tmp
SftpOperations 2010-10-04 15:45:21,585 -- TRACE -- Changing directory: activity_logs
SftpConsumer 2010-10-04 15:45:21,675 -- TRACE -- Polling directory: /tmp/activity_logs
SftpConsumer 2010-10-04 15:45:21,907 -- TRACE -- Found 4 in directory: /tmp/activity_logs
SftpOperations 2010-10-04 15:45:21,907 -- TRACE -- Changing directory: /
SftpOperations 2010-10-04 15:45:21,997 -- TRACE -- Changing directory: home
SftpOperations 2010-10-04 15:45:22,084 -- TRACE -- Changing directory: tomcat
SftpOperations 2010-10-04 15:45:22,177 -- TRACE -- Changing directory: tmp
SftpConsumer 2010-10-04 15:45:22,261 -- DEBUG -- Took 0.848 seconds to poll: /tmp/activity_logs/
SftpConsumer 2010-10-04 15:45:22,261 -- DEBUG -- Total 1 files to consume
SftpConsumer 2010-10-04 15:45:22,261 -- TRACE -- Processing file: GenericFile[activity_log.1285909200]
SftpConsumer 2010-10-04 15:45:22,261 -- TRACE -- Retrieving file: tmp/activity_logs/activity_log.1285909200 from: Endpoint[sftp://tomcat@host//tmp/activity_logs/?delay=15000&idempotent=true&idempotentRepository=%23apacheLogFilenameRepositoryAcceptNewest&include=.*.log.%5B-%2F%3A0-9%5D*&initialDelay=3000&noop=true&password=******]
SftpOperations 2010-10-04 15:45:22,262 -- TRACE -- Changing directory: tmp
SftpConsumer 2010-10-04 15:45:22,305 -- ERROR -- Caused by: [org.apache.camel.component.file.GenericFileOperationFailedException - Cannot change directory to: tmp]
org.apache.camel.component.file.GenericFileOperationFailedException: Cannot change directory to: tmp
at org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:383)
at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:368)
at org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:451)
at org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:430)
at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299)
at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:637)
Caused by: 2: No such file
at com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2289)
at com.jcraft.jsch.ChannelSftp._realpath(ChannelSftp.java:1822)
at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:268)
at org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:381)
... 16 more
ScheduledPollConsumer 2010-10-04 15:45:22,311 -- TRACE -- Finished polling: Endpoint[sftp://tomcat@host//tmp/activity_logs/?delay=15000&idempotent=true&idempotentRepository=%23apacheLogFilenameRepositoryAcceptNewest&include=.*.log.%5B-%2F%3A0-9%5D*&initialDelay=3000&noop=true&password=******]
On Oct 2, 2010, at 12:14 AM, Claus Ibsen wrote:
> Hi Loririn
>
> I think you/we have to step back. We have other users in the past
> running with SFTP with no problems at all. Also fetching files
> recursively.
> So I wonder if we should revert back and keep the old logic!
>
> 1)
> Is this the first time you use camel-ftp?
>
> 2)
> What OS and FTP server are you using?
>
> 3)
> Can you use regular FTP instead?
>
> 4)
> Use relative urls over absolute. They where the first supported.
>
> 5)
> Can you enable TRACE logging on org.apache.camel.component.file and
> setup a simple example so it makes it easier to follow and understand
> what its supposed to do, and what happens in reality so we can see and
> understand the difference.
>
>
>
>
> On Fri, Oct 1, 2010 at 10:45 PM, Lorrin Nelson
> <lh...@nerdylorrin.net> wrote:
>> Hi Claus. Ack, sorry about that. I'm pulling through github rather than SVN directly and then running via a Maven project pointing at my locally built Camel, so it's sometimes confusing how up-to-date I am.
>>
>> So I think I have current code, including checkin "CAMEL-3174: Changing dir with ftp must do one dir at a time." and I think SFTP polling is now broken for both absolute and relative URLs. See debug walk-throughs and stack traces below.
>>
>> Absolute URI with double leading /:
>> sftp://user@host//home/user/subdir?idempotent=true&idempotentRepository=#myCustomIR&include=.*.log&noop=true&password=******
>>
>> SftpOperations.changeCurrentDirectory("home/user/subdir") //note there's no leading slash on the supposedly absolute path!
>> //splits into chunks successfully, starts looping through chunks
>> doChangeDirectory("home")
>> ChannelSftp.cd("home")
>> //path=remoteAbsolutePath(path);
>> //yields "/home/user/home". Doh!
>>
>>
>> Relative URI with single leading /:
>> sftp://user@host/subdir?idempotent=true&idempotentRepository=#myCustomIR&include=.*.log&noop=true&password=******
>> SftpOperations.changeCurrentDirectory("subdir")
>> //splits into single chunk, starts looping through chunks
>> doChangeDirectory("subdir")
>> ChannelSftp.cd("home")
>> //path=remoteAbsolutePath(path);
>> //yields "/home/user/subdir". yay!
>>
>> ..but then, back in SftpOperations.retrieveFileToStreamInBody, it hits the "change back to current directory code"
>> changeCurrentDirectory("/home/tomcat"); //now we have a leading slash, unlike before
>> //splits info chunks "", "home", and "tomcat". Doh!
>> doChangeDirectory("")
>> ChannelSftp.cd("")
>> //path=remoteAbsolutePath(path) throws a StringIndexOutOfBoundsException when it receives empty string.
>>
>>
>> Stack trace when attempting to supply absolute path:
>> org.apache.camel.component.file.GenericFileOperationFailedException: Cannot change directory to: home
>> at org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:372)
>> at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:361)
>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:428)
>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:407)
>> at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299)
>> at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
>> at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
>> at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>> at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>> at java.lang.Thread.run(Thread.java:637)
>> Caused by: 2: No such file
>> at com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2289)
>> at com.jcraft.jsch.ChannelSftp._realpath(ChannelSftp.java:1822)
>> at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:268)
>> at org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:370)
>> ... 16 more
>>
>>
>> Stack trace when relative URL tries to restore current working directory:
>> SftpConsumer 2010-10-01 13:41:58,529 -- ERROR -- Caused by: [org.apache.camel.component.file.GenericFileOperationFailedException - Cannot change directory to: ]
>> org.apache.camel.component.file.GenericFileOperationFailedException: Cannot change directory to:
>> at org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:372)
>> at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:361)
>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:435)
>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:407)
>> at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299)
>> at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
>> at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
>> at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>> at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>> at java.lang.Thread.run(Thread.java:637)
>> Caused by: 4:
>> at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:285)
>> at org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:370)
>> ... 16 more
>> Caused by: java.lang.StringIndexOutOfBoundsException: String index out of range: 0
>> at java.lang.String.charAt(String.java:686)
>> at com.jcraft.jsch.ChannelSftp.remoteAbsolutePath(ChannelSftp.java:2359)
>> at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:264)
>> ... 17 more
>>
>> Cheers
>> -Lorrin
>>
>> On Sep 29, 2010, at 9:37 PM, Claus Ibsen wrote:
>>
>>> Hi
>>>
>>> You have not used the latest source code. The stacktrace points to
>>> empty lines etc.
>>> Make sure you build camel-core and camel-ftp using latest source code.
>>>
>>> Revision: 1002946
>>> Node Kind: directory
>>> Schedule: normal
>>> Last Changed Author: davsclaus
>>> Last Changed Rev: 1002821
>>> Last Changed Date: 2010-09-29 21:31:23 +0200 (Wed, 29 Sep 2010)
>>>
>>>
>>> On Wed, Sep 29, 2010 at 11:55 PM, Lorrin Nelson
>>> <lh...@nerdylorrin.net> wrote:
>>>> I'm pretty sure I was already at revision 1002541 before. Now I definitely am. Same Exception. I think the problem is the lack of leading /.
>>>>
>>>> SftpOperations.retrieveFileToStreamInBody computes path as "tmp/mult", calls changeCurrentDirectory(path), which in turn calls channel.cd(path), which (now we're in com.jcraft.jsch.ChannelSftp) calls path=remoteAbsolutePath(path); and computes path: "/home/test/tmp/mult", which does not exist.
>>>>
>>>> SftpConsumer 2010-09-29 14:51:50,487 -- ERROR -- Caused by: [org.apache.camel.component.file.GenericFileOperationFailedException - Cannot change current directory to: tmp/mult]
>>>> org.apache.camel.component.file.GenericFileOperationFailedException: Cannot change current directory to: tmp/mult
>>>> at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:352)
>>>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:408)
>>>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:387)
>>>> at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299)
>>>> at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
>>>> at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
>>>> at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
>>>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>>>> at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>> at java.lang.Thread.run(Thread.java:637)
>>>> Caused by: 2: No such file
>>>> at com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2289)
>>>> at com.jcraft.jsch.ChannelSftp._realpath(ChannelSftp.java:1822)
>>>> at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:268)
>>>> at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:350)
>>>> ... 15 more
>>>>
>>>>
>>>> Cheers!
>>>> -Lorrin
>>>>
>>>> On Sep 29, 2010, at 12:32 PM, Claus Ibsen wrote:
>>>>
>>>>> Hi
>>>>>
>>>>> Can you try again with latest source code from trunk. I have changed
>>>>> the logic to change dir one folder at a time.
>>>>>
>>>>>
>>>>> On Wed, Sep 29, 2010 at 7:47 PM, Lorrin Nelson
>>>>> <lh...@nerdylorrin.net> wrote:
>>>>>> Thanks for the quick response. I'm seeing a different failure now ("Cannot change current directory to: tmp/mult" rather than "Cannot retrieve file: tmp/mult/dummy-5.log"). Is it suspicious that in either case there is no leading /?
>>>>>>
>>>>>> I tried uris
>>>>>> sftp://test@host//tmp/mult?include=.*\.log
>>>>>> and
>>>>>> sftp://test@host///tmp/mult?include=.*\.log
>>>>>>
>>>>>> SftpConsumer 2010-09-29 10:31:48,401 -- ERROR -- Caused by: [org.apache.camel.component.file.GenericFileOperationFailedException - Cannot change current directory to: tmp/mult]
>>>>>> org.apache.camel.component.file.GenericFileOperationFailedException: Cannot change current directory to: tmp/mult
>>>>>> at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:352)
>>>>>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:408)
>>>>>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:387)
>>>>>> at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299)
>>>>>> at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
>>>>>> at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
>>>>>> at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
>>>>>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>>>>>> at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>>>>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
>>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
>>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>>>> at java.lang.Thread.run(Thread.java:637)
>>>>>> Caused by: 2: No such file
>>>>>> at com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2289)
>>>>>> at com.jcraft.jsch.ChannelSftp._realpath(ChannelSftp.java:1822)
>>>>>> at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:268)
>>>>>> at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:350)
>>>>>> ... 15 more
>>>>>>
>>>>>> On a tangentially related note, there's some file name processing during the move step that throws a NullPointerException when the URI contains a trailing slash:
>>>>>>
>>>>>> sftp://test@host/tmp/mult/?include=.*\.log
>>>>>>
>>>>>> The problem is that GenericFile.changeFileName calls
>>>>>> newFileName = ObjectHelper.after(newFileName, endpointPath + getFileSeparator());
>>>>>> ...without first checking if endPointPath contains a trailing separator.
>>>>>>
>>>>>> E.g.
>>>>>> newFileName: "tmp/mult/.done/dummy-2.log"
>>>>>> endPointPath: "tmp/mult/"
>>>>>>
>>>>>> ObjectHelper.after():
>>>>>> text: "tmp/mult/.done/dummy-2.log"
>>>>>> after: "tmp/mult//"
>>>>>>
>>>>>> text doesn't contain after (because of double //), returns null, changeFileName() doesn't expect null, throws NPE.
>>>>>>
>>>>>> GenericFileOnCompletion 2010-09-29 10:37:37,888 -- ERROR -- Caused by: [java.lang.NullPointerException - null]
>>>>>> java.lang.NullPointerException
>>>>>> at java.io.File.<init>(File.java:222)
>>>>>> at org.apache.camel.component.file.GenericFile.changeFileName(GenericFile.java:169)
>>>>>> at org.apache.camel.component.file.strategy.GenericFileExpressionRenamer.renameFile(GenericFileExpressionRenamer.java:41)
>>>>>> at org.apache.camel.component.file.strategy.GenericFileRenameProcessStrategy.commit(GenericFileRenameProcessStrategy.java:82)
>>>>>> at org.apache.camel.component.file.GenericFileOnCompletion.processStrategyCommit(GenericFileOnCompletion.java:121)
>>>>>> at org.apache.camel.component.file.GenericFileOnCompletion.onCompletion(GenericFileOnCompletion.java:83)
>>>>>> at org.apache.camel.component.file.GenericFileOnCompletion.onComplete(GenericFileOnCompletion.java:52)
>>>>>> at org.apache.camel.util.UnitOfWorkHelper.doneSynchronizations(UnitOfWorkHelper.java:55)
>>>>>> at org.apache.camel.impl.DefaultUnitOfWork.done(DefaultUnitOfWork.java:173)
>>>>>> at org.apache.camel.processor.UnitOfWorkProcessor.doneUow(UnitOfWorkProcessor.java:121)
>>>>>> at org.apache.camel.processor.UnitOfWorkProcessor.access$000(UnitOfWorkProcessor.java:36)
>>>>>> at org.apache.camel.processor.UnitOfWorkProcessor$1.done(UnitOfWorkProcessor.java:106)
>>>>>> at org.apache.camel.processor.Pipeline.process(Pipeline.java:130)
>>>>>> at org.apache.camel.processor.UnitOfWorkProcessor.process(UnitOfWorkProcessor.java:99)
>>>>>> at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:70)
>>>>>> at org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:98)
>>>>>> at org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:89)
>>>>>> at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:68)
>>>>>> at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:322)
>>>>>> at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
>>>>>> at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
>>>>>> at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
>>>>>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>>>>>> at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>>>>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
>>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
>>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>>>> at java.lang.Thread.run(Thread.java:637)
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Sep 29, 2010, at 2:46 AM, Claus Ibsen wrote:
>>>>>>
>>>>>>> Hi
>>>>>>>
>>>>>>> I have committed a fix to the SFTP component. Can you try with latest
>>>>>>> source code from trunk?
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Sep 28, 2010 at 7:36 PM, Lorrin <lh...@nerdylorrin.net> wrote:
>>>>>>>>
>>>>>>>> I just bumped into the same problem. I'm running 2.5-SNAPSHOT.
>>>>>>>>
>>>>>>>> SftpConsumer 2010-09-28 10:31:28,094 -- ERROR -- Caused by:
>>>>>>>> [org.apache.camel.component.file.GenericFileOperationFailedException -
>>>>>>>> Cannot retrieve file: tmp/mult/dummy-5.log]
>>>>>>>> org.apache.camel.component.file.GenericFileOperationFailedException: Cannot
>>>>>>>> retrieve file: tmp/mult/dummy-5.log
>>>>>>>> at
>>>>>>>> org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:403)
>>>>>>>> at
>>>>>>>> org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:387)
>>>>>>>> at
>>>>>>>> org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299)
>>>>>>>> at
>>>>>>>> org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
>>>>>>>> at
>>>>>>>> org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
>>>>>>>> at
>>>>>>>> org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
>>>>>>>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>>>>>>>> at
>>>>>>>> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>>>>>>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>>>>>>>> at
>>>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>>>>>>>> at
>>>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
>>>>>>>> at
>>>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
>>>>>>>> at
>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>>>>>> at
>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>>>>>> at java.lang.Thread.run(Thread.java:637)
>>>>>>>> Caused by: 2: No such file
>>>>>>>> at com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2289)
>>>>>>>> at com.jcraft.jsch.ChannelSftp._get(ChannelSftp.java:901)
>>>>>>>> at com.jcraft.jsch.ChannelSftp.get(ChannelSftp.java:870)
>>>>>>>> at com.jcraft.jsch.ChannelSftp.get(ChannelSftp.java:849)
>>>>>>>> at
>>>>>>>> org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:400)
>>>>>>>> ... 14 more
>>>>>>>>
>>>>>>>> --
>>>>>>>> View this message in context: http://camel.465427.n5.nabble.com/How-to-change-directory-while-using-sftp-component-tp2806817p2857187.html
>>>>>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Claus Ibsen
>>>>>>> Apache Camel Committer
>>>>>>>
>>>>>>> Author of Camel in Action: http://www.manning.com/ibsen/
>>>>>>> Open Source Integration: http://fusesource.com
>>>>>>> Blog: http://davsclaus.blogspot.com/
>>>>>>> Twitter: http://twitter.com/davsclaus
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Claus Ibsen
>>>>> Apache Camel Committer
>>>>>
>>>>> Author of Camel in Action: http://www.manning.com/ibsen/
>>>>> Open Source Integration: http://fusesource.com
>>>>> Blog: http://davsclaus.blogspot.com/
>>>>> Twitter: http://twitter.com/davsclaus
>>>>>
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Claus Ibsen
>>> Apache Camel Committer
>>>
>>> Author of Camel in Action: http://www.manning.com/ibsen/
>>> Open Source Integration: http://fusesource.com
>>> Blog: http://davsclaus.blogspot.com/
>>> Twitter: http://twitter.com/davsclaus
>>>
>>
>>
>
>
>
> --
> Claus Ibsen
> Apache Camel Committer
>
> Author of Camel in Action: http://www.manning.com/ibsen/
> Open Source Integration: http://fusesource.com
> Blog: http://davsclaus.blogspot.com/
> Twitter: http://twitter.com/davsclaus
>
Re: How to change directory while using sftp component
Posted by Claus Ibsen <cl...@gmail.com>.
Hi Loririn
I think you/we have to step back. We have other users in the past
running with SFTP with no problems at all. Also fetching files
recursively.
So I wonder if we should revert back and keep the old logic!
1)
Is this the first time you use camel-ftp?
2)
What OS and FTP server are you using?
3)
Can you use regular FTP instead?
4)
Use relative urls over absolute. They where the first supported.
5)
Can you enable TRACE logging on org.apache.camel.component.file and
setup a simple example so it makes it easier to follow and understand
what its supposed to do, and what happens in reality so we can see and
understand the difference.
On Fri, Oct 1, 2010 at 10:45 PM, Lorrin Nelson
<lh...@nerdylorrin.net> wrote:
> Hi Claus. Ack, sorry about that. I'm pulling through github rather than SVN directly and then running via a Maven project pointing at my locally built Camel, so it's sometimes confusing how up-to-date I am.
>
> So I think I have current code, including checkin "CAMEL-3174: Changing dir with ftp must do one dir at a time." and I think SFTP polling is now broken for both absolute and relative URLs. See debug walk-throughs and stack traces below.
>
> Absolute URI with double leading /:
> sftp://user@host//home/user/subdir?idempotent=true&idempotentRepository=#myCustomIR&include=.*.log&noop=true&password=******
>
> SftpOperations.changeCurrentDirectory("home/user/subdir") //note there's no leading slash on the supposedly absolute path!
> //splits into chunks successfully, starts looping through chunks
> doChangeDirectory("home")
> ChannelSftp.cd("home")
> //path=remoteAbsolutePath(path);
> //yields "/home/user/home". Doh!
>
>
> Relative URI with single leading /:
> sftp://user@host/subdir?idempotent=true&idempotentRepository=#myCustomIR&include=.*.log&noop=true&password=******
> SftpOperations.changeCurrentDirectory("subdir")
> //splits into single chunk, starts looping through chunks
> doChangeDirectory("subdir")
> ChannelSftp.cd("home")
> //path=remoteAbsolutePath(path);
> //yields "/home/user/subdir". yay!
>
> ..but then, back in SftpOperations.retrieveFileToStreamInBody, it hits the "change back to current directory code"
> changeCurrentDirectory("/home/tomcat"); //now we have a leading slash, unlike before
> //splits info chunks "", "home", and "tomcat". Doh!
> doChangeDirectory("")
> ChannelSftp.cd("")
> //path=remoteAbsolutePath(path) throws a StringIndexOutOfBoundsException when it receives empty string.
>
>
> Stack trace when attempting to supply absolute path:
> org.apache.camel.component.file.GenericFileOperationFailedException: Cannot change directory to: home
> at org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:372)
> at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:361)
> at org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:428)
> at org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:407)
> at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299)
> at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
> at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
> at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
> at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:637)
> Caused by: 2: No such file
> at com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2289)
> at com.jcraft.jsch.ChannelSftp._realpath(ChannelSftp.java:1822)
> at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:268)
> at org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:370)
> ... 16 more
>
>
> Stack trace when relative URL tries to restore current working directory:
> SftpConsumer 2010-10-01 13:41:58,529 -- ERROR -- Caused by: [org.apache.camel.component.file.GenericFileOperationFailedException - Cannot change directory to: ]
> org.apache.camel.component.file.GenericFileOperationFailedException: Cannot change directory to:
> at org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:372)
> at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:361)
> at org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:435)
> at org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:407)
> at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299)
> at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
> at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
> at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
> at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:637)
> Caused by: 4:
> at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:285)
> at org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:370)
> ... 16 more
> Caused by: java.lang.StringIndexOutOfBoundsException: String index out of range: 0
> at java.lang.String.charAt(String.java:686)
> at com.jcraft.jsch.ChannelSftp.remoteAbsolutePath(ChannelSftp.java:2359)
> at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:264)
> ... 17 more
>
> Cheers
> -Lorrin
>
> On Sep 29, 2010, at 9:37 PM, Claus Ibsen wrote:
>
>> Hi
>>
>> You have not used the latest source code. The stacktrace points to
>> empty lines etc.
>> Make sure you build camel-core and camel-ftp using latest source code.
>>
>> Revision: 1002946
>> Node Kind: directory
>> Schedule: normal
>> Last Changed Author: davsclaus
>> Last Changed Rev: 1002821
>> Last Changed Date: 2010-09-29 21:31:23 +0200 (Wed, 29 Sep 2010)
>>
>>
>> On Wed, Sep 29, 2010 at 11:55 PM, Lorrin Nelson
>> <lh...@nerdylorrin.net> wrote:
>>> I'm pretty sure I was already at revision 1002541 before. Now I definitely am. Same Exception. I think the problem is the lack of leading /.
>>>
>>> SftpOperations.retrieveFileToStreamInBody computes path as "tmp/mult", calls changeCurrentDirectory(path), which in turn calls channel.cd(path), which (now we're in com.jcraft.jsch.ChannelSftp) calls path=remoteAbsolutePath(path); and computes path: "/home/test/tmp/mult", which does not exist.
>>>
>>> SftpConsumer 2010-09-29 14:51:50,487 -- ERROR -- Caused by: [org.apache.camel.component.file.GenericFileOperationFailedException - Cannot change current directory to: tmp/mult]
>>> org.apache.camel.component.file.GenericFileOperationFailedException: Cannot change current directory to: tmp/mult
>>> at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:352)
>>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:408)
>>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:387)
>>> at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299)
>>> at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
>>> at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
>>> at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
>>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>>> at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>> at java.lang.Thread.run(Thread.java:637)
>>> Caused by: 2: No such file
>>> at com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2289)
>>> at com.jcraft.jsch.ChannelSftp._realpath(ChannelSftp.java:1822)
>>> at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:268)
>>> at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:350)
>>> ... 15 more
>>>
>>>
>>> Cheers!
>>> -Lorrin
>>>
>>> On Sep 29, 2010, at 12:32 PM, Claus Ibsen wrote:
>>>
>>>> Hi
>>>>
>>>> Can you try again with latest source code from trunk. I have changed
>>>> the logic to change dir one folder at a time.
>>>>
>>>>
>>>> On Wed, Sep 29, 2010 at 7:47 PM, Lorrin Nelson
>>>> <lh...@nerdylorrin.net> wrote:
>>>>> Thanks for the quick response. I'm seeing a different failure now ("Cannot change current directory to: tmp/mult" rather than "Cannot retrieve file: tmp/mult/dummy-5.log"). Is it suspicious that in either case there is no leading /?
>>>>>
>>>>> I tried uris
>>>>> sftp://test@host//tmp/mult?include=.*\.log
>>>>> and
>>>>> sftp://test@host///tmp/mult?include=.*\.log
>>>>>
>>>>> SftpConsumer 2010-09-29 10:31:48,401 -- ERROR -- Caused by: [org.apache.camel.component.file.GenericFileOperationFailedException - Cannot change current directory to: tmp/mult]
>>>>> org.apache.camel.component.file.GenericFileOperationFailedException: Cannot change current directory to: tmp/mult
>>>>> at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:352)
>>>>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:408)
>>>>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:387)
>>>>> at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299)
>>>>> at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
>>>>> at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
>>>>> at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
>>>>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>>>>> at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>>>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>>> at java.lang.Thread.run(Thread.java:637)
>>>>> Caused by: 2: No such file
>>>>> at com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2289)
>>>>> at com.jcraft.jsch.ChannelSftp._realpath(ChannelSftp.java:1822)
>>>>> at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:268)
>>>>> at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:350)
>>>>> ... 15 more
>>>>>
>>>>> On a tangentially related note, there's some file name processing during the move step that throws a NullPointerException when the URI contains a trailing slash:
>>>>>
>>>>> sftp://test@host/tmp/mult/?include=.*\.log
>>>>>
>>>>> The problem is that GenericFile.changeFileName calls
>>>>> newFileName = ObjectHelper.after(newFileName, endpointPath + getFileSeparator());
>>>>> ...without first checking if endPointPath contains a trailing separator.
>>>>>
>>>>> E.g.
>>>>> newFileName: "tmp/mult/.done/dummy-2.log"
>>>>> endPointPath: "tmp/mult/"
>>>>>
>>>>> ObjectHelper.after():
>>>>> text: "tmp/mult/.done/dummy-2.log"
>>>>> after: "tmp/mult//"
>>>>>
>>>>> text doesn't contain after (because of double //), returns null, changeFileName() doesn't expect null, throws NPE.
>>>>>
>>>>> GenericFileOnCompletion 2010-09-29 10:37:37,888 -- ERROR -- Caused by: [java.lang.NullPointerException - null]
>>>>> java.lang.NullPointerException
>>>>> at java.io.File.<init>(File.java:222)
>>>>> at org.apache.camel.component.file.GenericFile.changeFileName(GenericFile.java:169)
>>>>> at org.apache.camel.component.file.strategy.GenericFileExpressionRenamer.renameFile(GenericFileExpressionRenamer.java:41)
>>>>> at org.apache.camel.component.file.strategy.GenericFileRenameProcessStrategy.commit(GenericFileRenameProcessStrategy.java:82)
>>>>> at org.apache.camel.component.file.GenericFileOnCompletion.processStrategyCommit(GenericFileOnCompletion.java:121)
>>>>> at org.apache.camel.component.file.GenericFileOnCompletion.onCompletion(GenericFileOnCompletion.java:83)
>>>>> at org.apache.camel.component.file.GenericFileOnCompletion.onComplete(GenericFileOnCompletion.java:52)
>>>>> at org.apache.camel.util.UnitOfWorkHelper.doneSynchronizations(UnitOfWorkHelper.java:55)
>>>>> at org.apache.camel.impl.DefaultUnitOfWork.done(DefaultUnitOfWork.java:173)
>>>>> at org.apache.camel.processor.UnitOfWorkProcessor.doneUow(UnitOfWorkProcessor.java:121)
>>>>> at org.apache.camel.processor.UnitOfWorkProcessor.access$000(UnitOfWorkProcessor.java:36)
>>>>> at org.apache.camel.processor.UnitOfWorkProcessor$1.done(UnitOfWorkProcessor.java:106)
>>>>> at org.apache.camel.processor.Pipeline.process(Pipeline.java:130)
>>>>> at org.apache.camel.processor.UnitOfWorkProcessor.process(UnitOfWorkProcessor.java:99)
>>>>> at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:70)
>>>>> at org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:98)
>>>>> at org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:89)
>>>>> at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:68)
>>>>> at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:322)
>>>>> at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
>>>>> at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
>>>>> at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
>>>>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>>>>> at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>>>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>>> at java.lang.Thread.run(Thread.java:637)
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Sep 29, 2010, at 2:46 AM, Claus Ibsen wrote:
>>>>>
>>>>>> Hi
>>>>>>
>>>>>> I have committed a fix to the SFTP component. Can you try with latest
>>>>>> source code from trunk?
>>>>>>
>>>>>>
>>>>>> On Tue, Sep 28, 2010 at 7:36 PM, Lorrin <lh...@nerdylorrin.net> wrote:
>>>>>>>
>>>>>>> I just bumped into the same problem. I'm running 2.5-SNAPSHOT.
>>>>>>>
>>>>>>> SftpConsumer 2010-09-28 10:31:28,094 -- ERROR -- Caused by:
>>>>>>> [org.apache.camel.component.file.GenericFileOperationFailedException -
>>>>>>> Cannot retrieve file: tmp/mult/dummy-5.log]
>>>>>>> org.apache.camel.component.file.GenericFileOperationFailedException: Cannot
>>>>>>> retrieve file: tmp/mult/dummy-5.log
>>>>>>> at
>>>>>>> org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:403)
>>>>>>> at
>>>>>>> org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:387)
>>>>>>> at
>>>>>>> org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299)
>>>>>>> at
>>>>>>> org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
>>>>>>> at
>>>>>>> org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
>>>>>>> at
>>>>>>> org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
>>>>>>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>>>>>>> at
>>>>>>> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>>>>>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>>>>>>> at
>>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>>>>>>> at
>>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
>>>>>>> at
>>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
>>>>>>> at
>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>>>>> at
>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>>>>> at java.lang.Thread.run(Thread.java:637)
>>>>>>> Caused by: 2: No such file
>>>>>>> at com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2289)
>>>>>>> at com.jcraft.jsch.ChannelSftp._get(ChannelSftp.java:901)
>>>>>>> at com.jcraft.jsch.ChannelSftp.get(ChannelSftp.java:870)
>>>>>>> at com.jcraft.jsch.ChannelSftp.get(ChannelSftp.java:849)
>>>>>>> at
>>>>>>> org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:400)
>>>>>>> ... 14 more
>>>>>>>
>>>>>>> --
>>>>>>> View this message in context: http://camel.465427.n5.nabble.com/How-to-change-directory-while-using-sftp-component-tp2806817p2857187.html
>>>>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Claus Ibsen
>>>>>> Apache Camel Committer
>>>>>>
>>>>>> Author of Camel in Action: http://www.manning.com/ibsen/
>>>>>> Open Source Integration: http://fusesource.com
>>>>>> Blog: http://davsclaus.blogspot.com/
>>>>>> Twitter: http://twitter.com/davsclaus
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Claus Ibsen
>>>> Apache Camel Committer
>>>>
>>>> Author of Camel in Action: http://www.manning.com/ibsen/
>>>> Open Source Integration: http://fusesource.com
>>>> Blog: http://davsclaus.blogspot.com/
>>>> Twitter: http://twitter.com/davsclaus
>>>>
>>>
>>>
>>
>>
>>
>> --
>> Claus Ibsen
>> Apache Camel Committer
>>
>> Author of Camel in Action: http://www.manning.com/ibsen/
>> Open Source Integration: http://fusesource.com
>> Blog: http://davsclaus.blogspot.com/
>> Twitter: http://twitter.com/davsclaus
>>
>
>
--
Claus Ibsen
Apache Camel Committer
Author of Camel in Action: http://www.manning.com/ibsen/
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus