You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by codemickey <co...@gmail.com> on 2010/12/14 01:59:15 UTC
SFTP operation raises an exception, but the file is transferred
I am a new Camel user.
I am trying to SFTP a file over to a local directory. I am using the
following code for the same:
public static void main(String args[]) throws Exception {
// create CamelContext
CamelContext context = new DefaultCamelContext();
// add our route to the CamelContext
context.addRoutes(new RouteBuilder() {
@Override
public void configure() {
from("sftp://<account>@<sftp
server>/<dir>?password=<passwd>").to("file:data/outbox");
}
});
// start the route and let it do its work
context.start();
Thread.sleep(50000);
// stop the CamelContext
context.stop();
}
When i check the output folder, I see that the file is downloaded. But when
the content is stopped, I see the following exception:
SEVERE: Caused by:
[org.apache.camel.component.file.GenericFileOperationFailedException -
Cannot retrieve file: data/input/Set1.txt]
org.apache.camel.component.file.GenericFileOperationFailedException: Cannot
retrieve file: data/input/Set1.txt
at
org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:482)
at
org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:448)
at
org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299)
at
org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
at
org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
at
org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at
java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.io.IOException: inputstream is closed
at com.jcraft.jsch.ChannelSftp.fill(ChannelSftp.java:2325)
at com.jcraft.jsch.ChannelSftp.header(ChannelSftp.java:2349)
at com.jcraft.jsch.ChannelSftp.access$800(ChannelSftp.java:36)
at com.jcraft.jsch.ChannelSftp$2.read(ChannelSftp.java:1092)
at com.jcraft.jsch.ChannelSftp$2.read(ChannelSftp.java:1052)
at org.apache.camel.util.IOHelper.copy(IOHelper.java:104)
at org.apache.camel.util.IOHelper.copy(IOHelper.java:86)
at org.apache.camel.util.IOHelper.copyAndCloseInput(IOHelper.java:111)
at
org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:475)
... 14 more
Also one more thing observed is, the for smaller files in a few 100 KBs ..
the file is transferred and i see no exception. But for file in MBs, even if
the file is transferred .. I see the above exception.
May I get some help on why am I seeing such an exception. Is there some URI
option which I need to add to disconnect/not look for the file once the
transfer is complete?
Thanks in advance!
--
View this message in context: http://camel.465427.n5.nabble.com/SFTP-operation-raises-an-exception-but-the-file-is-transferred-tp3303905p3303905.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: SFTP operation raises an exception, but the file is transferred
Posted by codemickey <co...@gmail.com>.
My understanding of what is going on behind the scenes is:
It downloads the file. When the context.stop is executed, it tries to to do
graceful shutdown, but I get the following message:
INFO: Route: route1 suspended and shutdown deferred, was consuming from:
Endpoint[sftp://username@server/data/inbox?password=******]
The graceful shutdown procedure tries disconnecting from the SFTP server and
is successful:
INFO: JSCH -> Disconnecting from <server> port 22
But the route still continues to try reading from the server, for which the
input stream is closed now as its disconnected.
I believe it has to do more with the working of camel than with the Jcraft
sftp operations. Will have to dig deeper in the route execution process I
think!
In the meanwhile, i have a workaround to this problem. I try downloading the
file from the SFTP server using the 'localWorkDirectory' URI option. Here is
how I specify it:
from("sftp://<account>@<sftp
server>/<dir>?password=<passwd>&localWorkDirectory=/tmp").to("file:data/outbox");
Note: The directory you specify for localWorkDirectory needs to be from your
machine's root, else you have to deal with file permissions when you specify
the relative path .. atleast I had to!
Thanks,
Mansi Kanani
--
View this message in context: http://camel.465427.n5.nabble.com/SFTP-operation-raises-an-exception-but-the-file-is-transferred-tp3303905p3305515.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: SFTP operation raises an exception, but the file is transferred
Posted by Claus Ibsen <cl...@gmail.com>.
On Tue, Dec 14, 2010 at 7:18 PM, codemickey <co...@gmail.com> wrote:
>
> I did search for this issue but had no luck finding something related to it.
> I will try asking this at the JCraft mailing list!
>
> Thanks Claus! I will try and post a solution once I find it.
>
For the FTP component I thought of using some listener, which has
callbacks on the progress of the download.
Then with the listener we may be able to know the file has been fully
transfered, and then ignore that "mysterious" exception you see.
I dont know if SFTP has the same listener feature as the FTP component has.
> Thanks,
> Mansi Kanani
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/SFTP-operation-raises-an-exception-but-the-file-is-transferred-tp3303905p3305024.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
--
Claus Ibsen
-----------------
FuseSource
Email: cibsen@fusesource.com
Web: http://fusesource.com
Twitter: davsclaus
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/
Re: SFTP operation raises an exception, but the file is transferred
Posted by codemickey <co...@gmail.com>.
I did search for this issue but had no luck finding something related to it.
I will try asking this at the JCraft mailing list!
Thanks Claus! I will try and post a solution once I find it.
Thanks,
Mansi Kanani
--
View this message in context: http://camel.465427.n5.nabble.com/SFTP-operation-raises-an-exception-but-the-file-is-transferred-tp3303905p3305024.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: SFTP operation raises an exception, but the file is transferred
Posted by Claus Ibsen <cl...@gmail.com>.
Hi
I have been told by another Camel user they had something similar, but
I think it was the regular FTP component.
The file was downloaded but the FTP library threw an exception. Their
problem was their network topology which wasn't to well setup.
Try googling and asking at the JCraft mailing list as its their
library Camel uses in the SFTP component.
On Tue, Dec 14, 2010 at 1:59 AM, codemickey <co...@gmail.com> wrote:
>
> I am a new Camel user.
>
> I am trying to SFTP a file over to a local directory. I am using the
> following code for the same:
>
> public static void main(String args[]) throws Exception {
> // create CamelContext
> CamelContext context = new DefaultCamelContext();
>
> // add our route to the CamelContext
> context.addRoutes(new RouteBuilder() {
> @Override
> public void configure() {
> from("sftp://<account>@<sftp
> server>/<dir>?password=<passwd>").to("file:data/outbox");
> }
> });
>
> // start the route and let it do its work
> context.start();
> Thread.sleep(50000);
>
> // stop the CamelContext
> context.stop();
> }
>
> When i check the output folder, I see that the file is downloaded. But when
> the content is stopped, I see the following exception:
>
> SEVERE: Caused by:
> [org.apache.camel.component.file.GenericFileOperationFailedException -
> Cannot retrieve file: data/input/Set1.txt]
> org.apache.camel.component.file.GenericFileOperationFailedException: Cannot
> retrieve file: data/input/Set1.txt
> at
> org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:482)
> at
> org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:448)
> at
> org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299)
> at
> org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
> at
> org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
> at
> org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
> at
> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: java.io.IOException: inputstream is closed
> at com.jcraft.jsch.ChannelSftp.fill(ChannelSftp.java:2325)
> at com.jcraft.jsch.ChannelSftp.header(ChannelSftp.java:2349)
> at com.jcraft.jsch.ChannelSftp.access$800(ChannelSftp.java:36)
> at com.jcraft.jsch.ChannelSftp$2.read(ChannelSftp.java:1092)
> at com.jcraft.jsch.ChannelSftp$2.read(ChannelSftp.java:1052)
> at org.apache.camel.util.IOHelper.copy(IOHelper.java:104)
> at org.apache.camel.util.IOHelper.copy(IOHelper.java:86)
> at org.apache.camel.util.IOHelper.copyAndCloseInput(IOHelper.java:111)
> at
> org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:475)
> ... 14 more
>
>
>
> Also one more thing observed is, the for smaller files in a few 100 KBs ..
> the file is transferred and i see no exception. But for file in MBs, even if
> the file is transferred .. I see the above exception.
>
> May I get some help on why am I seeing such an exception. Is there some URI
> option which I need to add to disconnect/not look for the file once the
> transfer is complete?
>
>
> Thanks in advance!
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/SFTP-operation-raises-an-exception-but-the-file-is-transferred-tp3303905p3303905.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
--
Claus Ibsen
-----------------
FuseSource
Email: cibsen@fusesource.com
Web: http://fusesource.com
Twitter: davsclaus
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/