You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by ulsa <ul...@jayway.com> on 2015/03/16 14:07:45 UTC
Slow FTP
One of the files on my FTP server is ~200MB. I'm using Camel FTP from an EC2
instance (Ubuntu 14), and I get a connection reset error on that file after
25 minutes. However, when using command line ftp on the same machine, the
same file takes 38 seconds to download. I have set option
localWorkDirectory=/var/tmp/camel. Is there any other configuration trick
that I have missed?
--
View this message in context: http://camel.465427.n5.nabble.com/Slow-FTP-tp5764223.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Slow FTP
Posted by yogu13 <yo...@gmail.com>.
perhaps advising to set buffersize in the camel documentation for ftp can
help get around it for time being.
Regards,
-Yogesh
--
View this message in context: http://camel.465427.n5.nabble.com/Slow-FTP-tp5764223p5764406.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Slow FTP
Posted by Claus Ibsen <cl...@gmail.com>.
Their last release is almost 2 years old.
Users of Camel cannot wait another 2 years or so before they decide to
do a release.
On Thu, Mar 19, 2015 at 7:55 AM, yogu13 <yo...@gmail.com> wrote:
> Hello Claus,
>
> Not sure if camel needs to take that up as I think its already tracked by
> the Apache Commons Net team NET-550
> <https://issues.apache.org/jira/browse/NET-550> and this change would be
> available as part of their 3.4 release. Upgrading to 3.4 once release should
> do the trick for camel.
>
> Regards,
> -Yogesh
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Slow-FTP-tp5764223p5764401.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
--
Claus Ibsen
-----------------
Red Hat, Inc.
Email: cibsen@redhat.com
Twitter: davsclaus
Blog: http://davsclaus.com
Author of Camel in Action: http://www.manning.com/ibsen
hawtio: http://hawt.io/
fabric8: http://fabric8.io/
Re: Slow FTP
Posted by yogu13 <yo...@gmail.com>.
Hello Claus,
Not sure if camel needs to take that up as I think its already tracked by
the Apache Commons Net team NET-550
<https://issues.apache.org/jira/browse/NET-550> and this change would be
available as part of their 3.4 release. Upgrading to 3.4 once release should
do the trick for camel.
Regards,
-Yogesh
--
View this message in context: http://camel.465427.n5.nabble.com/Slow-FTP-tp5764223p5764401.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Slow FTP
Posted by Claus Ibsen <cl...@gmail.com>.
Hi
I logged a ticket about this
https://issues.apache.org/jira/browse/CAMEL-8513
On Tue, Mar 17, 2015 at 11:08 AM, Claus Ibsen <cl...@gmail.com> wrote:
> Hi
>
> Ah I dont think we set the endpoint buffersize on the ftp client by
> default. And that is also by default for writing. Currently the
> FTPClient is configured with the timeout options, but not the buffer
> size.
>
> That may be a good idea as it really seem that its internal buffer
> size is too low out of the box.
>
>
>
> On Tue, Mar 17, 2015 at 10:40 AM, ulsa <ul...@jayway.com> wrote:
>> Are you saying that the file bufferSize option is the same as
>> ftpClient.bufferSize?
>>
>>
>>
>> --
>> View this message in context: http://camel.465427.n5.nabble.com/Slow-FTP-tp5764223p5764263.html
>> Sent from the Camel - Users mailing list archive at Nabble.com.
>
>
>
> --
> Claus Ibsen
> -----------------
> Red Hat, Inc.
> Email: cibsen@redhat.com
> Twitter: davsclaus
> Blog: http://davsclaus.com
> Author of Camel in Action: http://www.manning.com/ibsen
> hawtio: http://hawt.io/
> fabric8: http://fabric8.io/
--
Claus Ibsen
-----------------
Red Hat, Inc.
Email: cibsen@redhat.com
Twitter: davsclaus
Blog: http://davsclaus.com
Author of Camel in Action: http://www.manning.com/ibsen
hawtio: http://hawt.io/
fabric8: http://fabric8.io/
Re: Slow FTP
Posted by Claus Ibsen <cl...@gmail.com>.
Hi
Ah I dont think we set the endpoint buffersize on the ftp client by
default. And that is also by default for writing. Currently the
FTPClient is configured with the timeout options, but not the buffer
size.
That may be a good idea as it really seem that its internal buffer
size is too low out of the box.
On Tue, Mar 17, 2015 at 10:40 AM, ulsa <ul...@jayway.com> wrote:
> Are you saying that the file bufferSize option is the same as
> ftpClient.bufferSize?
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Slow-FTP-tp5764223p5764263.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
--
Claus Ibsen
-----------------
Red Hat, Inc.
Email: cibsen@redhat.com
Twitter: davsclaus
Blog: http://davsclaus.com
Author of Camel in Action: http://www.manning.com/ibsen
hawtio: http://hawt.io/
fabric8: http://fabric8.io/
Re: Slow FTP
Posted by ulsa <ul...@jayway.com>.
Are you saying that the file bufferSize option is the same as
ftpClient.bufferSize?
--
View this message in context: http://camel.465427.n5.nabble.com/Slow-FTP-tp5764223p5764263.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Slow FTP
Posted by ulsa <ul...@jayway.com>.
I was indeed looking at ftp.html, but the links on ftp2.html are identical,
and I can't reach them:
http://commons.apache.org/proper/commons-net/api-2.2/org/apache/commons/net/ftp/FTPClientConfig.html
http://commons.apache.org/proper/commons-net/api-2.2/org/apache/commons/net/ftp/FTPClient.html
--
View this message in context: http://camel.465427.n5.nabble.com/Slow-FTP-tp5764223p5764262.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Slow FTP
Posted by Claus Ibsen <cl...@gmail.com>.
Hi
The write buffer is 128kb
http://camel.apache.org/file2
The links works fine for me ftp2 -> file2
Maybe you look at the old Camel 1.x docs which unfortunately may still
be there (cwiki is not ideal for project docs in 2015)
On Tue, Mar 17, 2015 at 9:31 AM, ulsa <ul...@jayway.com> wrote:
> Yes, setting a bigger buffer definitely did the trick. Like 5 secs to
> download instead of 25 mins. The default buffer size must be very small.
>
> It wasn't obvious (to me) from the Camel FTP page that it's possible to set
> other config params on the ftpClient than the ones listed under URI Options.
> There are two sentences:
>
>
>
> Both links are broken though.
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Slow-FTP-tp5764223p5764258.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
--
Claus Ibsen
-----------------
Red Hat, Inc.
Email: cibsen@redhat.com
Twitter: davsclaus
Blog: http://davsclaus.com
Author of Camel in Action: http://www.manning.com/ibsen
hawtio: http://hawt.io/
fabric8: http://fabric8.io/
Re: Slow FTP
Posted by ulsa <ul...@jayway.com>.
Yes, setting a bigger buffer definitely did the trick. Like 5 secs to
download instead of 25 mins. The default buffer size must be very small.
It wasn't obvious (to me) from the Camel FTP page that it's possible to set
other config params on the ftpClient than the ones listed under URI Options.
There are two sentences:
Both links are broken though.
--
View this message in context: http://camel.465427.n5.nabble.com/Slow-FTP-tp5764223p5764258.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Slow FTP
Posted by ulsa <ul...@jayway.com>.
Thanks for pointing me to the client config. I'll try a bigger buffer and
check out what other options I can tweak.
--
View this message in context: http://camel.465427.n5.nabble.com/Slow-FTP-tp5764223p5764253.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Slow FTP
Posted by Willem Jiang <wi...@gmail.com>.
Can you try to setup the FtpClient with a big buffer just like this ?
from("ftp://foo@myserver?password=secret&ftpClient.bufferSize=10000000").to("bean:foo");
--
Willem Jiang
Red Hat, Inc.
Web: http://www.redhat.com
Blog: http://willemjiang.blogspot.com (English)
http://jnn.iteye.com (Chinese)
Twitter: willemjiang
Weibo: 姜宁willem
On March 16, 2015 at 9:08:44 PM, ulsa (ulrik.sandberg@jayway.com) wrote:
> One of the files on my FTP server is ~200MB. I'm using Camel FTP from an EC2
> instance (Ubuntu 14), and I get a connection reset error on that file after
> 25 minutes. However, when using command line ftp on the same machine, the
> same file takes 38 seconds to download. I have set option
> localWorkDirectory=/var/tmp/camel. Is there any other configuration trick
> that I have missed?
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Slow-FTP-tp5764223.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
Re: Slow FTP
Posted by yogu13 <yo...@gmail.com>.
okay .. as williem pointed out setting buffer size definitely helps.. in case
you do not want to put a max limit then try setting the
ftpClient.bufferSize=0 (unlimited)
Hope that helps!
Regards,
-Yogesh
--
View this message in context: http://camel.465427.n5.nabble.com/Slow-FTP-tp5764223p5764257.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Slow FTP
Posted by ulsa <ul...@jayway.com>.
Apache Camel 2.15.0 and Java 1.8.0_11.
--
View this message in context: http://camel.465427.n5.nabble.com/Slow-FTP-tp5764223p5764254.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Slow FTP
Posted by yogu13 <yo...@gmail.com>.
Hello,
Which version of Apache Camel is being used along with the JDK version.
Regards,
-Yogesh
--
View this message in context: http://camel.465427.n5.nabble.com/Slow-FTP-tp5764223p5764240.html
Sent from the Camel - Users mailing list archive at Nabble.com.