You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@commons.apache.org by Jörg Schaible <Jo...@Elsag-Solutions.com> on 2007/12/10 14:59:49 UTC
[vfs] FTP extremely slow compared to SFTP
Hi,
can somebody explain the following numbers - especially why FTP is that dead slow compared to SFTP handling a lot of files? I measure a simple copy action of a directory containing some few files:
========================================
FTP
========================================
* Initial Run
- Exporter.............................................. 8252.0 ms
* Single File (25kb)
- Exporter.............................................. 8021.0 ms
* Single File (25000kb)
- Exporter.............................................. 11437.0 ms
* Single File (50000kb)
- Exporter.............................................. 16043.0 ms
* Single Directory (with 10 files of 1kb)
- Exporter.............................................. 47859.0 ms
* Single Directory (with 25 files of 1kb)
- Exporter.............................................. 113964.0 ms
* Single Directory (with 50 files of 1kb)
- Exporter.............................................. 224312.0 ms
========================================
SFTP
========================================
* Initial Run
- Exporter.............................................. 1252.0 ms
* Single File (25kb)
- Exporter.............................................. 921.0 ms
* Single File (25000kb)
- Exporter.............................................. 14952.0 ms
* Single File (50000kb)
- Exporter.............................................. 24885.0 ms
* Single Directory (with 10 files of 1kb)
- Exporter.............................................. 601.0 ms
* Single Directory (with 25 files of 1kb)
- Exporter.............................................. 922.0 ms
* Single Directory (with 50 files of 1kb)
- Exporter.............................................. 1552.0 ms
Any explanation or - even better - suggestions to improve FTP speed?
- Jörg
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org
RE: DBCP ConnectionProperties
Posted by To...@teliasonera.com.
I am not sure if I understand correctly, but isn't the database schema part of the connect URI?
For example: jdbc:oracle:thin:@database.server.name:port:SCHEMA
Torsten
-----Original Message-----
From: Ding, Qin [mailto:QDING@transunion.com]
Sent: 11. december 2007 16:22
To: Jakarta Commons Users List
Subject: DBCP ConnectionProperties
I need to pass to JDBC via DBCP configuration the property of database schema. How do I do this? Thanks.
Qin
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org
DBCP ConnectionProperties
Posted by "Ding, Qin" <QD...@transunion.com>.
I need to pass to JDBC via DBCP configuration the property of database
schema. How do I do this? Thanks.
Qin
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org
Re: [vfs] FTP extremely slow compared to SFTP
Posted by Mark Fortner <ph...@gmail.com>.
Jörg,
Do you have any code snippets for these examples?
Mark
On Dec 10, 2007 5:59 AM, Jörg Schaible <Jo...@elsag-solutions.com>
wrote:
> Hi,
>
> can somebody explain the following numbers - especially why FTP is that
> dead slow compared to SFTP handling a lot of files? I measure a simple copy
> action of a directory containing some few files:
>
> ========================================
> FTP
> ========================================
> * Initial Run
> - Exporter.............................................. 8252.0 ms
> * Single File (25kb)
> - Exporter.............................................. 8021.0 ms
> * Single File (25000kb)
> - Exporter.............................................. 11437.0 ms
> * Single File (50000kb)
> - Exporter.............................................. 16043.0 ms
> * Single Directory (with 10 files of 1kb)
> - Exporter.............................................. 47859.0 ms
> * Single Directory (with 25 files of 1kb)
> - Exporter.............................................. 113964.0 ms
> * Single Directory (with 50 files of 1kb)
> - Exporter.............................................. 224312.0 ms
>
> ========================================
> SFTP
> ========================================
> * Initial Run
> - Exporter.............................................. 1252.0 ms
> * Single File (25kb)
> - Exporter.............................................. 921.0 ms
> * Single File (25000kb)
> - Exporter.............................................. 14952.0 ms
> * Single File (50000kb)
> - Exporter.............................................. 24885.0 ms
> * Single Directory (with 10 files of 1kb)
> - Exporter.............................................. 601.0 ms
> * Single Directory (with 25 files of 1kb)
> - Exporter.............................................. 922.0 ms
> * Single Directory (with 50 files of 1kb)
> - Exporter.............................................. 1552.0 ms
>
>
> Any explanation or - even better - suggestions to improve FTP speed?
>
> - Jörg
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> For additional commands, e-mail: user-help@commons.apache.org
>
>
RE: [vfs] FTP extremely slow compared to SFTP
Posted by Jörg Schaible <Jo...@Elsag-Solutions.com>.
Thorbjørn Ravn Andersen wrote:
> Jörg Schaible skrev den 10-12-2007 14:59:
>> Any explanation or - even better - suggestions to improve FTP speed?
>>
> If you do this the obvious way you treat each file as a
> seperate entity,
> where a new data connection must be opened per file. This alone,
> however, should not be enough to give the times you see.
It seems, if I really want answers I have to debug the VFS code and compare it with the SFTP provider implementation.
- Jörg
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org
Re: [vfs] FTP extremely slow compared to SFTP
Posted by Thorbjørn Ravn Andersen <th...@gmail.com>.
Jörg Schaible skrev den 10-12-2007 14:59:
> Any explanation or - even better - suggestions to improve FTP speed?
>
If you do this the obvious way you treat each file as a seperate entity,
where a new data connection must be opened per file. This alone,
however, should not be enough to give the times you see.
--
Thorbjørn
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org
RE: [vfs] FTP extremely slow compared to SFTP
Posted by Jörg Schaible <Jo...@Elsag-Solutions.com>.
Mark Fortner wrote:
> Hi Jörg,
> I noticed a related thread on the list "FTPClient.storeFile is very
> slow". It might be worth looking into the FTP implementation to see
> if it's using the same library. Apparently the existing
> implementations don't use NIO.
Yep. commons-vfs is based on that implementation.
- Jörg
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org
Re: [vfs] FTP extremely slow compared to SFTP
Posted by Mark Fortner <ph...@yahoo.com>.
Hi Jörg,
I noticed a related thread on the list "FTPClient.storeFile is very
slow". It might be worth looking into the FTP implementation to see
if it's using the same library. Apparently the existing
implementations don't use NIO.
Regards,
Mark
On Dec 11, 2007 12:27 AM, Jörg Schaible
<Jo...@elsag-solutions.com> wrote:
> Hi Mark,
>
> Mark Fortner wrote:
> > Jörg,
> > Do you have any code snippets for these examples?
>
> Well, it is a bit more complex. Some files in the target directory are opened and read again, but this does not explain the numbers below. The only difference is the initialization of the FileSystem with a SFTP URL instead of one with FTP. Since SFTP adds overhead on top of FTP the only conclusion is, that the FTP implementation in use is horrible slow.
>
> - Jörg
>
>
> >
> > Mark
> >
> > On Dec 10, 2007 5:59 AM, Jörg Schaible
> > <Jo...@elsag-solutions.com>
> > wrote:
> >
> >> Hi,
> >>
> >> can somebody explain the following numbers - especially why FTP is
> >> that dead slow compared to SFTP handling a lot of files? I measure a
> >> simple copy action of a directory containing some few files:
> >>
> >> ========================================
> >> FTP
> >> ========================================
> >> * Initial Run
> >> - Exporter.............................................. 8252.0 ms
> >> * Single File (25kb)
> >> - Exporter.............................................. 8021.0 ms
> >> * Single File (25000kb)
> >> - Exporter.............................................. 11437.0 ms
> >> * Single File (50000kb)
> >> - Exporter.............................................. 16043.0 ms
> >> * Single Directory (with 10 files of 1kb)
> >> - Exporter.............................................. 47859.0 ms
> >> * Single Directory (with 25 files of 1kb)
> >> - Exporter.............................................. 113964.0 ms
> >> * Single Directory (with 50 files of 1kb)
> >> - Exporter.............................................. 224312.0 ms
> >>
> >> ========================================
> >> SFTP
> >> ========================================
> >> * Initial Run
> >> - Exporter.............................................. 1252.0 ms
> >> * Single File (25kb)
> >> - Exporter.............................................. 921.0 ms *
> >> Single File (25000kb)
> >> - Exporter.............................................. 14952.0 ms
> >> * Single File (50000kb)
> >> - Exporter.............................................. 24885.0 ms
> >> * Single Directory (with 10 files of 1kb)
> >> - Exporter.............................................. 601.0 ms
> >> * Single Directory (with 25 files of 1kb)
> >> - Exporter.............................................. 922.0 ms
> >> * Single Directory (with 50 files of 1kb)
> >> - Exporter.............................................. 1552.0 ms
> >>
> >>
> >> Any explanation or - even better - suggestions to improve FTP speed?
> >>
> >> - Jörg
> >>
> >>
> > ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> >> For additional commands, e-mail: user-help@commons.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> For additional commands, e-mail: user-help@commons.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org
RE: [vfs] FTP extremely slow compared to SFTP
Posted by Jörg Schaible <Jo...@Elsag-Solutions.com>.
Hi sebb,
sebb wrote:
> I don't know anything about VFS, but I do know that there are
> some systems
> that run a login script when FTP sessions start. If this
> takes a while to
> run, then it could perhaps explain the discrepancy if SFTP
> sessions run a
> different script - or none at all.
>
> Just a thought.
Thanks, but no, there's no such script.
> Perhaps it would be worth comparing command-line FTP and SFTP timings.
Standard FTP command-lines tend not to be very script friendly ;-)
- Jörg
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org
Re: [vfs] FTP extremely slow compared to SFTP
Posted by sebb <se...@gmail.com>.
I don't know anything about VFS, but I do know that there are some systems
that run a login script when FTP sessions start. If this takes a while to
run, then it could perhaps explain the discrepancy if SFTP sessions run a
different script - or none at all.
Just a thought.
Perhaps it would be worth comparing command-line FTP and SFTP timings.
On 11/12/2007, Jörg Schaible <Jo...@elsag-solutions.com> wrote:
>
> Hi Mark,
>
> Mark Fortner wrote:
> > Jörg,
> > Do you have any code snippets for these examples?
>
> Well, it is a bit more complex. Some files in the target directory are
> opened and read again, but this does not explain the numbers below. The only
> difference is the initialization of the FileSystem with a SFTP URL instead
> of one with FTP. Since SFTP adds overhead on top of FTP the only conclusion
> is, that the FTP implementation in use is horrible slow.
>
> - Jörg
>
> >
> > Mark
> >
> > On Dec 10, 2007 5:59 AM, Jörg Schaible
> > <Jo...@elsag-solutions.com>
> > wrote:
> >
> >> Hi,
> >>
> >> can somebody explain the following numbers - especially why FTP is
> >> that dead slow compared to SFTP handling a lot of files? I measure a
> >> simple copy action of a directory containing some few files:
> >>
> >> ========================================
> >> FTP
> >> ========================================
> >> * Initial Run
> >> - Exporter.............................................. 8252.0 ms
> >> * Single File (25kb)
> >> - Exporter.............................................. 8021.0 ms
> >> * Single File (25000kb)
> >> - Exporter.............................................. 11437.0 ms
> >> * Single File (50000kb)
> >> - Exporter.............................................. 16043.0 ms
> >> * Single Directory (with 10 files of 1kb)
> >> - Exporter.............................................. 47859.0 ms
> >> * Single Directory (with 25 files of 1kb)
> >> - Exporter.............................................. 113964.0 ms
> >> * Single Directory (with 50 files of 1kb)
> >> - Exporter.............................................. 224312.0 ms
> >>
> >> ========================================
> >> SFTP
> >> ========================================
> >> * Initial Run
> >> - Exporter.............................................. 1252.0 ms
> >> * Single File (25kb)
> >> - Exporter.............................................. 921.0 ms *
> >> Single File (25000kb)
> >> - Exporter.............................................. 14952.0 ms
> >> * Single File (50000kb)
> >> - Exporter.............................................. 24885.0 ms
> >> * Single Directory (with 10 files of 1kb)
> >> - Exporter.............................................. 601.0 ms
> >> * Single Directory (with 25 files of 1kb)
> >> - Exporter.............................................. 922.0 ms
> >> * Single Directory (with 50 files of 1kb)
> >> - Exporter.............................................. 1552.0 ms
> >>
> >>
> >> Any explanation or - even better - suggestions to improve FTP speed?
> >>
> >> - Jörg
> >>
> >>
> > ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> >> For additional commands, e-mail: user-help@commons.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> For additional commands, e-mail: user-help@commons.apache.org
>
>
RE: [vfs] FTP extremely slow compared to SFTP
Posted by Jörg Schaible <Jo...@Elsag-Solutions.com>.
Hi Mark,
Mark Fortner wrote:
> Jörg,
> Do you have any code snippets for these examples?
Well, it is a bit more complex. Some files in the target directory are opened and read again, but this does not explain the numbers below. The only difference is the initialization of the FileSystem with a SFTP URL instead of one with FTP. Since SFTP adds overhead on top of FTP the only conclusion is, that the FTP implementation in use is horrible slow.
- Jörg
>
> Mark
>
> On Dec 10, 2007 5:59 AM, Jörg Schaible
> <Jo...@elsag-solutions.com>
> wrote:
>
>> Hi,
>>
>> can somebody explain the following numbers - especially why FTP is
>> that dead slow compared to SFTP handling a lot of files? I measure a
>> simple copy action of a directory containing some few files:
>>
>> ========================================
>> FTP
>> ========================================
>> * Initial Run
>> - Exporter.............................................. 8252.0 ms
>> * Single File (25kb)
>> - Exporter.............................................. 8021.0 ms
>> * Single File (25000kb)
>> - Exporter.............................................. 11437.0 ms
>> * Single File (50000kb)
>> - Exporter.............................................. 16043.0 ms
>> * Single Directory (with 10 files of 1kb)
>> - Exporter.............................................. 47859.0 ms
>> * Single Directory (with 25 files of 1kb)
>> - Exporter.............................................. 113964.0 ms
>> * Single Directory (with 50 files of 1kb)
>> - Exporter.............................................. 224312.0 ms
>>
>> ========================================
>> SFTP
>> ========================================
>> * Initial Run
>> - Exporter.............................................. 1252.0 ms
>> * Single File (25kb)
>> - Exporter.............................................. 921.0 ms *
>> Single File (25000kb)
>> - Exporter.............................................. 14952.0 ms
>> * Single File (50000kb)
>> - Exporter.............................................. 24885.0 ms
>> * Single Directory (with 10 files of 1kb)
>> - Exporter.............................................. 601.0 ms
>> * Single Directory (with 25 files of 1kb)
>> - Exporter.............................................. 922.0 ms
>> * Single Directory (with 50 files of 1kb)
>> - Exporter.............................................. 1552.0 ms
>>
>>
>> Any explanation or - even better - suggestions to improve FTP speed?
>>
>> - Jörg
>>
>>
> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
>> For additional commands, e-mail: user-help@commons.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org
Re: [vfs] FTP extremely slow compared to SFTP
Posted by Mark Fortner <ph...@yahoo.com>.
Jörg,
Do you have any code snippets for these examples?
Mark
On Dec 10, 2007 5:59 AM, Jörg Schaible <Jo...@elsag-solutions.com>
wrote:
> Hi,
>
> can somebody explain the following numbers - especially why FTP is that
> dead slow compared to SFTP handling a lot of files? I measure a simple copy
> action of a directory containing some few files:
>
> ========================================
> FTP
> ========================================
> * Initial Run
> - Exporter.............................................. 8252.0 ms
> * Single File (25kb)
> - Exporter.............................................. 8021.0 ms
> * Single File (25000kb)
> - Exporter.............................................. 11437.0 ms
> * Single File (50000kb)
> - Exporter.............................................. 16043.0 ms
> * Single Directory (with 10 files of 1kb)
> - Exporter.............................................. 47859.0 ms
> * Single Directory (with 25 files of 1kb)
> - Exporter.............................................. 113964.0 ms
> * Single Directory (with 50 files of 1kb)
> - Exporter.............................................. 224312.0 ms
>
> ========================================
> SFTP
> ========================================
> * Initial Run
> - Exporter.............................................. 1252.0 ms
> * Single File (25kb)
> - Exporter.............................................. 921.0 ms
> * Single File (25000kb)
> - Exporter.............................................. 14952.0 ms
> * Single File (50000kb)
> - Exporter.............................................. 24885.0 ms
> * Single Directory (with 10 files of 1kb)
> - Exporter.............................................. 601.0 ms
> * Single Directory (with 25 files of 1kb)
> - Exporter.............................................. 922.0 ms
> * Single Directory (with 50 files of 1kb)
> - Exporter.............................................. 1552.0 ms
>
>
> Any explanation or - even better - suggestions to improve FTP speed?
>
> - Jörg
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> For additional commands, e-mail: user-help@commons.apache.org
>
>