You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@commons.apache.org by "Alex Crane (JIRA)" <ji...@apache.org> on 2007/05/18 09:07:17 UTC
[jira] Created: (VFS-161) Minimize using of LIST command in the FTP
provider
Minimize using of LIST command in the FTP provider
--------------------------------------------------
Key: VFS-161
URL: https://issues.apache.org/jira/browse/VFS-161
Project: Commons VFS
Issue Type: Improvement
Affects Versions: 1.1
Reporter: Alex Crane
Priority: Minor
I have a big bis storage of files (about 10 TBytes).
Path to file looks like /storage/storage1/node1/76423fds7g78df6gdf7hdfh/2384dg7sdfg8dfh.dat
Existance checking and deleteing of files takes very looooooooong time because of LIST command:
LIST /storage
...
LIST /storage/storage1
...
LIST /storage/storage1/node1
....
LIST /storage/storage1/node1/76423fds7g78df6gdf7hdfh
and so on.
For example, there is no need to perform LIST command to delete file. I`m using FTPClients from commons.net and it works 1000x faster.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
[jira] Updated: (VFS-161) Minimize using of LIST command in the FTP
provider
Posted by "Mario Ivankovits (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/VFS-161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mario Ivankovits updated VFS-161:
---------------------------------
Priority: Major (was: Minor)
> Minimize using of LIST command in the FTP provider
> --------------------------------------------------
>
> Key: VFS-161
> URL: https://issues.apache.org/jira/browse/VFS-161
> Project: Commons VFS
> Issue Type: Improvement
> Affects Versions: 1.1
> Reporter: Alex Crane
>
> I have a big bis storage of files (about 10 TBytes).
> Path to file looks like /storage/storage1/node1/76423fds7g78df6gdf7hdfh/2384dg7sdfg8dfh.dat
> Existance checking and deleteing of files takes very looooooooong time because of LIST command:
> LIST /storage
> ...
> LIST /storage/storage1
> ...
> LIST /storage/storage1/node1
> ....
> LIST /storage/storage1/node1/76423fds7g78df6gdf7hdfh
> and so on.
> For example, there is no need to perform LIST command to delete file. I`m using FTPClients from commons.net and it works 1000x faster.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
[jira] Commented: (VFS-161) Minimize using of LIST command in the
FTP provider
Posted by "Sebastian Skoglund (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/VFS-161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12509808 ]
Sebastian Skoglund commented on VFS-161:
----------------------------------------
This is a major performance problem when handling many tasks in a "batch-job".
If I for instance want to create a numerous of files, rename and delete some others it can take several minutes for a fairly small amount of operations. I have done some testing and measurments regarding this matter and if I want to create, rename and delete only 22 files (with a total size of ~300kB) it takes as long as 72 seconds, which is unacceptable in my opinion.
> Minimize using of LIST command in the FTP provider
> --------------------------------------------------
>
> Key: VFS-161
> URL: https://issues.apache.org/jira/browse/VFS-161
> Project: Commons VFS
> Issue Type: Improvement
> Affects Versions: 1.1
> Reporter: Alex Crane
> Priority: Minor
>
> I have a big bis storage of files (about 10 TBytes).
> Path to file looks like /storage/storage1/node1/76423fds7g78df6gdf7hdfh/2384dg7sdfg8dfh.dat
> Existance checking and deleteing of files takes very looooooooong time because of LIST command:
> LIST /storage
> ...
> LIST /storage/storage1
> ...
> LIST /storage/storage1/node1
> ....
> LIST /storage/storage1/node1/76423fds7g78df6gdf7hdfh
> and so on.
> For example, there is no need to perform LIST command to delete file. I`m using FTPClients from commons.net and it works 1000x faster.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
[jira] Issue Comment Edited: (VFS-161) Minimize using of LIST
command in the FTP provider
Posted by "Sebastian Skoglund (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/VFS-161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12509808 ]
Sebastian Skoglund edited comment on VFS-161 at 7/3/07 2:53 AM:
----------------------------------------------------------------
This is a major performance problem when handling many tasks in a "batch-job".
If I for instance want to create a numerous of files, rename and delete some others it can take several minutes for a fairly small amount of operations. I have done some testing and measurments regarding this matter and if I want to create, rename and delete only 22 files (with a total size of ~300kB) it takes as long as 72 seconds, which is unacceptable in my opinion.
FYI.. the same job takes 2 seconds using SFTP.
was:
This is a major performance problem when handling many tasks in a "batch-job".
If I for instance want to create a numerous of files, rename and delete some others it can take several minutes for a fairly small amount of operations. I have done some testing and measurments regarding this matter and if I want to create, rename and delete only 22 files (with a total size of ~300kB) it takes as long as 72 seconds, which is unacceptable in my opinion.
> Minimize using of LIST command in the FTP provider
> --------------------------------------------------
>
> Key: VFS-161
> URL: https://issues.apache.org/jira/browse/VFS-161
> Project: Commons VFS
> Issue Type: Improvement
> Affects Versions: 1.1
> Reporter: Alex Crane
> Priority: Minor
>
> I have a big bis storage of files (about 10 TBytes).
> Path to file looks like /storage/storage1/node1/76423fds7g78df6gdf7hdfh/2384dg7sdfg8dfh.dat
> Existance checking and deleteing of files takes very looooooooong time because of LIST command:
> LIST /storage
> ...
> LIST /storage/storage1
> ...
> LIST /storage/storage1/node1
> ....
> LIST /storage/storage1/node1/76423fds7g78df6gdf7hdfh
> and so on.
> For example, there is no need to perform LIST command to delete file. I`m using FTPClients from commons.net and it works 1000x faster.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org