[PATCH] Whole directory moves optimized

[Magesh Umasankar <um...@rediffmail.com>]

The move task, as it is now, does not optimally perform
moves of entire directories.  For example, if directory
A is to be moved to directory B, all files in directory
A are moved to directory B one by one recursively, 
thereby accomplishing the directory move.  

If the directory itself can be renamed instead of 
taking the above approach, it will greatly improve 
performance.

This patch does precisely that when 'normal' directory
moves are performed.  It operates at the fileset level.
Each fileset that has an empty 'notIncluded' file/dir 
list is treated as a candidate for a dir rename since
this means that all files in the dir has been included.
Of course, if flatten or mappers are being used, a
straight forward directory rename isn't attempted.

Magesh
 

Re: [PATCH] Whole directory moves optimized

[Stefan Bodewig <bo...@apache.org>]

On 1 Nov 2001, Magesh Umasankar <um...@rediffmail.com> wrote:

> This patch does precisely that when 'normal' directory
> moves are performed.  It operates at the fileset level.
> Each fileset that has an empty 'notIncluded' file/dir 
> list is treated as a candidate for a dir rename since
> this means that all files in the dir has been included.

Depending on the fileset we are operating on, this may seriously
decrease performance, achieving the opposite of what you want to do.

Say we are copying only a few files out of a very large directory
tree, almost all directories are excluded by default.  In the current
situation many directories will not be scanned at all (the fast mode),
but if we apply your patch, all directories will be scanned.

It would probably be OK to just perform the additional scan in the
move case, but even then it may make things worse than they are now.

Stefan

--
To unsubscribe, e-mail:   <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>

Re: [PATCH] Whole directory moves optimized

[Jesse Glick <je...@sun.com>]

Magesh Umasankar wrote:
> The move task, as it is now, does not optimally perform
> moves of entire directories.  For example, if directory
> A is to be moved to directory B, all files in directory
> A are moved to directory B one by one recursively,
> thereby accomplishing the directory move.
> 
> If the directory itself can be renamed instead of
> taking the above approach, it will greatly improve
> performance.
> [snip]

Won't this fail on e.g. Unix machines when moving a directory between file
systems? In such a case a raw inode rename is impossible. Smart Unix tools
like GNU mv (I think, not sure) will try to do such an optimization. But they
have access to the filesystem mount point information, which is not available
from Java code. And of course other platforms probably have different rules
about when this is permitted.

I guess you could try the rename first, and if it fails, fall back to the copy
and delete. Hopefully File.renameTo is atomic, i.e. it either succeeds or does
nothing. Javadoc does not say.

-Jesse

-- 
Jesse Glick        <ma...@sun.com>
NetBeans, Open APIs  <http://www.netbeans.org/>

--
To unsubscribe, e-mail:   <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>