You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ant.apache.org by Kev Jackson <ke...@it.fts-vn.com> on 2006/04/21 06:41:24 UTC
[patch] NioFileUtils, Java6FileUtils, FileUtilsAdapter + factory,
build.xml (was Re: AW: Adding a methof to StringUtils)
Ok, this is what I've managed to bash together this morning:
- NioFileUtils (uses Java1.4 NIO libraries, altered the TRANSFER_SIZE as
suggested to 32000, currently no way to override that (yet))
- Java6FileUtils (currently has no implementation, but could be used in
future for implementing Java6 features (file permisions etc))
- FileUtils (refactored to imlement interface, still needs to have all
comments refactored into @see -> interface)
- FileUtilsAdapter (interface of FileUtils)
- FileUtilsFactory (uses runtime class-loading to select correct
version, also allows for selection of impl via system property or ant
property (project.getProperty), as suggested, caches the instance of the
FileUtilsAdapter on first use. This assumes that during a build a user
won't want to swap out FilUtils implementations.)
- build.xml (contains the selector code for [Nio|Java6]FileUtils)
Current problems:
I cannot get the current code to select the implementation based on the
property I'm specifying - (I'm using .ant.properties and I've also tried
-D style props). Any hint as to how this mechanism works in general
would be appreciated.
Code is just in util package, it would be nicer in it's own separate
package - open to suggestions on this.
Not sure that the selector in the build is working - after bootstrap +
build, the ant-nodeps.jar doesn't contain the Nio or the Java6FileUtils
- should they be here?
Patch attached for consideration and comments.
Btw, swapping to NIO does help the AppFuse build to get past the copying
- it now fails on HtmlUnit - which we can't do much about. So
(anecdotally anyway), NIO seems to use less memory than traditional IO.
Thanks
Kev
Re: [patch] NioFileUtils, Java6FileUtils, FileUtilsAdapter + factory, build.xml (was Re: AW: Adding a methof to StringUtils)
Posted by Matt Benson <gu...@yahoo.com>.
--- Antoine Levy-Lambert <an...@gmx.de> wrote:
[SNIP]
>
> If anyway ResourceUtils does the work now,
> ResourceUtils could test
> whether running under Java 1.4 or more and call
> NIOUtils to do the work
> happening between lines 364 to 379 of ResourceUtils
> (copying files with no filters, no bells and
> whistles).
>
Shouldn't we simply extend nio copy support to
ResourceUtils? Another version-dependent delegate
could use Channels.newChannel() with the resources'
Input/OutputStreams... assuming that the
implementation is smart enough to call getChannel()
against a FileInputStream, for example. We could test
this; if it doesn't work we can always code that
ourselves. Could be overall worthwhile if we would
pick up performance increases on e.g. url streams...
-Matt
> Regards,
>
> Antoine
>
>
>
---------------------------------------------------------------------
> To unsubscribe, e-mail:
> dev-unsubscribe@ant.apache.org
> For additional commands, e-mail:
> dev-help@ant.apache.org
>
>
__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org
Re: [patch] NioFileUtils, Java6FileUtils, FileUtilsAdapter + factory,
build.xml (was Re: AW: Adding a methof to StringUtils)
Posted by Antoine Levy-Lambert <an...@gmx.de>.
Kev Jackson wrote:
>
>> In that case I would like some time to think things over, as to
>> provide one face, and to have different implementations for currently
>> different parts.
>> Classes that support copying, things that support permissions, things
>> that support path manipulation, without getting a really big number
>> of classes.
>>
>> FileUtils for Java 1.6 on Windows
>> FileUtils for Java 1.4 on VMS
>> etc.
>>
>> I had an idea to turn FileUtils into a Facade that delegates
>> different actions to different back-ends.
>>
> I think I can see where you are going with this, and it's definitely
> an alternative to what I thought to begin with and the direction I was
> heading in the code. I'm viewing the cleanest split as using an
> interface, from your perspective you think a master FileUtils (class
> not interface) that delegates all the real work to a specialization
> class. I think that your view would provide the quickest benefit - ie
> it would be more easily added to the entire codebase (as there would
> be no need to remove references to FileUtils from all the tasks).
>
> Thanks for your thoughts
> Kev
>
Hi,
in this case, what we could do would be to create a new package
org.apache.tools.ant.util.java14 (this is OK for bootstrapping and
build.xml)
and create there a class NIOUtils or something like that with some
static methods, including simply one method which copies one file to the
other.
If anyway ResourceUtils does the work now, ResourceUtils could test
whether running under Java 1.4 or more and call NIOUtils to do the work
happening between lines 364 to 379 of ResourceUtils
(copying files with no filters, no bells and whistles).
Regards,
Antoine
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org
Re: [patch] NioFileUtils, Java6FileUtils, FileUtilsAdapter + factory,
build.xml (was Re: AW: Adding a methof to StringUtils)
Posted by jm...@kruithof.xs4all.nl.
>> Ist es nicht schrecklich, dass der menschlichen Klugheit so enge Grenzen
>> gesetzt sind und der menschlichen Dummheit �berhaupt keine? (Konrad
>> Adenauer)
>>
>> Isn't it terrible that so close borders are set to the human intelligence
>> and the human stupidity at all none?
>>
> This is a nice quote (totally OT I hope :)
>
It never is totally off topic, and is definitely including myself.
The sheer fact that it came through is my own stupidity.
(For some reason one of my mail clients puts this signature on top of
every reply, which I normally kill out when sending things to mailing
lists).
Martijn
Re: [patch] NioFileUtils, Java6FileUtils, FileUtilsAdapter + factory,
build.xml (was Re: AW: Adding a methof to StringUtils)
Posted by Kev Jackson <ke...@it.fts-vn.com>.
>
> Ist es nicht schrecklich, dass der menschlichen Klugheit so enge
> Grenzen gesetzt sind und der menschlichen Dummheit überhaupt keine?
> (Konrad Adenauer)
>
> Isn't it terrible that so close borders are set to the human
> intelligence and the human stupidity at all none?
>
This is a nice quote (totally OT I hope :)
> On Fri, 21 Apr 2006, Kev Jackson wrote:
>
>> I'm viewing the cleanest split as using an interface, from your
>> perspective you think a master FileUtils (class not interface) that
>> delegates all the real work to a specialization class.
>
>
> Not exactly, I am thinking about delegating to different
> specialization classes. Some aspects depend on java version, some
> aspects on os version, some aspects on availability of 3pp. To be able
> to handle that I think it is better to split functionality up.
sorry yes in my mail I said class when of course I meant class(es) plural.
Kev
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org
Re: [patch] NioFileUtils, Java6FileUtils, FileUtilsAdapter + factory,
build.xml (was Re: AW: Adding a methof to StringUtils)
Posted by jm...@kruithof.xs4all.nl.
Ist es nicht schrecklich, dass der menschlichen Klugheit so enge Grenzen gesetzt sind und der menschlichen Dummheit �berhaupt keine? (Konrad Adenauer)
Isn't it terrible that so close borders are set to the human intelligence and the human stupidity at all none?
On Fri, 21 Apr 2006, Kev Jackson wrote:
> I'm viewing the cleanest split as using an interface, from your
> perspective you think a master FileUtils (class not interface) that delegates
> all the real work to a specialization class.
Not exactly, I am thinking about delegating to different specialization
classes. Some aspects depend on java version, some aspects on os version,
some aspects on availability of 3pp. To be able to handle that I think it
is better to split functionality up.
Martijn
Re: [patch] NioFileUtils, Java6FileUtils, FileUtilsAdapter + factory,
build.xml (was Re: AW: Adding a methof to StringUtils)
Posted by Kev Jackson <ke...@it.fts-vn.com>.
> In that case I would like some time to think things over, as to
> provide one face, and to have different implementations for currently
> different parts.
> Classes that support copying, things that support permissions, things
> that support path manipulation, without getting a really big number of
> classes.
>
> FileUtils for Java 1.6 on Windows
> FileUtils for Java 1.4 on VMS
> etc.
>
> I had an idea to turn FileUtils into a Facade that delegates different
> actions to different back-ends.
>
I think I can see where you are going with this, and it's definitely an
alternative to what I thought to begin with and the direction I was
heading in the code. I'm viewing the cleanest split as using an
interface, from your perspective you think a master FileUtils (class not
interface) that delegates all the real work to a specialization class.
I think that your view would provide the quickest benefit - ie it would
be more easily added to the entire codebase (as there would be no need
to remove references to FileUtils from all the tasks).
Thanks for your thoughts
Kev
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org
Re: [patch] NioFileUtils, Java6FileUtils, FileUtilsAdapter + factory,
build.xml (was Re: AW: Adding a methof to StringUtils)
Posted by Martijn Kruithof <jm...@kruithof.xs4all.nl>.
Kev Jackson wrote:
>> 2) If the FileUtilsFactory is called from or implemented in
>> FileUtils.getFileUtils() / FileUtils.newFileUtils(), every task is
>> going to benefit.
>>
> I think the implementation I provided does precisely this, or rather
> it calls from the interface (FileUtilsAdapter) instead of from the
> implementation. If you mean that the current FileUtils should
> delegate to another implementation using the factory that's different.
>
> I think it would be cleaner to gradually transition all the tasks over
> to an interface (while retaining the FileUtils semi-singleton). At
> the point where all the tasks are using FileUtilsAdapter, we can
> refactor and remove the parts of FileUtils that are exposed correctly
> in the interface, maintaining bwc that way - the only thing left would
> be a small stub which contains the static methods - that's my view of
> how a transition would work anyway.
>
> I think that using FileUtils to delegate/proxy to a real class would
> be less 'elegant' than using an interface - although I can see the
> immediate advantages of using the approach you mentioned (all tasks
> benefit without actually having to change each class over to the
> interface).
In that case I would like some time to think things over, as to provide
one face, and to have different implementations for currently different
parts.
Classes that support copying, things that support permissions, things
that support path manipulation, without getting a really big number of
classes.
FileUtils for Java 1.6 on Windows
FileUtils for Java 1.4 on VMS
etc.
I had an idea to turn FileUtils into a Facade that delegates different
actions to different back-ends.
Thanks for your elaborate reaction.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org
Re: [patch] NioFileUtils, Java6FileUtils, FileUtilsAdapter + factory,
build.xml (was Re: AW: Adding a methof to StringUtils)
Posted by Kev Jackson <ke...@it.fts-vn.com>.
Martijn Kruithof wrote:
> Hi,
>
> Adding setting the property using -D to ANT_OPTS should do.
>
> 5 things:
>
> 1) I saw you removed public members (constants) from FileUtils (Non
> backward compatible change, keeping everything BC also frustrates me
> sometimes, but still I think it is best not to break parts we do not
> know of.)
>
Yes that was a mistake, I thought that they'd be available from the
interface, but realised after I sent the email that only Copy is using
the interface and all other tasks are still tied to the implementation.
I need to put this back in the default impl to retain bwc as you mentioned.
> 2) If the FileUtilsFactory is called from or implemented in
> FileUtils.getFileUtils() / FileUtils.newFileUtils(), every task is
> going to benefit.
>
I think the implementation I provided does precisely this, or rather it
calls from the interface (FileUtilsAdapter) instead of from the
implementation. If you mean that the current FileUtils should delegate
to another implementation using the factory that's different.
I think it would be cleaner to gradually transition all the tasks over
to an interface (while retaining the FileUtils semi-singleton). At the
point where all the tasks are using FileUtilsAdapter, we can refactor
and remove the parts of FileUtils that are exposed correctly in the
interface, maintaining bwc that way - the only thing left would be a
small stub which contains the static methods - that's my view of how a
transition would work anyway.
I think that using FileUtils to delegate/proxy to a real class would be
less 'elegant' than using an interface - although I can see the
immediate advantages of using the approach you mentioned (all tasks
benefit without actually having to change each class over to the interface).
> 3) Again / Still the actual copying has been moved to the
> ResourceUtils, how do we make sure that files addressed as resources
> also benefit from this change.
>
I'm not sure what you mean by this - the code is proof of concept to
show that it would be possible to add in nio based file handling in a
way that minimizes problems of bwc. I didn't attempt to add nio for
handling resources - indeed that would mean changing the ResourceUtils
class, which I didn't want to do as it would have been a big waste of
time if the overall strategy was wrong. Right now I just want to play
with some basic ideas and see if people here on dev list thing that the
strategy is good/hopelessly broken/not worth pursuing
The intention isn't to be a complete solution to all resource handling,
just to be complete enough to provide a background for further
discussion - I fully expect most of the code to be thrown away, but
without something that executes it's hard to see the potential flaws.
> Regarding the changes in the Copy task:
>
> 4) Stringbuffer is not more efficient than concatenation using + in a
> single statement.
>
[snip]
Sorry this was change that I didn't roll-back from earlier, it shouldn't
be there
> 5) StringBuffer handling is incorrect for the second part, (you will
> concatenate all to filenames.)
again please dis-regard this part of the code as it isn't meant to be there
This shows what happens when you have several different ideas being
worked on at the same time and you create a unified diff without
looking! (my bad)
Thanks for the comments
Kev
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org
Re: [patch] NioFileUtils, Java6FileUtils, FileUtilsAdapter + factory,
build.xml (was Re: AW: Adding a methof to StringUtils)
Posted by Martijn Kruithof <jm...@kruithof.xs4all.nl>.
Hi,
Adding setting the property using -D to ANT_OPTS should do.
5 things:
1) I saw you removed public members (constants) from FileUtils (Non
backward compatible change, keeping everything BC also frustrates me
sometimes, but still I think it is best not to break parts we do not
know of.)
2) If the FileUtilsFactory is called from or implemented in
FileUtils.getFileUtils() / FileUtils.newFileUtils(), every task is going
to benefit.
3) Again / Still the actual copying has been moved to the ResourceUtils,
how do we make sure that files addressed as resources also benefit from
this change.
Regarding the changes in the Copy task:
4) Stringbuffer is not more efficient than concatenation using + in a
single statement.
- log("Copying " + fileCopyMap.size()
- + " file" + (fileCopyMap.size() == 1 ? "" : "s")
- + " to " + destDir.getAbsolutePath());
+ StringBuffer sb = new StringBuffer("Copying ")
+ .append(fileCopyMap.size())
+ .append(" file")
+ .append((fileCopyMap.size() == 1 ? "" : "s"))
+ .append(" to ")
+ .append(destDir.getAbsolutePath());
Why are you replacing the log statmenent with the StringBuffer.
It is less readable, and equivalent.
5) StringBuffer handling is incorrect for the second part, (you will
concatenate all to filenames.)
Martijn
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org
Re: [patch] NioFileUtils, Java6FileUtils, FileUtilsAdapter +
factory, build.xml
Posted by Stefan Bodewig <bo...@apache.org>.
On Sun, 23 Apr 2006, Antoine Levy-Lambert <an...@gmx.de> wrote:
> Similarly, I wonder what happens for a user who would
>
> 1) build ant under JDK 1.2,
> 2) run ant under JDK 1.4,
>
> and attempt to use the RegexpFactory, for instance through the
> <replaceregexp/> task ?.
>
> The RegexpFactory does not check whether
> org.apache.tools.ant.util.regexp.Jdk14RegexpRegexp is present,
> only whether java.util.regex.Matcher is present.
>
> Actually, I was wondering whether we are not already forbidding - de
> facto and silently - this type of use cases.
I vaguely recall a bug report along those lines. Yes, you are
correct, the current code simply assumes you have all implementations
around.
Stefan
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org
Re: [patch] NioFileUtils, Java6FileUtils, FileUtilsAdapter + factory,
build.xml (was Re: AW: Adding a methof to StringUtils)
Posted by Antoine Levy-Lambert <an...@gmx.de>.
Kev Jackson wrote:
>
> I'm coming round to think that Martijn's approach would be the least
> disruptive to the code base - simply as the delegate approach wouldn't
> mean recoding all tasks to use an interface instead of an
> implementation. This reduces the effort a lot and would make it less
> likely that errors would creep in. Definitely my +1 for the delegate
> approach, +0 for the interfaces and factory approach (ie if for some
> reason we can't do the delagate approach, the interfaces should be
> reconsidered)
>
Sounds good.
When you plugin the nio, it would be good if you test the following :
- building under 1.4 or 1.5, running under 1.2
- I am also concerned about users building their own ant distribution
under Java 1.2, then running it under 1.4 and 1.5, and then complaining
that org.apache.tools.ant.types.jdk14 cannot be found.
Similarly, I wonder what happens for a user who would
1) build ant under JDK 1.2,
2) run ant under JDK 1.4,
and attempt to use the RegexpFactory, for instance through the
<replaceregexp/> task ?.
The RegexpFactory does not check whether
org.apache.tools.ant.util.regexp.Jdk14RegexpRegexp is present,
only whether java.util.regex.Matcher is present.
Actually, I was wondering whether we are not already forbidding - de
facto and silently - this type of use cases.
Regards,
Antoine
Antoine
> Thanks
> Kev
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org
Re: [patch] NioFileUtils, Java6FileUtils, FileUtilsAdapter + factory,
build.xml (was Re: AW: Adding a methof to StringUtils)
Posted by Kev Jackson <ke...@it.fts-vn.com>.
>FileUtils to select (Martijn's approach is
>ridiculously insane enough to have captured my fancy
>;) ), it must also take into account whether the
>desired classes are available, falling back to the
>basic impl. otherwise.
>
>
I'm coming round to think that Martijn's approach would be the least
disruptive to the code base - simply as the delegate approach wouldn't
mean recoding all tasks to use an interface instead of an
implementation. This reduces the effort a lot and would make it less
likely that errors would creep in. Definitely my +1 for the delegate
approach, +0 for the interfaces and factory approach (ie if for some
reason we can't do the delagate approach, the interfaces should be
reconsidered)
Thanks
Kev
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org
Re: [patch] NioFileUtils, Java6FileUtils, FileUtilsAdapter + factory, build.xml (was Re: AW: Adding a methof to StringUtils)
Posted by Matt Benson <gu...@yahoo.com>.
--- Antoine Levy-Lambert <an...@gmx.de> wrote:
[SNIP]
> One issue :
>
> If NIOFileUtils is in the same package as FileUtils,
> bootstrapping under
> JDK 1.2 or 1.3 would not work.
> The bootstrap scripts (build.bat, build.sh) are
> using primitive
> mechanisms (directories) to find out what should be
> built first.
>
Oh yeah... forgot about this! :) Now I see what you
meant here. We must keep that in mind... the
bootstrap scripts build enough of Ant that it can
build the rest of itself, itself... correct? Meaning
that whatever mechanism is used to determine which
FileUtils to select (Martijn's approach is
ridiculously insane enough to have captured my fancy
;) ), it must also take into account whether the
desired classes are available, falling back to the
basic impl. otherwise.
Right?
-Matt
__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org
Re: [patch] NioFileUtils, Java6FileUtils, FileUtilsAdapter + factory,
build.xml (was Re: AW: Adding a methof to StringUtils)
Posted by Antoine Levy-Lambert <an...@gmx.de>.
Hello Kevin,
Kev Jackson wrote:
> Ok, this is what I've managed to bash together this morning:
>
> - NioFileUtils (uses Java1.4 NIO libraries, altered the TRANSFER_SIZE
> as suggested to 32000, currently no way to override that (yet))
> - Java6FileUtils (currently has no implementation, but could be used
> in future for implementing Java6 features (file permisions etc))
> - FileUtils (refactored to imlement interface, still needs to have all
> comments refactored into @see -> interface)
> - FileUtilsAdapter (interface of FileUtils)
> - FileUtilsFactory (uses runtime class-loading to select correct
> version, also allows for selection of impl via system property or ant
> property (project.getProperty), as suggested, caches the instance of
> the FileUtilsAdapter on first use. This assumes that during a build a
> user won't want to swap out FilUtils implementations.)
> - build.xml (contains the selector code for [Nio|Java6]FileUtils)
>
> Current problems:
> I cannot get the current code to select the implementation based on
> the property I'm specifying - (I'm using .ant.properties and I've also
> tried -D style props). Any hint as to how this mechanism works in
> general would be appreciated.
Did you try to debug it ? It seems to work well for regular expressions.
> Code is just in util package, it would be nicer in it's own separate
> package - open to suggestions on this.
> Not sure that the selector in the build is working - after bootstrap +
> build, the ant-nodeps.jar doesn't contain the Nio or the
> Java6FileUtils - should they be here?
>
> Patch attached for consideration and comments.
>
One issue :
If NIOFileUtils is in the same package as FileUtils, bootstrapping under
JDK 1.2 or 1.3 would not work.
The bootstrap scripts (build.bat, build.sh) are using primitive
mechanisms (directories) to find out what should be built first.
> Btw, swapping to NIO does help the AppFuse build to get past the
> copying - it now fails on HtmlUnit - which we can't do much about. So
> (anecdotally anyway), NIO seems to use less memory than traditional IO.
>
Good news.
> Thanks
> Kev
> ------------------------------------------------------------------------
Regards,
Antoine
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org