You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ant.apache.org by Alexey Panchenko <al...@olmisoft.com> on 2006/04/19 12:38:48 UTC
Re[2]: AW: Adding a methof to StringUtils
Kev Jackson wrote:
> It's less to do with speed performance and more to do with memory
> performance. "a" + "b" + "c" creates "a" + "bc" <- intermediate String,
> and then "abc"
Many years ago - yes, but not now.
java version "1.5.0_06"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_06-b05)
Java HotSpot(TM) Client VM (build 1.5.0_06-b05, mixed mode)
public class AAA {
public String concat(String a, String b, String c) {
return a + b + c;
}
}
javac AAA.java
javap -c AAA
public java.lang.String concat(java.lang.String, java.lang.String, java.lang.String);
Code:
0: new #2; //class java/lang/StringBuilder
3: dup
4: invokespecial #3; //Method java/lang/StringBuilder."<init>":()V
7: aload_1
8: invokevirtual #4; //Method java/lang/StringBuilder.append:(Ljava/lang/String;)Ljava/lang/StringBuilder;
11: aload_2
12: invokevirtual #4; //Method java/lang/StringBuilder.append:(Ljava/lang/String;)Ljava/lang/StringBuilder;
15: aload_3
16: invokevirtual #4; //Method java/lang/StringBuilder.append:(Ljava/lang/String;)Ljava/lang/StringBuilder;
19: invokevirtual #5; //Method java/lang/StringBuilder.toString:()Ljava/lang/String;
22: areturn
javac -source 1.4 -target 1.4 AAA.java
javap -c AAA
public java.lang.String concat(java.lang.String, java.lang.String, java.lang.String);
Code:
0: new #2; //class java/lang/StringBuffer
3: dup
4: invokespecial #3; //Method java/lang/StringBuffer."<init>":()V
7: aload_1
8: invokevirtual #4; //Method java/lang/StringBuffer.append:(Ljava/lang/String;)Ljava/lang/StringBuffer;
11: aload_2
12: invokevirtual #4; //Method java/lang/StringBuffer.append:(Ljava/lang/String;)Ljava/lang/StringBuffer;
15: aload_3
16: invokevirtual #4; //Method java/lang/StringBuffer.append:(Ljava/lang/String;)Ljava/lang/StringBuffer;
19: invokevirtual #5; //Method java/lang/StringBuffer.toString:()Ljava/lang/String;
22: areturn
--
Best regards,
Alexey mailto:alex+news@olmisoft.com
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org
Re: AW: Adding a methof to StringUtils
Posted by Matt Benson <gu...@yahoo.com>.
--- Dominique Devienne <dd...@gmail.com> wrote:
> > The point about building the strings when they
> aren't used (because
> > logging verbosity is set too low) still stands
> though - this is less
> > than efficient
>
> The point, which Matt already raised, is that you
> can't know the level
> at which the logger and the listeners are set. There
Actually, I think that was Jan. :)
> is nothing ATM in
> the Ant codebase to tip off the project as to the
> min log level of all
> logger+listener to implement your optimization. This
> could be can be
> added though, and would be useful IMHO, but that's a
> separate thread.
>
> The StringBuffer optimization is well known. I've
> either read about it
> in Effective Java or the "K&R" of Java,
> The.Java.Prog.Lang.
>
> I think what you propose to do would clutter the
> code, and make it
> ugly frankly ;-) I'd probably -1 it unless you can
> show hard evidence
> of it's usefulness.
>
> If <delete> is causing problem, replacing it by a
> custom task which is
> streamlined and doesn't need to buffer all files to
> delete beforehand
> for example. --DD
>
I had tried to make <delete> do as little buffering as
possible. The original resourceCollection-enabled
version was reported by Alexey to present a major
slowdown for existing large fileset deletions, but I
refactored the fileset handling to try to avoid this.
At present I can see where more speed could possibly
be coaxed out here, but it would cost memory which is
apparently already a problem. Any relevant profiling
results would be of interest to me.
-Matt
>
---------------------------------------------------------------------
> To unsubscribe, e-mail:
> dev-unsubscribe@ant.apache.org
> For additional commands, e-mail:
> dev-help@ant.apache.org
>
>
__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org
Re: AW: Adding a methof to StringUtils
Posted by Steve Loughran <st...@apache.org>.
Kev Jackson wrote:
>>
> If it came to it I'd -1 it too! I don't like any of the solutions I
> could come up with yesterday, the one I showed was the 'least worst'
> that I could think of, with a semi-upgrade path to Java5 style varargs
> (use an object array). I was mainly throwing the idea out to see what
> peoples reactions were - overall I don't think it's the right way to
> solve the problem - the real problem is that the level of logging cannot
> currently be determined and so any optimization (for memory or
> performance) is actually changing the behaviour of the code - which it
> shouldn't do.
>
> The delete task was just an example - I was looking at it to fix 'delete
> task won't be quiet' bug in bugzilla, and I was also thinking about the
> problem with AppFuse, (which does use Delete a little, but not as much
> as Copy and other tasks), so it seemed like a handy guinea pig as I had
> the code open at the time.
>
> I was also looking at Copy and saw that ResourceUtils.copyResource is a
> static method, but FileUtils.copyFile is not even though it delegates to
> copyResource, this means that in Copy there must be an instantiated
> fileUtils object, just to perform the copy, unfortunately the FileUtils
> interface/API is public so changing it would break bwc, but I'd like to
> add a static method for copyFile so that Copy wont need to instantiate
> FileUtils.
One thing I've always been curious about is how much speedup we'd get by
turning off all logging. That is, if I modified log() to discard its
contents, how much faster would everything be? We couldnt use the test
suite as a benchmark, because too many tests depend on the log, and it
probably isnt realistic.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org
Re: AW: Adding a methof to StringUtils
Posted by Kev Jackson <ke...@it.fts-vn.com>.
Antoine Levy-Lambert wrote:
>Matt Benson wrote:
>
>
>>Kev: I've asked some stuff about FileUtils before...
>>historically it was thought it might be pluggable,
>>hence all the instance methods. But that never
>>happened so the static methods crept in. I think it
>>was Martijn who really pushed in the FileUtils
>>singleton. Most places in Ant's code don't
>>instantiate their own FileUtils anymore, and if they
>>do they shouldn't; they should use the singleton.
>>
>>-Matt
>>
>>
>>
>
>Hello Kev,
>
>if you want to do some optimization work for copy,
>can you have a look at this bug report :
>http://issues.apache.org/bugzilla/show_bug.cgi?id=30094
>
>Using nio for copy operations to network drives under Windows brings a
>huge performance improvement.
>I used it on Windows 2000 or 2003 and JDK 1.4, with as target some
>drives automounted from a NAS box.
>
>With NIO the copy speed is at least 4 times as big.
>
>
Last time I used NIO was just when 1.4 came out, and while the speed was
impressive (over 1.3), the memory usage was very large (for file mapped
stuff). At the time I was processing large files, so I didn't get to
play with it as it made our server (Sun something) cry :(.
I've knocked up a quick implementation of what is suggested in the bug
report (and Steve L's suggested strategy, at least my understanding of
said strategy). Now I have a FileUtilsAdapterFactory, a
FileUtilsAdapter and 3 implementations FileUtils - for bwc, NioFileUtils
and Java6FileUtils. So far all is well - indeed during bootstrap, Ant
handily picks up the new NioFileUtils to perform it's copies - yay!
I'll post full code for comments/review later, but to implement this
across the whole code base will take a while - I'm not sure everyone
will think it's worthwhile.
Kev
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org
Re: AW: Adding a methof to StringUtils
Posted by Antoine Levy-Lambert <an...@gmx.de>.
Matt Benson wrote:
>
> Kev: I've asked some stuff about FileUtils before...
> historically it was thought it might be pluggable,
> hence all the instance methods. But that never
> happened so the static methods crept in. I think it
> was Martijn who really pushed in the FileUtils
> singleton. Most places in Ant's code don't
> instantiate their own FileUtils anymore, and if they
> do they shouldn't; they should use the singleton.
>
> -Matt
>
Hello Kev,
if you want to do some optimization work for copy,
can you have a look at this bug report :
http://issues.apache.org/bugzilla/show_bug.cgi?id=30094
Using nio for copy operations to network drives under Windows brings a
huge performance improvement.
I used it on Windows 2000 or 2003 and JDK 1.4, with as target some
drives automounted from a NAS box.
With NIO the copy speed is at least 4 times as big.
This type of progress dwarfs logging optimization.
Of course, this is mainly to be used (or only) in the case when one
copies 100% of the content of a file to another location,
no filtering takes place.
Regards,
Antoine
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org
Re: AW: Adding a methof to StringUtils
Posted by Matt Benson <gu...@yahoo.com>.
--- Kev Jackson <ke...@it.fts-vn.com> wrote:
> Dominique Devienne wrote:
>
> >>The point about building the strings when they
> aren't used (because
> >>logging verbosity is set too low) still stands
> though - this is less
> >>than efficient
> >>
> >>
> >
> >The point, which Matt already raised, is that you
> can't know the level
> >at which the logger and the listeners are set.
> There is nothing ATM in
> >the Ant codebase to tip off the project as to the
> min log level of all
> >logger+listener to implement your optimization.
> This could be can be
> >added though, and would be useful IMHO, but that's
> a separate thread.
> >
> >
> >
> I know, that was why I didn't want to wrap the log
> code in if(verbosity
> >= Project.MSG_INFO) conditionals - it's sort of a
> catch-22 situation,
> you don't want to log the message (or even bother
> building it) if it
> won't be used, but that can't be determined, so you
> must build the
> message just in case - I agree a mechanism for you
> to retrieve the
> minimum logging level of the entire project would be
> useful in this
> situation
>
> >The StringBuffer optimization is well known. I've
> either read about it
> >in Effective Java or the "K&R" of Java,
> The.Java.Prog.Lang.
> >
> >
> I think it was in Effective Java, but as Alexey
> showed previously,
> Java1.4+ actually perform this operation for you,
> letting you be as lazy
> as you like with String concatenation (now that's
> progress!). I have to
> stop beating up the junior devs here whenever I see
> sqlString = "Select
> " + field + " from " + table + " where " + field + "
> = value"; as we are
> on 1.4 internally so it doesn't degrade performance
> anymore. Still I'd
> prefer to be able to do something like "yada yada
> ${var1} yada yada
> ${var2}" like the property expansion that is already
> in Ant - I
> personally think it's much cleaner than "yada yada "
> + var1 + "yada
> yada" + var2., even the Java5 printf isn't as nice
> as it could be for
> constructing messages (although it's a long way
> forward).
>
> >I think what you propose to do would clutter the
> code, and make it
> >ugly frankly ;-) I'd probably -1 it unless you can
> show hard evidence
> >of it's usefulness.
> >
> >
> If it came to it I'd -1 it too! I don't like any of
> the solutions I
> could come up with yesterday, the one I showed was
> the 'least worst'
> that I could think of, with a semi-upgrade path to
> Java5 style varargs
> (use an object array). I was mainly throwing the
> idea out to see what
> peoples reactions were - overall I don't think it's
> the right way to
> solve the problem - the real problem is that the
> level of logging cannot
> currently be determined and so any optimization (for
> memory or
> performance) is actually changing the behaviour of
> the code - which it
> shouldn't do.
>
> The delete task was just an example - I was looking
> at it to fix 'delete
> task won't be quiet' bug in bugzilla, and I was also
> thinking about the
> problem with AppFuse, (which does use Delete a
> little, but not as much
> as Copy and other tasks), so it seemed like a handy
> guinea pig as I had
> the code open at the time.
>
> I was also looking at Copy and saw that
> ResourceUtils.copyResource is a
> static method, but FileUtils.copyFile is not even
> though it delegates to
> copyResource, this means that in Copy there must be
> an instantiated
> fileUtils object, just to perform the copy,
> unfortunately the FileUtils
> interface/API is public so changing it would break
> bwc, but I'd like to
> add a static method for copyFile so that Copy wont
> need to instantiate
> FileUtils.
Kev: I've asked some stuff about FileUtils before...
historically it was thought it might be pluggable,
hence all the instance methods. But that never
happened so the static methods crept in. I think it
was Martijn who really pushed in the FileUtils
singleton. Most places in Ant's code don't
instantiate their own FileUtils anymore, and if they
do they shouldn't; they should use the singleton.
-Matt
>
> Thanks everyone for feedback
> Kev
>
>
---------------------------------------------------------------------
> To unsubscribe, e-mail:
> dev-unsubscribe@ant.apache.org
> For additional commands, e-mail:
> dev-help@ant.apache.org
>
>
__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org
Re: AW: Adding a methof to StringUtils
Posted by Kev Jackson <ke...@it.fts-vn.com>.
Dominique Devienne wrote:
>>The point about building the strings when they aren't used (because
>>logging verbosity is set too low) still stands though - this is less
>>than efficient
>>
>>
>
>The point, which Matt already raised, is that you can't know the level
>at which the logger and the listeners are set. There is nothing ATM in
>the Ant codebase to tip off the project as to the min log level of all
>logger+listener to implement your optimization. This could be can be
>added though, and would be useful IMHO, but that's a separate thread.
>
>
>
I know, that was why I didn't want to wrap the log code in if(verbosity
>= Project.MSG_INFO) conditionals - it's sort of a catch-22 situation,
you don't want to log the message (or even bother building it) if it
won't be used, but that can't be determined, so you must build the
message just in case - I agree a mechanism for you to retrieve the
minimum logging level of the entire project would be useful in this
situation
>The StringBuffer optimization is well known. I've either read about it
>in Effective Java or the "K&R" of Java, The.Java.Prog.Lang.
>
>
I think it was in Effective Java, but as Alexey showed previously,
Java1.4+ actually perform this operation for you, letting you be as lazy
as you like with String concatenation (now that's progress!). I have to
stop beating up the junior devs here whenever I see sqlString = "Select
" + field + " from " + table + " where " + field + " = value"; as we are
on 1.4 internally so it doesn't degrade performance anymore. Still I'd
prefer to be able to do something like "yada yada ${var1} yada yada
${var2}" like the property expansion that is already in Ant - I
personally think it's much cleaner than "yada yada " + var1 + "yada
yada" + var2., even the Java5 printf isn't as nice as it could be for
constructing messages (although it's a long way forward).
>I think what you propose to do would clutter the code, and make it
>ugly frankly ;-) I'd probably -1 it unless you can show hard evidence
>of it's usefulness.
>
>
If it came to it I'd -1 it too! I don't like any of the solutions I
could come up with yesterday, the one I showed was the 'least worst'
that I could think of, with a semi-upgrade path to Java5 style varargs
(use an object array). I was mainly throwing the idea out to see what
peoples reactions were - overall I don't think it's the right way to
solve the problem - the real problem is that the level of logging cannot
currently be determined and so any optimization (for memory or
performance) is actually changing the behaviour of the code - which it
shouldn't do.
The delete task was just an example - I was looking at it to fix 'delete
task won't be quiet' bug in bugzilla, and I was also thinking about the
problem with AppFuse, (which does use Delete a little, but not as much
as Copy and other tasks), so it seemed like a handy guinea pig as I had
the code open at the time.
I was also looking at Copy and saw that ResourceUtils.copyResource is a
static method, but FileUtils.copyFile is not even though it delegates to
copyResource, this means that in Copy there must be an instantiated
fileUtils object, just to perform the copy, unfortunately the FileUtils
interface/API is public so changing it would break bwc, but I'd like to
add a static method for copyFile so that Copy wont need to instantiate
FileUtils.
Thanks everyone for feedback
Kev
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org
Re: AW: Adding a methof to StringUtils
Posted by Dominique Devienne <dd...@gmail.com>.
> The point about building the strings when they aren't used (because
> logging verbosity is set too low) still stands though - this is less
> than efficient
The point, which Matt already raised, is that you can't know the level
at which the logger and the listeners are set. There is nothing ATM in
the Ant codebase to tip off the project as to the min log level of all
logger+listener to implement your optimization. This could be can be
added though, and would be useful IMHO, but that's a separate thread.
The StringBuffer optimization is well known. I've either read about it
in Effective Java or the "K&R" of Java, The.Java.Prog.Lang.
I think what you propose to do would clutter the code, and make it
ugly frankly ;-) I'd probably -1 it unless you can show hard evidence
of it's usefulness.
If <delete> is causing problem, replacing it by a custom task which is
streamlined and doesn't need to buffer all files to delete beforehand
for example. --DD
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org
Re: AW: Adding a methof to StringUtils
Posted by Kev Jackson <ke...@it.fts-vn.com>.
>javap -c AAA
>
>public java.lang.String concat(java.lang.String, java.lang.String, java.lang.String);
> Code:
> 0: new #2; //class java/lang/StringBuilder
> 3: dup
> 4: invokespecial #3; //Method java/lang/StringBuilder."<init>":()V
> 7: aload_1
> 8: invokevirtual #4; //Method java/lang/StringBuilder.append:(Ljava/lang/String;)Ljava/lang/StringBuilder;
> 11: aload_2
> 12: invokevirtual #4; //Method java/lang/StringBuilder.append:(Ljava/lang/String;)Ljava/lang/StringBuilder;
> 15: aload_3
> 16: invokevirtual #4; //Method java/lang/StringBuilder.append:(Ljava/lang/String;)Ljava/lang/StringBuilder;
> 19: invokevirtual #5; //Method java/lang/StringBuilder.toString:()Ljava/lang/String;
> 22: areturn
>
>
>
[snip 1.4 output]
Thanks for showing me this, it's time I re-read the JVM spec I suppose
as it's obviously been updated!
It is strange though that by making changes to an explicit StringBuffer
instead of using String concatenation, the memory usage of the AppFuse
build was reduced enough to avoid an OutOfMemory error.
The point about building the strings when they aren't used (because
logging verbosity is set too low) still stands though - this is less
than efficient
Thanks
Kev
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org