You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@harmony.apache.org by Mark Hindess <ma...@googlemail.com> on 2006/06/02 07:30:57 UTC

Re: [classlib] [testing] Coverage (was Re: 37% of total test execution time is spent in a single test)

On 2 June 2006 at 10:37, Paulex Yang <pa...@gmail.com> wrote:
>
> Mark,
> 
> I'm glad that there is someone else has interest on emma, I've tried it 
> before. AFAIK, emma works by instrumentation, but sometimes for classes 
> in bootclasspath, the instrumentation cannot work, there are two cases:
> 1. Some instrumented classes cannot be loaded by VM.
> 2. Some classes cannot be instrumented
> 
> I have tried to look more inside to find some way to work around but I 
> haven't got enough time yet.
> 
> Specifically for nio-channel module, I had a list for these two cases (I 
> believe the data is a little outdated and should be reevaluated)
> case 1.
> BaseByteBuffer.class
> Buffer.class
> BufferFactory.class
> ByteBuffer.class
> CharArrayBuffer.class
> CharBuffer.class
> HeapByteBuffer.class
> ReadWriteCharArrayBuffer.class
> ReadWriteHeapByteBuffer.class
> FileChannel.class
> AbstractInterruptibleChannel.class
> FileChannelImpl.class
> WriteOnlyFileChannel.class
> LockManager.class
> LockManager$1.class
> ReadOnlyFileChannel.class
> 
> case 2:
> ByteChannel.class
> Channel.class
> GatheringByteChannel.class
> InterruptibleChannel.class
> WritableByteChannel.class
> 
> And I have got some ant script and more excluded list for emma, if 
> anyone has interests, I can upload it to JIRA.

Yes!

-Mark.

> Mark Hindess wrote:
> > Anyone tried using emma (emma.sf.net) to look at test coverage for our
> > tests?  That might help ensure we don't trim them down too much?
> >
> > I suspect we will soon reach a point where there are significantly fewer
> > obvious opportunities for small contributions - e.g. 1.5 generification,
> > etc from looking at JAPI reports.  Having test coverage reports might
> > help by highlighting areas for improvement.
> >
> > I had a quick try with:
> >
> >   ant -f make/build.xml
> >   ant -f modules/nio/make/build.xml test
> >   java -cp emma.jar emma instr -m overwrite -cp deploy/jdk/jre/lib/boot/nio
> .jar
> >
> > and then:
> >
> >   java -Xbootclasspath/p:emma.jar \
> >        -cp build/tests:modules/nio/bin/test:junit.jar:. \
> >        junit.textui.TestRunner \
> >        org.apache.harmony.tests.java.nio.channels.SocketChannelTest
> >
> > (Running all the tests is a little tricky because test depends on build, 
> > which depends on clean which wipes out the instr'd jar. ;-( )
> >
> > Sadly it dies horribly with the stack trace below.  I'll take another
> > look when I have a little more time, but I thought I'd mention it in
> > case anyone else had any thoughts on the subject.
> >
> > Regards,
> >  
> > Mark.
> >
> > Exception in thread "main" java/lang/ExceptionInInitializerError
> >         at java/lang/J9VMInternals.initialize (J9VMInternals.java:203)
> >         at java/nio/Buffer.$VRi (Buffer.java:4294967295)
> >         at java/nio/Buffer.<clinit> (Buffer.java:4294967295)
> >         at java/lang/J9VMInternals.initializeImpl (Native Method)
> >         at java/lang/J9VMInternals.initialize (J9VMInternals.java:185)
> >         at java/lang/J9VMInternals.initialize (J9VMInternals.java:154)
> >         at java/lang/String.getBytes (String.java:783)
> >         at org/apache/harmony/luni/util/Util.getBytes (Util.java:54)
> >         at java/lang/ClassLoader.loadLibraryWithPath (ClassLoader.java:998)
> >         at java/lang/ClassLoader.loadLibraryWithClassLoader (ClassLoader.ja
> va:970)
> >         at java/lang/System.loadLibrary (System.java:706)
> >         at com/ibm/icu4jni/common/ICU4JNILoader.loadLibrary (Unknown Source
> )
> >         at com/ibm/icu4jni/converters/NativeConverter.<clinit> (Unknown Sou
> rce)
> >         at java/lang/J9VMInternals.initializeImpl (Native Method)
> >         at java/lang/J9VMInternals.initialize (J9VMInternals.java:185)
> >         at com/ibm/icu4jni/charset/CharsetProviderICU.charsetForName (Unkno
> wn Source)
> >         at java/nio/charset/Charset.forNameInternal (Charset.java:450)
> >         at java/nio/charset/Charset.forName (Charset.java:513)
> >         at java/lang/String.defaultCharset (String.java:688)
> >         at java/lang/String.<init> (String.java:252)
> >         at org/apache/harmony/luni/util/Util.toString (Util.java:74)
> >         at java/lang/System.getPropertyList (Native Method)
> >         at java/lang/System.ensureProperties (System.java:496)
> >         at java/lang/System.<clinit> (System.java:55)
> >         at java/lang/J9VMInternals.initializeImpl (Native Method)
> >         at java/lang/J9VMInternals.initialize (J9VMInternals.java:185)
> >         at java/lang/ClassLoader.initializeClassLoaders (ClassLoader.java:6
> 5)
> >         at java/lang/Thread.initialize (Thread.java:301)
> >         at java/lang/Thread.<init> (Thread.java:120)
> > java/lang/NullPointerException
> >         at java/lang/Throwable.printStackTrace (Throwable.java:231)
> >         at com/vladium/emma/rt/RTExitHook.createClassLoaderClosure (RTExitH
> ook.java:63)
> >         at com/vladium/emma/rt/RT.reset (RT.java:92)
> >         at com/vladium/emma/rt/RT.<clinit> (RT.java:238)
> >         at java/lang/J9VMInternals.initializeImpl (Native Method)
> >         at java/lang/J9VMInternals.initialize (J9VMInternals.java:185)
> >         at java/nio/Buffer.$VRi (Buffer.java:4294967295)
> >         at java/nio/Buffer.<clinit> (Buffer.java:4294967295)
> >         at java/lang/J9VMInternals.initializeImpl (Native Method)
> >         at java/lang/J9VMInternals.initialize (J9VMInternals.java:185)
> >         at java/lang/J9VMInternals.initialize (J9VMInternals.java:154)
> >         at java/lang/String.getBytes (String.java:783)
> >         at org/apache/harmony/luni/util/Util.getBytes (Util.java:54)
> >         at java/lang/ClassLoader.loadLibraryWithPath (ClassLoader.java:998)
> >         at java/lang/ClassLoader.loadLibraryWithClassLoader (ClassLoader.ja
> va:970)
> >         at java/lang/System.loadLibrary (System.java:706)
> >         at com/ibm/icu4jni/common/ICU4JNILoader.loadLibrary (Unknown Source
> )
> >         at com/ibm/icu4jni/converters/NativeConverter.<clinit> (Unknown Sou
> rce)
> >         at java/lang/J9VMInternals.initializeImpl (Native Method)
> >         at java/lang/J9VMInternals.initialize (J9VMInternals.java:185)
> >         at com/ibm/icu4jni/charset/CharsetProviderICU.charsetForName (Unkno
> wn Source)
> >         at java/nio/charset/Charset.forNameInternal (Charset.java:450)
> >         at java/nio/charset/Charset.forName (Charset.java:513)
> >         at java/lang/String.defaultCharset (String.java:688)
> >         at java/lang/String.<init> (String.java:252)
> >         at org/apache/harmony/luni/util/Util.toString (Util.java:74)
> >         at java/lang/System.getPropertyList (Native Method)
> >         at java/lang/System.ensureProperties (System.java:496)
> >         at java/lang/System.<clinit> (System.java:55)
> >         at java/lang/J9VMInternals.initializeImpl (Native Method)
> >         at java/lang/J9VMInternals.initialize (J9VMInternals.java:185)
> >         at java/lang/ClassLoader.initializeClassLoaders (ClassLoader.java:6
> 5)
> >         at java/lang/Thread.initialize (Thread.java:301)
> >         at java/lang/Thread.<init> (Thread.java:120)
> > JVMJ9VM015W Initialization error for library jclclear_23(14): JVMJ9VM009E J
> 9VMDllMain failed
> > HMYEXEL062E Internal VM error: Failed to create Java VM
> > FAILED.
> >
> >
> > On 1 June 2006 at 22:26, "Mikhail Loenko" <ml...@gmail.com> wrote:
> >   
> >> 2006/6/1, Geir Magnusson Jr <ge...@pobox.com>:
> >>     
> >>> Mikhail Loenko wrote:
> >>>       
> >>>> 2006/6/1, Geir Magnusson Jr <ge...@pobox.com>:
> >>>>         
> >>>>> I don't quite grok what you mean - could you post what you are
> >>>>> suggesting in code here on -dev?
> >>>>>           
> >>>> Index: modules/luni/src/test/java/tests/api/java/net/Inet6AddressTest.ja
> v
> >>>>         
> >> a
> >>     
> >>>> ===================================================================
> >>>> ---
> >>>> modules/luni/src/test/java/tests/api/java/net/Inet6AddressTest.java
> >>>> (revision
> >>>> 410480)
> >>>> +++
> >>>> modules/luni/src/test/java/tests/api/java/net/Inet6AddressTest.java
> >>>> (working
> >>>> copy)
> >>>> @@ -785,7 +785,7 @@
> >>>>                 "FFFF:FFFF:FFFF:FFFF:FFFF:FFFF:255.255.255.255",
> >>>>                 "0:0:0:0:0:0:0:0", "0:0:0:0:0:0:0.0.0.0" };
> >>>>
> >>>> -        String invalidIPAddresses[] = { ":", "FFFF:FFFF", "1:1",
> >>>> "::1.2.3.444" };
> >>>> +        String invalidIPAddresses[] = { "FFFF:FFFF" };
> >>>>
> >>>>         for (int i = 0; i < validIPAddresses.length; i++) {
> >>>>
> >>>>
> >>>>         
> >>> Thx - why do you want to remove testing the others?  Because you believe
> >>> that it is in effect testing the resolver in the OS since each returns
> >>> the same answer,and thus it's testing the same thing over and over?
> >>>       
> >> I believe so, but I'd like to hear confirmation from code authors
> >>
> >> Thanks,
> >> Mikhail
> >>     
> >
> >
> >
> > ---------------------------------------------------------------------
> > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >
> >
> >   
> 
> 
> -- 
> Paulex Yang
> China Software Development Lab
> IBM
> 
> 
> 
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org



---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org


Re: [classlib] [testing] Coverage (was Re: 37% of total test execution time is spent in a single test)

Posted by Paulex Yang <pa...@gmail.com>.
Mikhail Loenko wrote:
> 2006/6/8, Paulex Yang <pa...@gmail.com>:
>> Vladimir,
>>
>> Vladimir Ivanov wrote:
>> > Thanks Paulex!
>> >
>> > I did the same, but could not send results due to spam filter J
>> > Observations:
>> >
>> >   1. Coverage results look pretty much similar.
>> >   2. Exclude list looks pretty much similar too, but, looks like it
>> >   depends on the way of data collection (I didn't run ant task and the
>> > list is
>> >   a little bit different).
>> Great.
>> >
>> > In any case, I think, when we run harmony on another VM exclude 
>> list will
>> > have to be updated.
>> >
>> >
>> >
>> > May be we can start publishing the coverage information on wiki 
>> pages and
>> > provide some updates time to time (I can do it)?
>> >
>> +1,  and of course, you can only if  no one in the mailing list objects
>> and you'll have my welcome, and I think it will be even greater if these
>> reports can be generated regularly like what JAPI is doing:)
>
> Does it make sense to create corresponding build target?
Sure, I think so.
>
> Thanks,
> Mikhail
>
>> >
>> >
>> > One note:
>> >
>> > I noticed that different unit tests have very different package names
>> >
>> > Now the directory with all built tests copied to one place looks like:
>> I think we have got a agreement on the test naming convention[1], but
>> for sure, there have been many (legacy) test cases before this
>> agreement, and I think the volunteer is highly welcome to provide patch
>> for them.
>>
>> [1]http://incubator.apache.org/harmony/subcomponents/classlibrary/testing.html 
>>
>> >
>> >
>> >
>> > C:\coverage\tests\test>ls
>> >
>> > GZIPOutClose2.txt        api           config         javax
>> > tests
>> >
>> > GZIPOutFinish.txt        api.injected  dazzle         
>> org            xml
>> >
>> > GZIPOutWrite.txt         binary        gif            prefs
>> >
>> > Inet6Address.golden.ser  bundles       impl           serialization
>> >
>> > JDK2-3gabba.zip          com           impl.injected  test.txt
>> >
>> >
>> >
>> > I think, it would be good if tests had unified package names.
>> >
>> > Why? – so far, just common sense, just to have an order in test suite
>> >
>> > Organization (if consider all unit tests as solid test suite).
>> >
>> > Thanks,
>> >  Vladimir
>> >
>> > For example, my exclude list for java.io is:
>> > -java.io.BufferedInputStream,
>> > -java.io.BufferedOutputStream,
>> > -java.io.File,
>> > -java.io.FileChannelFactory,
>> > -java.io.FileDescriptor,
>> > -java.io.FileInputStream,
>> > -java.io.FileOutputStream,
>> > -java.io.FilterInputStream ,
>> > -java.io.FilterOutputStream,
>> > -java.io.InputStream,
>> > -java.io.OutputStream,
>> > -java.io.ObjectStreamField,
>> > -java.io.PrintStream
>> >
>> >
>> > On 6/6/06, Paulex Yang <pa...@gmail.com> wrote:
>> >>
>> >> I've attach the scripts and excluded class lists to JIRA,  please 
>> refer
>> >> to https://issues.apache.org/jira/browse/HARMONY-564.  Enjoy it:).
>> >>
>> >> Mark Hindess wrote:
>> >> > On 2 June 2006 at 10:37, Paulex Yang <pa...@gmail.com> wrote:
>> >> >
>> >> >> Mark,
>> >> >>
>> >> >> I'm glad that there is someone else has interest on emma, I've
>> >> tried it
>> >> >> before. AFAIK, emma works by instrumentation, but sometimes for
>> >> classes
>> >> >> in bootclasspath, the instrumentation cannot work, there are two
>> >> cases:
>> >>
>> >> >> 1. Some instrumented classes cannot be loaded by VM.
>> >> >> 2. Some classes cannot be instrumented
>> >> >>
>> >> >> I have tried to look more inside to find some way to work around
>> >> but I
>> >> >> haven't got enough time yet.
>> >> >>
>> >> >> Specifically for nio-channel module, I had a list for these two 
>> cases
>> >> (I
>> >> >> believe the data is a little outdated and should be reevaluated)
>> >> >> case 1.
>> >> >> BaseByteBuffer.class
>> >> >> Buffer.class
>> >> >> BufferFactory.class
>> >> >> ByteBuffer.class
>> >> >> CharArrayBuffer.class
>> >> >> CharBuffer.class
>> >> >> HeapByteBuffer.class
>> >> >> ReadWriteCharArrayBuffer.class
>> >> >> ReadWriteHeapByteBuffer.class
>> >> >> FileChannel.class
>> >> >> AbstractInterruptibleChannel.class
>> >> >> FileChannelImpl.class
>> >> >> WriteOnlyFileChannel.class
>> >> >> LockManager.class
>> >> >> LockManager$1.class
>> >> >> ReadOnlyFileChannel.class
>> >> >>
>> >> >> case 2:
>> >> >> ByteChannel.class
>> >> >> Channel.class
>> >> >> GatheringByteChannel.class
>> >> >> InterruptibleChannel.class
>> >> >> WritableByteChannel.class
>> >> >>
>> >> >> And I have got some ant script and more excluded list for emma, if
>> >> >> anyone has interests, I can upload it to JIRA.
>> >> >>
>> >> >
>> >> > Yes!
>> >> >
>> >> > -Mark.
>> >> >
>> >> >
>> >> >> Mark Hindess wrote:
>> >> >>
>> >> >>> Anyone tried using emma (emma.sf.net) to look at test coverage
>> >> for our
>> >> >>> tests?  That might help ensure we don't trim them down too much?
>> >> >>>
>> >> >>> I suspect we will soon reach a point where there are 
>> significantly
>> >> fewer
>> >> >>> obvious opportunities for small contributions - e.g.
>> >> 1.5generification,
>> >> >>> etc from looking at JAPI reports.  Having test coverage reports
>> >> might
>> >> >>> help by highlighting areas for improvement.
>> >> >>>
>> >> >>> I had a quick try with:
>> >> >>>
>> >> >>>   ant -f make/build.xml
>> >> >>>   ant -f modules/nio/make/build.xml test
>> >> >>>   java -cp emma.jar emma instr -m overwrite -cp
>> >> deploy/jdk/jre/lib/boot/nio
>> >> >>>
>> >> >> .jar
>> >> >>
>> >> >>> and then:
>> >> >>>
>> >> >>>   java -Xbootclasspath/p: emma.jar \
>> >> >>>        -cp build/tests:modules/nio/bin/test:junit.jar:. \
>> >> >>>        junit.textui.TestRunner \
>> >> >>>        
>> org.apache.harmony.tests.java.nio.channels.SocketChannelTest
>> >> >>>
>> >> >>> (Running all the tests is a little tricky because test depends on
>> >> build,
>> >> >>> which depends on clean which wipes out the instr'd jar. ;-( )
>> >> >>>
>> >> >>> Sadly it dies horribly with the stack trace below.  I'll take
>> >> another
>> >> >>> look when I have a little more time, but I thought I'd mention 
>> it in
>> >> >>> case anyone else had any thoughts on the subject.
>> >> >>>
>> >> >>> Regards,
>> >> >>>
>> >> >>> Mark.
>> >> >>>
>> >> >>> Exception in thread "main" java/lang/ExceptionInInitializerError
>> >> >>>         at java/lang/J9VMInternals.initialize
>> >> (J9VMInternals.java:203)
>> >> >>>         at java/nio/Buffer.$VRi ( Buffer.java:4294967295)
>> >> >>>         at java/nio/Buffer.<clinit> (Buffer.java:4294967295)
>> >> >>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
>> >> >>>         at java/lang/J9VMInternals.initialize ( 
>> J9VMInternals.java
>> >> :185)
>> >> >>>         at java/lang/J9VMInternals.initialize
>> >> (J9VMInternals.java:154)
>> >> >>>         at java/lang/String.getBytes (String.java:783)
>> >> >>>         at org/apache/harmony/luni/util/Util.getBytes (
>> >> Util.java:54)
>> >> >>>         at java/lang/ClassLoader.loadLibraryWithPath
>> >> (ClassLoader.java
>> >> :998)
>> >> >>>         at java/lang/ClassLoader.loadLibraryWithClassLoader (
>> >> ClassLoader.ja
>> >> >>>
>> >> >> va:970)
>> >> >>
>> >> >>>         at java/lang/System.loadLibrary (System.java:706)
>> >> >>>         at com/ibm/icu4jni/common/ICU4JNILoader.loadLibrary 
>> (Unknown
>> >> Source
>> >> >>>
>> >> >> )
>> >> >>
>> >> >>>         at com/ibm/icu4jni/converters/NativeConverter.<clinit>
>> >> (Unknown Sou
>> >> >>>
>> >> >> rce)
>> >> >>
>> >> >>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
>> >> >>>         at java/lang/J9VMInternals.initialize
>> >> (J9VMInternals.java:185)
>> >> >>>         at 
>> com/ibm/icu4jni/charset/CharsetProviderICU.charsetForName
>> >> (Unkno
>> >> >>>
>> >> >> wn Source)
>> >> >>
>> >> >>>         at java/nio/charset/Charset.forNameInternal
>> >> (Charset.java:450)
>> >> >>>         at java/nio/charset/Charset.forName (Charset.java:513)
>> >> >>>         at java/lang/String.defaultCharset ( String.java:688)
>> >> >>>         at java/lang/String.<init> (String.java:252)
>> >> >>>         at org/apache/harmony/luni/util/Util.toString 
>> (Util.java:74)
>> >> >>>         at java/lang/System.getPropertyList (Native Method)
>> >> >>>         at java/lang/System.ensureProperties (System.java:496)
>> >> >>>         at java/lang/System.<clinit> (System.java:55)
>> >> >>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
>> >> >>>         at java/lang/J9VMInternals.initialize
>> >> (J9VMInternals.java:185)
>> >> >>>         at java/lang/ClassLoader.initializeClassLoaders (
>> >> ClassLoader.java:6
>> >> >>>
>> >> >> 5)
>> >> >>
>> >> >>>         at java/lang/Thread.initialize (Thread.java:301)
>> >> >>>         at java/lang/Thread.<init> (Thread.java:120)
>> >> >>> java/lang/NullPointerException
>> >> >>>         at java/lang/Throwable.printStackTrace ( 
>> Throwable.java:231)
>> >> >>>         at 
>> com/vladium/emma/rt/RTExitHook.createClassLoaderClosure
>> >> (RTExitH
>> >> >>>
>> >> >> ook.java:63)
>> >> >>
>> >> >>>         at com/vladium/emma/rt/RT.reset ( RT.java:92)
>> >> >>>         at com/vladium/emma/rt/RT.<clinit> (RT.java:238)
>> >> >>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
>> >> >>>         at java/lang/J9VMInternals.initialize ( 
>> J9VMInternals.java
>> >> :185)
>> >> >>>         at java/nio/Buffer.$VRi (Buffer.java:4294967295)
>> >> >>>         at java/nio/Buffer.<clinit> (Buffer.java:4294967295)
>> >> >>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
>> >> >>>         at java/lang/J9VMInternals.initialize
>> >> (J9VMInternals.java:185)
>> >> >>>         at java/lang/J9VMInternals.initialize
>> >> (J9VMInternals.java:154)
>> >> >>>         at java/lang/String.getBytes ( String.java:783)
>> >> >>>         at org/apache/harmony/luni/util/Util.getBytes 
>> (Util.java:54)
>> >> >>>         at java/lang/ClassLoader.loadLibraryWithPath
>> >> (ClassLoader.java
>> >> :998)
>> >> >>>         at java/lang/ClassLoader.loadLibraryWithClassLoader (
>> >> ClassLoader.ja
>> >> >>>
>> >> >> va:970)
>> >> >>
>> >> >>>         at java/lang/System.loadLibrary (System.java:706)
>> >> >>>         at com/ibm/icu4jni/common/ICU4JNILoader.loadLibrary 
>> (Unknown
>> >> Source
>> >> >>>
>> >> >> )
>> >> >>
>> >> >>>         at com/ibm/icu4jni/converters/NativeConverter.<clinit>
>> >> (Unknown Sou
>> >> >>>
>> >> >> rce)
>> >> >>
>> >> >>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
>> >> >>>         at java/lang/J9VMInternals.initialize
>> >> (J9VMInternals.java:185)
>> >> >>>         at 
>> com/ibm/icu4jni/charset/CharsetProviderICU.charsetForName
>> >> (Unkno
>> >> >>>
>> >> >> wn Source)
>> >> >>
>> >> >>>         at java/nio/charset/Charset.forNameInternal
>> >> (Charset.java:450)
>> >> >>>         at java/nio/charset/Charset.forName (Charset.java:513)
>> >> >>>         at java/lang/String.defaultCharset ( String.java:688)
>> >> >>>         at java/lang/String.<init> (String.java:252)
>> >> >>>         at org/apache/harmony/luni/util/Util.toString 
>> (Util.java:74)
>> >> >>>         at java/lang/System.getPropertyList (Native Method)
>> >> >>>         at java/lang/System.ensureProperties (System.java:496)
>> >> >>>         at java/lang/System.<clinit> (System.java:55)
>> >> >>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
>> >> >>>         at java/lang/J9VMInternals.initialize
>> >> (J9VMInternals.java:185)
>> >> >>>         at java/lang/ClassLoader.initializeClassLoaders (
>> >> ClassLoader.java:6
>> >> >>>
>> >> >> 5)
>> >> >>
>> >> >>>         at java/lang/Thread.initialize (Thread.java:301)
>> >> >>>         at java/lang/Thread.<init> (Thread.java:120)
>> >> >>> JVMJ9VM015W Initialization error for library jclclear_23(14):
>> >> JVMJ9VM009E J
>> >> >>>
>> >> >> 9VMDllMain failed
>> >> >>
>> >> >>> HMYEXEL062E Internal VM error: Failed to create Java VM
>> >> >>> FAILED.
>> >> >>>
>> >> >>>
>> >> >>> On 1 June 2006 at 22:26, "Mikhail Loenko" < mloenko@gmail.com>
>> >> wrote:
>> >> >>>
>> >> >>>
>> >> >>>> 2006/6/1, Geir Magnusson Jr <ge...@pobox.com>:
>> >> >>>>
>> >> >>>>
>> >> >>>>> Mikhail Loenko wrote:
>> >> >>>>>
>> >> >>>>>
>> >> >>>>>> 2006/6/1, Geir Magnusson Jr <geir@pobox.com >:
>> >> >>>>>>
>> >> >>>>>>
>> >> >>>>>>> I don't quite grok what you mean - could you post what you 
>> are
>> >> >>>>>>> suggesting in code here on -dev?
>> >> >>>>>>>
>> >> >>>>>>>
>> >> >>>>>> Index:
>> >> modules/luni/src/test/java/tests/api/java/net/Inet6AddressTest.ja
>> >> >>>>>>
>> >> >> v
>> >> >>
>> >> >>>>>>
>> >> >>>>>>
>> >> >>>> a
>> >> >>>>
>> >> >>>>
>> >> >>>>>>
>> >> ===================================================================
>> >> >>>>>> ---
>> >> >>>>>>
>> >> modules/luni/src/test/java/tests/api/java/net/Inet6AddressTest.java
>> >> >>>>>> (revision
>> >> >>>>>> 410480)
>> >> >>>>>> +++
>> >> >>>>>>
>> >> modules/luni/src/test/java/tests/api/java/net/Inet6AddressTest.java
>> >>
>> >> >>>>>> (working
>> >> >>>>>> copy)
>> >> >>>>>> @@ -785,7 +785,7 @@
>> >> >>>>>>                 "FFFF:FFFF:FFFF:FFFF:FFFF:FFFF: 
>> 255.255.255.255",
>> >> >>>>>>                 "0:0:0:0:0:0:0:0", "0:0:0:0:0:0:0.0.0.0" };
>> >> >>>>>>
>> >> >>>>>> -        String invalidIPAddresses[] = { ":", "FFFF:FFFF", 
>> "1:1",
>> >> >>>>>> "::1.2.3.444" };
>> >> >>>>>> +        String invalidIPAddresses[] = { "FFFF:FFFF" };
>> >> >>>>>>
>> >> >>>>>>         for (int i = 0; i < validIPAddresses.length; i++) {
>> >> >>>>>>
>> >> >>>>>>
>> >> >>>>>>
>> >> >>>>>>
>> >> >>>>> Thx - why do you want to remove testing the others?  Because 
>> you
>> >> believe
>> >> >>>>> that it is in effect testing the resolver in the OS since each
>> >> returns
>> >> >>>>> the same answer,and thus it's testing the same thing over and
>> >> over?
>> >> >>>>>
>> >> >>>>>
>> >> >>>> I believe so, but I'd like to hear confirmation from code 
>> authors
>> >> >>>>
>> >> >>>> Thanks,
>> >> >>>> Mikhail
>> >> >>>>
>> >> >>>>
>> >> >>>
>> >> >>>
>> >> ---------------------------------------------------------------------
>> >> >>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>> >> >>> To unsubscribe, e-mail: 
>> harmony-dev-unsubscribe@incubator.apache.org
>> >> >>> For additional commands, e-mail:
>> >> harmony-dev-help@incubator.apache.org
>> >> >>>
>> >> >>>
>> >> >>>
>> >> >>>
>> >> >> --
>> >> >> Paulex Yang
>> >> >> China Software Development Lab
>> >> >> IBM
>> >> >>
>> >> >>
>> >> >>
>> >> >> 
>> ---------------------------------------------------------------------
>> >> >> Terms of use : http://incubator.apache.org/harmony/mailing.html
>> >> >> To unsubscribe, e-mail: 
>> harmony-dev-unsubscribe@incubator.apache.org
>> >> >> For additional commands, e-mail:
>> >> harmony-dev-help@incubator.apache.org
>> >> >>
>> >> >
>> >> >
>> >> >
>> >> > 
>> ---------------------------------------------------------------------
>> >> > Terms of use : http://incubator.apache.org/harmony/mailing.html
>> >> > To unsubscribe, e-mail: 
>> harmony-dev-unsubscribe@incubator.apache.org
>> >> > For additional commands, e-mail: 
>> harmony-dev-help@incubator.apache.org
>> >> >
>> >> >
>> >> >
>> >>
>> >>
>> >> --
>> >> Paulex Yang
>> >> China Software Development Lab
>> >> IBM
>> >>
>> >>
>> >>
>> >> ---------------------------------------------------------------------
>> >> Terms of use : http://incubator.apache.org/harmony/mailing.html
>> >> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>> >> For additional commands, e-mail: 
>> harmony-dev-help@incubator.apache.org
>> >>
>> >>
>> >
>>
>>
>> -- 
>> Paulex Yang
>> China Software Development Lab
>> IBM
>>
>>
>>
>> ---------------------------------------------------------------------
>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>


-- 
Paulex Yang
China Software Development Lab
IBM



---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org


Re: [classlib] [testing] Coverage (was Re: 37% of total test execution time is spent in a single test)

Posted by Mikhail Loenko <ml...@gmail.com>.
2006/6/8, Paulex Yang <pa...@gmail.com>:
> Vladimir,
>
> Vladimir Ivanov wrote:
> > Thanks Paulex!
> >
> > I did the same, but could not send results due to spam filter J
> > Observations:
> >
> >   1. Coverage results look pretty much similar.
> >   2. Exclude list looks pretty much similar too, but, looks like it
> >   depends on the way of data collection (I didn't run ant task and the
> > list is
> >   a little bit different).
> Great.
> >
> > In any case, I think, when we run harmony on another VM exclude list will
> > have to be updated.
> >
> >
> >
> > May be we can start publishing the coverage information on wiki pages and
> > provide some updates time to time (I can do it)?
> >
> +1,  and of course, you can only if  no one in the mailing list objects
> and you'll have my welcome, and I think it will be even greater if these
> reports can be generated regularly like what JAPI is doing:)

Does it make sense to create corresponding build target?

Thanks,
Mikhail

> >
> >
> > One note:
> >
> > I noticed that different unit tests have very different package names
> >
> > Now the directory with all built tests copied to one place looks like:
> I think we have got a agreement on the test naming convention[1], but
> for sure, there have been many (legacy) test cases before this
> agreement, and I think the volunteer is highly welcome to provide patch
> for them.
>
> [1]http://incubator.apache.org/harmony/subcomponents/classlibrary/testing.html
> >
> >
> >
> > C:\coverage\tests\test>ls
> >
> > GZIPOutClose2.txt        api           config         javax
> > tests
> >
> > GZIPOutFinish.txt        api.injected  dazzle         org            xml
> >
> > GZIPOutWrite.txt         binary        gif            prefs
> >
> > Inet6Address.golden.ser  bundles       impl           serialization
> >
> > JDK2-3gabba.zip          com           impl.injected  test.txt
> >
> >
> >
> > I think, it would be good if tests had unified package names.
> >
> > Why? – so far, just common sense, just to have an order in test suite
> >
> > Organization (if consider all unit tests as solid test suite).
> >
> > Thanks,
> >  Vladimir
> >
> > For example, my exclude list for java.io is:
> > -java.io.BufferedInputStream,
> > -java.io.BufferedOutputStream,
> > -java.io.File,
> > -java.io.FileChannelFactory,
> > -java.io.FileDescriptor,
> > -java.io.FileInputStream,
> > -java.io.FileOutputStream,
> > -java.io.FilterInputStream ,
> > -java.io.FilterOutputStream,
> > -java.io.InputStream,
> > -java.io.OutputStream,
> > -java.io.ObjectStreamField,
> > -java.io.PrintStream
> >
> >
> > On 6/6/06, Paulex Yang <pa...@gmail.com> wrote:
> >>
> >> I've attach the scripts and excluded class lists to JIRA,  please refer
> >> to https://issues.apache.org/jira/browse/HARMONY-564.  Enjoy it:).
> >>
> >> Mark Hindess wrote:
> >> > On 2 June 2006 at 10:37, Paulex Yang <pa...@gmail.com> wrote:
> >> >
> >> >> Mark,
> >> >>
> >> >> I'm glad that there is someone else has interest on emma, I've
> >> tried it
> >> >> before. AFAIK, emma works by instrumentation, but sometimes for
> >> classes
> >> >> in bootclasspath, the instrumentation cannot work, there are two
> >> cases:
> >>
> >> >> 1. Some instrumented classes cannot be loaded by VM.
> >> >> 2. Some classes cannot be instrumented
> >> >>
> >> >> I have tried to look more inside to find some way to work around
> >> but I
> >> >> haven't got enough time yet.
> >> >>
> >> >> Specifically for nio-channel module, I had a list for these two cases
> >> (I
> >> >> believe the data is a little outdated and should be reevaluated)
> >> >> case 1.
> >> >> BaseByteBuffer.class
> >> >> Buffer.class
> >> >> BufferFactory.class
> >> >> ByteBuffer.class
> >> >> CharArrayBuffer.class
> >> >> CharBuffer.class
> >> >> HeapByteBuffer.class
> >> >> ReadWriteCharArrayBuffer.class
> >> >> ReadWriteHeapByteBuffer.class
> >> >> FileChannel.class
> >> >> AbstractInterruptibleChannel.class
> >> >> FileChannelImpl.class
> >> >> WriteOnlyFileChannel.class
> >> >> LockManager.class
> >> >> LockManager$1.class
> >> >> ReadOnlyFileChannel.class
> >> >>
> >> >> case 2:
> >> >> ByteChannel.class
> >> >> Channel.class
> >> >> GatheringByteChannel.class
> >> >> InterruptibleChannel.class
> >> >> WritableByteChannel.class
> >> >>
> >> >> And I have got some ant script and more excluded list for emma, if
> >> >> anyone has interests, I can upload it to JIRA.
> >> >>
> >> >
> >> > Yes!
> >> >
> >> > -Mark.
> >> >
> >> >
> >> >> Mark Hindess wrote:
> >> >>
> >> >>> Anyone tried using emma (emma.sf.net) to look at test coverage
> >> for our
> >> >>> tests?  That might help ensure we don't trim them down too much?
> >> >>>
> >> >>> I suspect we will soon reach a point where there are significantly
> >> fewer
> >> >>> obvious opportunities for small contributions - e.g.
> >> 1.5generification,
> >> >>> etc from looking at JAPI reports.  Having test coverage reports
> >> might
> >> >>> help by highlighting areas for improvement.
> >> >>>
> >> >>> I had a quick try with:
> >> >>>
> >> >>>   ant -f make/build.xml
> >> >>>   ant -f modules/nio/make/build.xml test
> >> >>>   java -cp emma.jar emma instr -m overwrite -cp
> >> deploy/jdk/jre/lib/boot/nio
> >> >>>
> >> >> .jar
> >> >>
> >> >>> and then:
> >> >>>
> >> >>>   java -Xbootclasspath/p: emma.jar \
> >> >>>        -cp build/tests:modules/nio/bin/test:junit.jar:. \
> >> >>>        junit.textui.TestRunner \
> >> >>>        org.apache.harmony.tests.java.nio.channels.SocketChannelTest
> >> >>>
> >> >>> (Running all the tests is a little tricky because test depends on
> >> build,
> >> >>> which depends on clean which wipes out the instr'd jar. ;-( )
> >> >>>
> >> >>> Sadly it dies horribly with the stack trace below.  I'll take
> >> another
> >> >>> look when I have a little more time, but I thought I'd mention it in
> >> >>> case anyone else had any thoughts on the subject.
> >> >>>
> >> >>> Regards,
> >> >>>
> >> >>> Mark.
> >> >>>
> >> >>> Exception in thread "main" java/lang/ExceptionInInitializerError
> >> >>>         at java/lang/J9VMInternals.initialize
> >> (J9VMInternals.java:203)
> >> >>>         at java/nio/Buffer.$VRi ( Buffer.java:4294967295)
> >> >>>         at java/nio/Buffer.<clinit> (Buffer.java:4294967295)
> >> >>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
> >> >>>         at java/lang/J9VMInternals.initialize ( J9VMInternals.java
> >> :185)
> >> >>>         at java/lang/J9VMInternals.initialize
> >> (J9VMInternals.java:154)
> >> >>>         at java/lang/String.getBytes (String.java:783)
> >> >>>         at org/apache/harmony/luni/util/Util.getBytes (
> >> Util.java:54)
> >> >>>         at java/lang/ClassLoader.loadLibraryWithPath
> >> (ClassLoader.java
> >> :998)
> >> >>>         at java/lang/ClassLoader.loadLibraryWithClassLoader (
> >> ClassLoader.ja
> >> >>>
> >> >> va:970)
> >> >>
> >> >>>         at java/lang/System.loadLibrary (System.java:706)
> >> >>>         at com/ibm/icu4jni/common/ICU4JNILoader.loadLibrary (Unknown
> >> Source
> >> >>>
> >> >> )
> >> >>
> >> >>>         at com/ibm/icu4jni/converters/NativeConverter.<clinit>
> >> (Unknown Sou
> >> >>>
> >> >> rce)
> >> >>
> >> >>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
> >> >>>         at java/lang/J9VMInternals.initialize
> >> (J9VMInternals.java:185)
> >> >>>         at com/ibm/icu4jni/charset/CharsetProviderICU.charsetForName
> >> (Unkno
> >> >>>
> >> >> wn Source)
> >> >>
> >> >>>         at java/nio/charset/Charset.forNameInternal
> >> (Charset.java:450)
> >> >>>         at java/nio/charset/Charset.forName (Charset.java:513)
> >> >>>         at java/lang/String.defaultCharset ( String.java:688)
> >> >>>         at java/lang/String.<init> (String.java:252)
> >> >>>         at org/apache/harmony/luni/util/Util.toString (Util.java:74)
> >> >>>         at java/lang/System.getPropertyList (Native Method)
> >> >>>         at java/lang/System.ensureProperties (System.java:496)
> >> >>>         at java/lang/System.<clinit> (System.java:55)
> >> >>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
> >> >>>         at java/lang/J9VMInternals.initialize
> >> (J9VMInternals.java:185)
> >> >>>         at java/lang/ClassLoader.initializeClassLoaders (
> >> ClassLoader.java:6
> >> >>>
> >> >> 5)
> >> >>
> >> >>>         at java/lang/Thread.initialize (Thread.java:301)
> >> >>>         at java/lang/Thread.<init> (Thread.java:120)
> >> >>> java/lang/NullPointerException
> >> >>>         at java/lang/Throwable.printStackTrace ( Throwable.java:231)
> >> >>>         at com/vladium/emma/rt/RTExitHook.createClassLoaderClosure
> >> (RTExitH
> >> >>>
> >> >> ook.java:63)
> >> >>
> >> >>>         at com/vladium/emma/rt/RT.reset ( RT.java:92)
> >> >>>         at com/vladium/emma/rt/RT.<clinit> (RT.java:238)
> >> >>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
> >> >>>         at java/lang/J9VMInternals.initialize ( J9VMInternals.java
> >> :185)
> >> >>>         at java/nio/Buffer.$VRi (Buffer.java:4294967295)
> >> >>>         at java/nio/Buffer.<clinit> (Buffer.java:4294967295)
> >> >>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
> >> >>>         at java/lang/J9VMInternals.initialize
> >> (J9VMInternals.java:185)
> >> >>>         at java/lang/J9VMInternals.initialize
> >> (J9VMInternals.java:154)
> >> >>>         at java/lang/String.getBytes ( String.java:783)
> >> >>>         at org/apache/harmony/luni/util/Util.getBytes (Util.java:54)
> >> >>>         at java/lang/ClassLoader.loadLibraryWithPath
> >> (ClassLoader.java
> >> :998)
> >> >>>         at java/lang/ClassLoader.loadLibraryWithClassLoader (
> >> ClassLoader.ja
> >> >>>
> >> >> va:970)
> >> >>
> >> >>>         at java/lang/System.loadLibrary (System.java:706)
> >> >>>         at com/ibm/icu4jni/common/ICU4JNILoader.loadLibrary (Unknown
> >> Source
> >> >>>
> >> >> )
> >> >>
> >> >>>         at com/ibm/icu4jni/converters/NativeConverter.<clinit>
> >> (Unknown Sou
> >> >>>
> >> >> rce)
> >> >>
> >> >>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
> >> >>>         at java/lang/J9VMInternals.initialize
> >> (J9VMInternals.java:185)
> >> >>>         at com/ibm/icu4jni/charset/CharsetProviderICU.charsetForName
> >> (Unkno
> >> >>>
> >> >> wn Source)
> >> >>
> >> >>>         at java/nio/charset/Charset.forNameInternal
> >> (Charset.java:450)
> >> >>>         at java/nio/charset/Charset.forName (Charset.java:513)
> >> >>>         at java/lang/String.defaultCharset ( String.java:688)
> >> >>>         at java/lang/String.<init> (String.java:252)
> >> >>>         at org/apache/harmony/luni/util/Util.toString (Util.java:74)
> >> >>>         at java/lang/System.getPropertyList (Native Method)
> >> >>>         at java/lang/System.ensureProperties (System.java:496)
> >> >>>         at java/lang/System.<clinit> (System.java:55)
> >> >>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
> >> >>>         at java/lang/J9VMInternals.initialize
> >> (J9VMInternals.java:185)
> >> >>>         at java/lang/ClassLoader.initializeClassLoaders (
> >> ClassLoader.java:6
> >> >>>
> >> >> 5)
> >> >>
> >> >>>         at java/lang/Thread.initialize (Thread.java:301)
> >> >>>         at java/lang/Thread.<init> (Thread.java:120)
> >> >>> JVMJ9VM015W Initialization error for library jclclear_23(14):
> >> JVMJ9VM009E J
> >> >>>
> >> >> 9VMDllMain failed
> >> >>
> >> >>> HMYEXEL062E Internal VM error: Failed to create Java VM
> >> >>> FAILED.
> >> >>>
> >> >>>
> >> >>> On 1 June 2006 at 22:26, "Mikhail Loenko" < mloenko@gmail.com>
> >> wrote:
> >> >>>
> >> >>>
> >> >>>> 2006/6/1, Geir Magnusson Jr <ge...@pobox.com>:
> >> >>>>
> >> >>>>
> >> >>>>> Mikhail Loenko wrote:
> >> >>>>>
> >> >>>>>
> >> >>>>>> 2006/6/1, Geir Magnusson Jr <geir@pobox.com >:
> >> >>>>>>
> >> >>>>>>
> >> >>>>>>> I don't quite grok what you mean - could you post what you are
> >> >>>>>>> suggesting in code here on -dev?
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>> Index:
> >> modules/luni/src/test/java/tests/api/java/net/Inet6AddressTest.ja
> >> >>>>>>
> >> >> v
> >> >>
> >> >>>>>>
> >> >>>>>>
> >> >>>> a
> >> >>>>
> >> >>>>
> >> >>>>>>
> >> ===================================================================
> >> >>>>>> ---
> >> >>>>>>
> >> modules/luni/src/test/java/tests/api/java/net/Inet6AddressTest.java
> >> >>>>>> (revision
> >> >>>>>> 410480)
> >> >>>>>> +++
> >> >>>>>>
> >> modules/luni/src/test/java/tests/api/java/net/Inet6AddressTest.java
> >>
> >> >>>>>> (working
> >> >>>>>> copy)
> >> >>>>>> @@ -785,7 +785,7 @@
> >> >>>>>>                 "FFFF:FFFF:FFFF:FFFF:FFFF:FFFF: 255.255.255.255",
> >> >>>>>>                 "0:0:0:0:0:0:0:0", "0:0:0:0:0:0:0.0.0.0" };
> >> >>>>>>
> >> >>>>>> -        String invalidIPAddresses[] = { ":", "FFFF:FFFF", "1:1",
> >> >>>>>> "::1.2.3.444" };
> >> >>>>>> +        String invalidIPAddresses[] = { "FFFF:FFFF" };
> >> >>>>>>
> >> >>>>>>         for (int i = 0; i < validIPAddresses.length; i++) {
> >> >>>>>>
> >> >>>>>>
> >> >>>>>>
> >> >>>>>>
> >> >>>>> Thx - why do you want to remove testing the others?  Because you
> >> believe
> >> >>>>> that it is in effect testing the resolver in the OS since each
> >> returns
> >> >>>>> the same answer,and thus it's testing the same thing over and
> >> over?
> >> >>>>>
> >> >>>>>
> >> >>>> I believe so, but I'd like to hear confirmation from code authors
> >> >>>>
> >> >>>> Thanks,
> >> >>>> Mikhail
> >> >>>>
> >> >>>>
> >> >>>
> >> >>>
> >> ---------------------------------------------------------------------
> >> >>> Terms of use : http://incubator.apache.org/harmony/mailing.html
> >> >>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> >> >>> For additional commands, e-mail:
> >> harmony-dev-help@incubator.apache.org
> >> >>>
> >> >>>
> >> >>>
> >> >>>
> >> >> --
> >> >> Paulex Yang
> >> >> China Software Development Lab
> >> >> IBM
> >> >>
> >> >>
> >> >>
> >> >> ---------------------------------------------------------------------
> >> >> Terms of use : http://incubator.apache.org/harmony/mailing.html
> >> >> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> >> >> For additional commands, e-mail:
> >> harmony-dev-help@incubator.apache.org
> >> >>
> >> >
> >> >
> >> >
> >> > ---------------------------------------------------------------------
> >> > Terms of use : http://incubator.apache.org/harmony/mailing.html
> >> > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> >> > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >> >
> >> >
> >> >
> >>
> >>
> >> --
> >> Paulex Yang
> >> China Software Development Lab
> >> IBM
> >>
> >>
> >>
> >> ---------------------------------------------------------------------
> >> Terms of use : http://incubator.apache.org/harmony/mailing.html
> >> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> >> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >>
> >>
> >
>
>
> --
> Paulex Yang
> China Software Development Lab
> IBM
>
>
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>

---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org


Re: [classlib] [testing] Coverage (was Re: 37% of total test execution time is spent in a single test)

Posted by Geir Magnusson Jr <ge...@pobox.com>.
yes.

And/or add it to the upcoming donation of the [testing] infrastructure
to make it part of the CI runs.

geir

Nathan Beyer wrote:
> Is there any possibility of adding this to an Ant script that can optionally
> be run with the build scripts? I'd like to make this easier for everyone to
> run while developing.
> 
> Has anyone tried any other coverage tools? Like TPTP for Eclipse, Clover or
> Corbetura (sp)?
> 
> -Nathan
> 
>> -----Original Message-----
>> From: Vladimir Ivanov [mailto:ivavladimir@gmail.com]
>>
>> Latest Harmony API source coverage by Harmony API unit tests results I
>> stored at wiki page
>>
>> http://wiki.apache.org/harmony/Coverage_information
>>
>>
>>
>> I'm going to refresh it bi-weekly (seems, it is enough for coverage).
>>
>>
>>
>>> I think we have got a agreement on the test naming convention[1], but
>>> for sure, there have been many (legacy) test cases before this
>>> agreement, and I think the volunteer is highly welcome to provide patch
>>> for them.
>>>
>>> [1]http://incubator.apache.org/harmony/subcomponents/classlibrary/testing
>> .html
>>
>> If nobody objects I'm going to look through the unit tests to correct
>> package names according to the agreement (where needed).
>>
>>
>>
>>  Thanks,
>>
>>    Vladimir
>>
>>
>>
>> On 6/8/06, Paulex Yang <paulex.yang@gmail.com > wrote:
>>> Vladimir,
>>>
>>> Vladimir Ivanov wrote:
>>>> Thanks Paulex!
>>>>
>>>> I did the same, but could not send results due to spam filter J
>>>> Observations:
>>>>
>>>>   1. Coverage results look pretty much similar.
>>>>   2. Exclude list looks pretty much similar too, but, looks like it
>>>>   depends on the way of data collection (I didn't run ant task and the
>>>> list is
>>>>   a little bit different).
>>> Great.
>>>> In any case, I think, when we run harmony on another VM exclude list
>>> will
>>>> have to be updated.
>>>>
>>>>
>>>>
>>>> May be we can start publishing the coverage information on wiki pages
>>> and
>>>> provide some updates time to time (I can do it)?
>>>>
>>> +1,  and of course, you can only if  no one in the mailing list objects
>>> and you'll have my welcome, and I think it will be even greater if these
>>> reports can be generated regularly like what JAPI is doing:)
>>>>
>>>> One note:
>>>>
>>>> I noticed that different unit tests have very different package names
>>>>
>>>> Now the directory with all built tests copied to one place looks like:
>>> I think we have got a agreement on the test naming convention[1], but
>>> for sure, there have been many (legacy) test cases before this
>>> agreement, and I think the volunteer is highly welcome to provide patch
>>> for them.
>>>
>>>
>> [1]http://incubator.apache.org/harmony/subcomponents/classlibrary/testing.
>> html
>>>>
>>>>
>>>> C:\coverage\tests\test>ls
>>>>
>>>> GZIPOutClose2.txt        api           config         javax
>>>> tests
>>>>
>>>> GZIPOutFinish.txt        api.injected  dazzle         org
>> xml
>>>> GZIPOutWrite.txt         binary        gif            prefs
>>>>
>>>> Inet6Address.golden.ser  bundles       impl           serialization
>>>>
>>>> JDK2-3gabba.zip          com           impl.injected   test.txt
>>>>
>>>>
>>>>
>>>> I think, it would be good if tests had unified package names.
>>>>
>>>> Why? - so far, just common sense, just to have an order in test suite
>>>>
>>>> Organization (if consider all unit tests as solid test suite).
>>>>
>>>> Thanks,
>>>>  Vladimir
>>>>
>>>> For example, my exclude list for java.io is:
>>>> -java.io.BufferedInputStream ,
>>>> -java.io.BufferedOutputStream,
>>>> -java.io.File ,
>>>> -java.io.FileChannelFactory,
>>>> -java.io.FileDescriptor,
>>>> -java.io.FileInputStream,
>>>> -java.io.FileOutputStream,
>>>> -java.io.FilterInputStream ,
>>>> -java.io.FilterOutputStream,
>>>> - java.io.InputStream,
>>>> -java.io.OutputStream,
>>>> -java.io.ObjectStreamField,
>>>> -java.io.PrintStream
>>>>
>>>>
>>>> On 6/6/06, Paulex Yang < paulex.yang@gmail.com > wrote:
>>>>> I've attach the scripts and excluded class lists to JIRA,  please
>> refer
>>>>> to https://issues.apache.org/jira/browse/HARMONY-564 .  Enjoy it:).
>>>>>
>>>>> Mark Hindess wrote:
>>>>>> On 2 June 2006 at 10:37, Paulex Yang < paulex.yang@gmail.com>
>> wrote:
>>>>>>> Mark,
>>>>>>>
>>>>>>> I'm glad that there is someone else has interest on emma, I've
>>>>> tried it
>>>>>>> before. AFAIK, emma works by instrumentation, but sometimes for
>>>>> classes
>>>>>>> in bootclasspath, the instrumentation cannot work, there are two
>>>>> cases:
>>>>>
>>>>>>> 1. Some instrumented classes cannot be loaded by VM.
>>>>>>> 2. Some classes cannot be instrumented
>>>>>>>
>>>>>>> I have tried to look more inside to find some way to work around
>>>>> but I
>>>>>>> haven't got enough time yet.
>>>>>>>
>>>>>>> Specifically for nio-channel module, I had a list for these two
>>> cases
>>>>> (I
>>>>>>> believe the data is a little outdated and should be reevaluated)
>>>>>>> case 1.
>>>>>>> BaseByteBuffer.class
>>>>>>> Buffer.class
>>>>>>> BufferFactory.class
>>>>>>> ByteBuffer.class
>>>>>>> CharArrayBuffer.class
>>>>>>> CharBuffer.class
>>>>>>> HeapByteBuffer.class
>>>>>>> ReadWriteCharArrayBuffer.class
>>>>>>> ReadWriteHeapByteBuffer.class
>>>>>>> FileChannel.class
>>>>>>> AbstractInterruptibleChannel.class
>>>>>>> FileChannelImpl.class
>>>>>>> WriteOnlyFileChannel.class
>>>>>>> LockManager.class
>>>>>>> LockManager$1.class
>>>>>>> ReadOnlyFileChannel.class
>>>>>>>
>>>>>>> case 2:
>>>>>>> ByteChannel.class
>>>>>>> Channel.class
>>>>>>> GatheringByteChannel.class
>>>>>>> InterruptibleChannel.class
>>>>>>> WritableByteChannel.class
>>>>>>>
>>>>>>> And I have got some ant script and more excluded list for emma, if
>>>>>>> anyone has interests, I can upload it to JIRA.
>>>>>>>
>>>>>> Yes!
>>>>>>
>>>>>> -Mark.
>>>>>>
>>>>>>
>>>>>>> Mark Hindess wrote:
>>>>>>>
>>>>>>>> Anyone tried using emma (emma.sf.net) to look at test coverage
>>>>> for our
>>>>>>>> tests?  That might help ensure we don't trim them down too much?
>>>>>>>>
>>>>>>>> I suspect we will soon reach a point where there are
>> significantly
>>>>> fewer
>>>>>>>> obvious opportunities for small contributions - e.g.
>>>>> 1.5generification,
>>>>>>>> etc from looking at JAPI reports.  Having test coverage reports
>>>>> might
>>>>>>>> help by highlighting areas for improvement.
>>>>>>>>
>>>>>>>> I had a quick try with:
>>>>>>>>
>>>>>>>>   ant -f make/build.xml
>>>>>>>>   ant -f modules/nio/make/build.xml test
>>>>>>>>   java -cp emma.jar emma instr -m overwrite -cp
>>>>> deploy/jdk/jre/lib/boot/nio
>>>>>>> .jar
>>>>>>>
>>>>>>>> and then:
>>>>>>>>
>>>>>>>>   java -Xbootclasspath/p: emma.jar \
>>>>>>>>        -cp build/tests:modules/nio/bin/test:junit.jar:. \
>>>>>>>>        junit.textui.TestRunner \
>>>>>>>>
>> org.apache.harmony.tests.java.nio.channels.SocketChannelTest
>>>>>>>> (Running all the tests is a little tricky because test depends on
>>>>> build,
>>>>>>>> which depends on clean which wipes out the instr'd jar. ;-( )
>>>>>>>>
>>>>>>>> Sadly it dies horribly with the stack trace below.  I'll take
>>>>> another
>>>>>>>> look when I have a little more time, but I thought I'd mention it
>>> in
>>>>>>>> case anyone else had any thoughts on the subject.
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>>
>>>>>>>> Mark.
>>>>>>>>
>>>>>>>> Exception in thread "main" java/lang/ExceptionInInitializerError
>>>>>>>>         at java/lang/J9VMInternals.initialize
>>>>> (J9VMInternals.java:203)
>>>>>>>>         at java/nio/Buffer.$VRi ( Buffer.java:4294967295)
>>>>>>>>         at java/nio/Buffer.<clinit> ( Buffer.java:4294967295)
>>>>>>>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
>>>>>>>>         at java/lang/J9VMInternals.initialize (
>> J9VMInternals.java
>>>>> :185)
>>>>>>>>         at java/lang/J9VMInternals.initialize
>>>>> (J9VMInternals.java:154)
>>>>>>>>         at java/lang/String.getBytes (String.java:783)
>>>>>>>>         at org/apache/harmony/luni/util/Util.getBytes (
>>>>> Util.java:54)
>>>>>>>>         at java/lang/ClassLoader.loadLibraryWithPath
>>>>> (ClassLoader.java
>>>>> :998)
>>>>>>>>         at java/lang/ClassLoader.loadLibraryWithClassLoader (
>>>>> ClassLoader.ja
>>>>>>> va:970)
>>>>>>>
>>>>>>>>         at java/lang/System.loadLibrary (System.java:706)
>>>>>>>>         at com/ibm/icu4jni/common/ICU4JNILoader.loadLibrary
>>> (Unknown
>>>>> Source
>>>>>>> )
>>>>>>>
>>>>>>>>         at com/ibm/icu4jni/converters/NativeConverter.<clinit>
>>>>> (Unknown Sou
>>>>>>> rce)
>>>>>>>
>>>>>>>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
>>>>>>>>         at java/lang/J9VMInternals.initialize
>>>>> ( J9VMInternals.java:185)
>>>>>>>>         at
>>> com/ibm/icu4jni/charset/CharsetProviderICU.charsetForName
>>>>> (Unkno
>>>>>>> wn Source)
>>>>>>>
>>>>>>>>         at java/nio/charset/Charset.forNameInternal
>>>>> (Charset.java:450)
>>>>>>>>         at java/nio/charset/Charset.forName (Charset.java:513)
>>>>>>>>         at java/lang/String.defaultCharset ( String.java:688)
>>>>>>>>         at java/lang/String.<init> ( String.java:252)
>>>>>>>>         at org/apache/harmony/luni/util/Util.toString (Util.java
>>> :74)
>>>>>>>>         at java/lang/System.getPropertyList (Native Method)
>>>>>>>>         at java/lang/System.ensureProperties ( System.java:496)
>>>>>>>>         at java/lang/System.<clinit> (System.java:55)
>>>>>>>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
>>>>>>>>         at java/lang/J9VMInternals.initialize
>>>>> (J9VMInternals.java:185)
>>>>>>>>         at java/lang/ClassLoader.initializeClassLoaders (
>>>>> ClassLoader.java:6
>>>>>>> 5)
>>>>>>>
>>>>>>>>         at java/lang/Thread.initialize (Thread.java:301)
>>>>>>>>         at java/lang/Thread.<init> (Thread.java:120)
>>>>>>>> java/lang/NullPointerException
>>>>>>>>         at java/lang/Throwable.printStackTrace ( Throwable.java
>>> :231)
>>>>>>>>         at
>> com/vladium/emma/rt/RTExitHook.createClassLoaderClosure
>>>>> (RTExitH
>>>>>>> ook.java:63)
>>>>>>>
>>>>>>>>         at com/vladium/emma/rt/RT.reset ( RT.java:92)
>>>>>>>>         at com/vladium/emma/rt/RT.<clinit> (RT.java:238)
>>>>>>>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
>>>>>>>>         at java/lang/J9VMInternals.initialize (
>> J9VMInternals.java
>>>>> :185)
>>>>>>>>         at java/nio/Buffer.$VRi ( Buffer.java:4294967295)
>>>>>>>>         at java/nio/Buffer.<clinit> (Buffer.java:4294967295)
>>>>>>>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
>>>>>>>>         at java/lang/J9VMInternals.initialize
>>>>> (J9VMInternals.java:185)
>>>>>>>>         at java/lang/J9VMInternals.initialize
>>>>> (J9VMInternals.java:154)
>>>>>>>>         at java/lang/String.getBytes ( String.java :783)
>>>>>>>>         at org/apache/harmony/luni/util/Util.getBytes (Util.java
>>> :54)
>>>>>>>>         at java/lang/ClassLoader.loadLibraryWithPath
>>>>> (ClassLoader.java
>>>>> :998)
>>>>>>>>         at java/lang/ClassLoader.loadLibraryWithClassLoader (
>>>>> ClassLoader.ja
>>>>>>> va:970)
>>>>>>>
>>>>>>>>         at java/lang/System.loadLibrary ( System.java:706)
>>>>>>>>         at com/ibm/icu4jni/common/ICU4JNILoader.loadLibrary
>>> (Unknown
>>>>> Source
>>>>>>> )
>>>>>>>
>>>>>>>>         at com/ibm/icu4jni/converters/NativeConverter.<clinit>
>>>>> (Unknown Sou
>>>>>>> rce)
>>>>>>>
>>>>>>>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
>>>>>>>>         at java/lang/J9VMInternals.initialize
>>>>> (J9VMInternals.java:185)
>>>>>>>>         at
>>> com/ibm/icu4jni/charset/CharsetProviderICU.charsetForName
>>>>> (Unkno
>>>>>>> wn Source)
>>>>>>>
>>>>>>>>         at java/nio/charset/Charset.forNameInternal
>>>>> (Charset.java:450)
>>>>>>>>         at java/nio/charset/Charset.forName (Charset.java:513)
>>>>>>>>         at java/lang/String.defaultCharset ( String.java:688)
>>>>>>>>         at java/lang/String.<init> (String.java:252)
>>>>>>>>         at org/apache/harmony/luni/util/Util.toString (Util.java
>>> :74)
>>>>>>>>         at java/lang/System.getPropertyList (Native Method)
>>>>>>>>         at java/lang/System.ensureProperties (System.java:496)
>>>>>>>>         at java/lang/System.<clinit> (System.java:55)
>>>>>>>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
>>>>>>>>         at java/lang/J9VMInternals.initialize
>>>>> (J9VMInternals.java:185)
>>>>>>>>         at java/lang/ClassLoader.initializeClassLoaders (
>>>>> ClassLoader.java:6
>>>>>>> 5)
>>>>>>>
>>>>>>>>         at java/lang/Thread.initialize (Thread.java:301)
>>>>>>>>         at java/lang/Thread.<init> (Thread.java :120)
>>>>>>>> JVMJ9VM015W Initialization error for library jclclear_23(14):
>>>>> JVMJ9VM009E J
>>>>>>> 9VMDllMain failed
>>>>>>>
>>>>>>>> HMYEXEL062E Internal VM error: Failed to create Java VM
>>>>>>>> FAILED.
>>>>>>>>
>>>>>>>>
>>>>>>>> On 1 June 2006 at 22:26, "Mikhail Loenko" < mloenko@gmail.com >
>>>>> wrote:
>>>>>>>>
>>>>>>>>> 2006/6/1, Geir Magnusson Jr < geir@pobox.com>:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> Mikhail Loenko wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> 2006/6/1, Geir Magnusson Jr < geir@pobox.com >:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> I don't quite grok what you mean - could you post what you
>> are
>>>>>>>>>>>> suggesting in code here on -dev?
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> Index:
>>>>> modules/luni/src/test/java/tests/api/java/net/Inet6AddressTest.ja
>>>>>>> v
>>>>>>>
>>>>>>>>>>>
>>>>>>>>> a
>>>>>>>>>
>>>>>>>>>
>>>>> ===================================================================
>>>>>>>>>>> ---
>>>>>>>>>>>
>>>>> modules/luni/src/test/java/tests/api/java/net/Inet6AddressTest.java
>>>>>>>>>>> (revision
>>>>>>>>>>> 410480)
>>>>>>>>>>> +++
>>>>>>>>>>>
>>>>> modules/luni/src/test/java/tests/api/java/net/Inet6AddressTest.java
>>>>>
>>>>>>>>>>> (working
>>>>>>>>>>> copy)
>>>>>>>>>>> @@ -785,7 +785,7 @@
>>>>>>>>>>>                 "FFFF:FFFF:FFFF:FFFF:FFFF:FFFF:
>> 255.255.255.255
>>> ",
>>>>>>>>>>>                 "0:0:0:0:0:0:0:0", "0:0:0:0:0:0: 0.0.0.0" };
>>>>>>>>>>>
>>>>>>>>>>> -        String invalidIPAddresses[] = { ":", "FFFF:FFFF",
>>> "1:1",
>>>>>>>>>>> "::1.2.3.444" };
>>>>>>>>>>> +        String invalidIPAddresses[] = { "FFFF:FFFF" };
>>>>>>>>>>>
>>>>>>>>>>>         for (int i = 0; i < validIPAddresses.length; i++) {
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>> Thx - why do you want to remove testing the others?  Because
>> you
>>>>> believe
>>>>>>>>>> that it is in effect testing the resolver in the OS since each
>>>>> returns
>>>>>>>>>> the same answer,and thus it's testing the same thing over and
>>>>> over?
>>>>>>>>>>
>>>>>>>>> I believe so, but I'd like to hear confirmation from code
>> authors
>>>>>>>>> Thanks,
>>>>>>>>> Mikhail
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>> ---------------------------------------------------------------------
>>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>>>> To unsubscribe, e-mail: harmony-dev-
>> unsubscribe@incubator.apache.org
>>>>>>>> For additional commands, e-mail:
>>>>> harmony-dev-help@incubator.apache.org
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>> --
>>>>>>> Paulex Yang
>>>>>>> China Software Development Lab
>>>>>>> IBM
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>> ---------------------------------------------------------------------
>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>>> To unsubscribe, e-mail: harmony-dev-
>> unsubscribe@incubator.apache.org
>>>>>>> For additional commands, e-mail:
>>>>> harmony-dev-help@incubator.apache.org
>>>>>>
>>>>>>
>>>>>> -------------------------------------------------------------------
>> --
>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>> To unsubscribe, e-mail: harmony-dev-
>> unsubscribe@incubator.apache.org
>>>>>> For additional commands, e-mail:
>>> harmony-dev-help@incubator.apache.org
>>>>>>
>>>>>>
>>>>>
>>>>> --
>>>>> Paulex Yang
>>>>> China Software Development Lab
>>>>> IBM
>>>>>
>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>>>> For additional commands, e-mail: harmony-dev-
>> help@incubator.apache.org
>>>>>
>>>
>>> --
>>> Paulex Yang
>>> China Software Development Lab
>>> IBM
>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>>
>>>
> 
> 
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> 
> 


---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org


RE: [classlib] [testing] Coverage (was Re: 37% of total test execution time is spent in a single test)

Posted by Nathan Beyer <nb...@kc.rr.com>.
Is there any possibility of adding this to an Ant script that can optionally
be run with the build scripts? I'd like to make this easier for everyone to
run while developing.

Has anyone tried any other coverage tools? Like TPTP for Eclipse, Clover or
Corbetura (sp)?

-Nathan

> -----Original Message-----
> From: Vladimir Ivanov [mailto:ivavladimir@gmail.com]
> 
> Latest Harmony API source coverage by Harmony API unit tests results I
> stored at wiki page
> 
> http://wiki.apache.org/harmony/Coverage_information
> 
> 
> 
> I'm going to refresh it bi-weekly (seems, it is enough for coverage).
> 
> 
> 
> >I think we have got a agreement on the test naming convention[1], but
> >for sure, there have been many (legacy) test cases before this
> >agreement, and I think the volunteer is highly welcome to provide patch
> >for them.
> >
> >[1]http://incubator.apache.org/harmony/subcomponents/classlibrary/testing
> .html
> 
> >
> 
> If nobody objects I'm going to look through the unit tests to correct
> package names according to the agreement (where needed).
> 
> 
> 
>  Thanks,
> 
>    Vladimir
> 
> 
> 
> On 6/8/06, Paulex Yang <paulex.yang@gmail.com > wrote:
> >
> > Vladimir,
> >
> > Vladimir Ivanov wrote:
> > > Thanks Paulex!
> > >
> > > I did the same, but could not send results due to spam filter J
> > > Observations:
> > >
> > >   1. Coverage results look pretty much similar.
> > >   2. Exclude list looks pretty much similar too, but, looks like it
> > >   depends on the way of data collection (I didn't run ant task and the
> > > list is
> > >   a little bit different).
> > Great.
> > >
> > > In any case, I think, when we run harmony on another VM exclude list
> > will
> > > have to be updated.
> > >
> > >
> > >
> > > May be we can start publishing the coverage information on wiki pages
> > and
> > > provide some updates time to time (I can do it)?
> > >
> > +1,  and of course, you can only if  no one in the mailing list objects
> > and you'll have my welcome, and I think it will be even greater if these
> > reports can be generated regularly like what JAPI is doing:)
> > >
> > >
> > > One note:
> > >
> > > I noticed that different unit tests have very different package names
> > >
> > > Now the directory with all built tests copied to one place looks like:
> > I think we have got a agreement on the test naming convention[1], but
> > for sure, there have been many (legacy) test cases before this
> > agreement, and I think the volunteer is highly welcome to provide patch
> > for them.
> >
> >
> [1]http://incubator.apache.org/harmony/subcomponents/classlibrary/testing.
> html
> >
> > >
> > >
> > >
> > > C:\coverage\tests\test>ls
> > >
> > > GZIPOutClose2.txt        api           config         javax
> > > tests
> > >
> > > GZIPOutFinish.txt        api.injected  dazzle         org
> xml
> >
> > >
> > > GZIPOutWrite.txt         binary        gif            prefs
> > >
> > > Inet6Address.golden.ser  bundles       impl           serialization
> > >
> > > JDK2-3gabba.zip          com           impl.injected   test.txt
> > >
> > >
> > >
> > > I think, it would be good if tests had unified package names.
> > >
> > > Why? - so far, just common sense, just to have an order in test suite
> > >
> > > Organization (if consider all unit tests as solid test suite).
> > >
> > > Thanks,
> > >  Vladimir
> > >
> > > For example, my exclude list for java.io is:
> > > -java.io.BufferedInputStream ,
> > > -java.io.BufferedOutputStream,
> > > -java.io.File ,
> > > -java.io.FileChannelFactory,
> > > -java.io.FileDescriptor,
> > > -java.io.FileInputStream,
> > > -java.io.FileOutputStream,
> > > -java.io.FilterInputStream ,
> > > -java.io.FilterOutputStream,
> > > - java.io.InputStream,
> > > -java.io.OutputStream,
> > > -java.io.ObjectStreamField,
> > > -java.io.PrintStream
> > >
> > >
> > > On 6/6/06, Paulex Yang < paulex.yang@gmail.com > wrote:
> > >>
> > >> I've attach the scripts and excluded class lists to JIRA,  please
> refer
> > >> to https://issues.apache.org/jira/browse/HARMONY-564 .  Enjoy it:).
> > >>
> > >> Mark Hindess wrote:
> > >> > On 2 June 2006 at 10:37, Paulex Yang < paulex.yang@gmail.com>
> wrote:
> > >> >
> > >> >> Mark,
> > >> >>
> > >> >> I'm glad that there is someone else has interest on emma, I've
> > >> tried it
> > >> >> before. AFAIK, emma works by instrumentation, but sometimes for
> > >> classes
> > >> >> in bootclasspath, the instrumentation cannot work, there are two
> > >> cases:
> > >>
> > >> >> 1. Some instrumented classes cannot be loaded by VM.
> > >> >> 2. Some classes cannot be instrumented
> > >> >>
> > >> >> I have tried to look more inside to find some way to work around
> > >> but I
> > >> >> haven't got enough time yet.
> > >> >>
> > >> >> Specifically for nio-channel module, I had a list for these two
> > cases
> > >> (I
> > >> >> believe the data is a little outdated and should be reevaluated)
> > >> >> case 1.
> > >> >> BaseByteBuffer.class
> > >> >> Buffer.class
> > >> >> BufferFactory.class
> > >> >> ByteBuffer.class
> > >> >> CharArrayBuffer.class
> > >> >> CharBuffer.class
> > >> >> HeapByteBuffer.class
> > >> >> ReadWriteCharArrayBuffer.class
> > >> >> ReadWriteHeapByteBuffer.class
> > >> >> FileChannel.class
> > >> >> AbstractInterruptibleChannel.class
> > >> >> FileChannelImpl.class
> > >> >> WriteOnlyFileChannel.class
> > >> >> LockManager.class
> > >> >> LockManager$1.class
> > >> >> ReadOnlyFileChannel.class
> > >> >>
> > >> >> case 2:
> > >> >> ByteChannel.class
> > >> >> Channel.class
> > >> >> GatheringByteChannel.class
> > >> >> InterruptibleChannel.class
> > >> >> WritableByteChannel.class
> > >> >>
> > >> >> And I have got some ant script and more excluded list for emma, if
> > >> >> anyone has interests, I can upload it to JIRA.
> > >> >>
> > >> >
> > >> > Yes!
> > >> >
> > >> > -Mark.
> > >> >
> > >> >
> > >> >> Mark Hindess wrote:
> > >> >>
> > >> >>> Anyone tried using emma (emma.sf.net) to look at test coverage
> > >> for our
> > >> >>> tests?  That might help ensure we don't trim them down too much?
> > >> >>>
> > >> >>> I suspect we will soon reach a point where there are
> significantly
> > >> fewer
> > >> >>> obvious opportunities for small contributions - e.g.
> > >> 1.5generification,
> > >> >>> etc from looking at JAPI reports.  Having test coverage reports
> > >> might
> > >> >>> help by highlighting areas for improvement.
> > >> >>>
> > >> >>> I had a quick try with:
> > >> >>>
> > >> >>>   ant -f make/build.xml
> > >> >>>   ant -f modules/nio/make/build.xml test
> > >> >>>   java -cp emma.jar emma instr -m overwrite -cp
> > >> deploy/jdk/jre/lib/boot/nio
> > >> >>>
> > >> >> .jar
> > >> >>
> > >> >>> and then:
> > >> >>>
> > >> >>>   java -Xbootclasspath/p: emma.jar \
> > >> >>>        -cp build/tests:modules/nio/bin/test:junit.jar:. \
> > >> >>>        junit.textui.TestRunner \
> > >> >>>
> org.apache.harmony.tests.java.nio.channels.SocketChannelTest
> > >> >>>
> > >> >>> (Running all the tests is a little tricky because test depends on
> > >> build,
> > >> >>> which depends on clean which wipes out the instr'd jar. ;-( )
> > >> >>>
> > >> >>> Sadly it dies horribly with the stack trace below.  I'll take
> > >> another
> > >> >>> look when I have a little more time, but I thought I'd mention it
> > in
> > >> >>> case anyone else had any thoughts on the subject.
> > >> >>>
> > >> >>> Regards,
> > >> >>>
> > >> >>> Mark.
> > >> >>>
> > >> >>> Exception in thread "main" java/lang/ExceptionInInitializerError
> > >> >>>         at java/lang/J9VMInternals.initialize
> > >> (J9VMInternals.java:203)
> > >> >>>         at java/nio/Buffer.$VRi ( Buffer.java:4294967295)
> > >> >>>         at java/nio/Buffer.<clinit> ( Buffer.java:4294967295)
> > >> >>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
> > >> >>>         at java/lang/J9VMInternals.initialize (
> J9VMInternals.java
> > >> :185)
> > >> >>>         at java/lang/J9VMInternals.initialize
> > >> (J9VMInternals.java:154)
> > >> >>>         at java/lang/String.getBytes (String.java:783)
> > >> >>>         at org/apache/harmony/luni/util/Util.getBytes (
> > >> Util.java:54)
> > >> >>>         at java/lang/ClassLoader.loadLibraryWithPath
> > >> (ClassLoader.java
> > >> :998)
> > >> >>>         at java/lang/ClassLoader.loadLibraryWithClassLoader (
> > >> ClassLoader.ja
> > >> >>>
> > >> >> va:970)
> > >> >>
> > >> >>>         at java/lang/System.loadLibrary (System.java:706)
> > >> >>>         at com/ibm/icu4jni/common/ICU4JNILoader.loadLibrary
> > (Unknown
> > >> Source
> > >> >>>
> > >> >> )
> > >> >>
> > >> >>>         at com/ibm/icu4jni/converters/NativeConverter.<clinit>
> > >> (Unknown Sou
> > >> >>>
> > >> >> rce)
> > >> >>
> > >> >>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
> > >> >>>         at java/lang/J9VMInternals.initialize
> > >> ( J9VMInternals.java:185)
> > >> >>>         at
> > com/ibm/icu4jni/charset/CharsetProviderICU.charsetForName
> > >> (Unkno
> > >> >>>
> > >> >> wn Source)
> > >> >>
> > >> >>>         at java/nio/charset/Charset.forNameInternal
> > >> (Charset.java:450)
> > >> >>>         at java/nio/charset/Charset.forName (Charset.java:513)
> > >> >>>         at java/lang/String.defaultCharset ( String.java:688)
> > >> >>>         at java/lang/String.<init> ( String.java:252)
> > >> >>>         at org/apache/harmony/luni/util/Util.toString (Util.java
> > :74)
> > >> >>>         at java/lang/System.getPropertyList (Native Method)
> > >> >>>         at java/lang/System.ensureProperties ( System.java:496)
> > >> >>>         at java/lang/System.<clinit> (System.java:55)
> > >> >>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
> > >> >>>         at java/lang/J9VMInternals.initialize
> > >> (J9VMInternals.java:185)
> > >> >>>         at java/lang/ClassLoader.initializeClassLoaders (
> > >> ClassLoader.java:6
> > >> >>>
> > >> >> 5)
> > >> >>
> > >> >>>         at java/lang/Thread.initialize (Thread.java:301)
> > >> >>>         at java/lang/Thread.<init> (Thread.java:120)
> > >> >>> java/lang/NullPointerException
> > >> >>>         at java/lang/Throwable.printStackTrace ( Throwable.java
> > :231)
> > >> >>>         at
> com/vladium/emma/rt/RTExitHook.createClassLoaderClosure
> > >> (RTExitH
> > >> >>>
> > >> >> ook.java:63)
> > >> >>
> > >> >>>         at com/vladium/emma/rt/RT.reset ( RT.java:92)
> > >> >>>         at com/vladium/emma/rt/RT.<clinit> (RT.java:238)
> > >> >>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
> > >> >>>         at java/lang/J9VMInternals.initialize (
> J9VMInternals.java
> > >> :185)
> > >> >>>         at java/nio/Buffer.$VRi ( Buffer.java:4294967295)
> > >> >>>         at java/nio/Buffer.<clinit> (Buffer.java:4294967295)
> > >> >>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
> > >> >>>         at java/lang/J9VMInternals.initialize
> > >> (J9VMInternals.java:185)
> > >> >>>         at java/lang/J9VMInternals.initialize
> > >> (J9VMInternals.java:154)
> > >> >>>         at java/lang/String.getBytes ( String.java :783)
> > >> >>>         at org/apache/harmony/luni/util/Util.getBytes (Util.java
> > :54)
> > >> >>>         at java/lang/ClassLoader.loadLibraryWithPath
> > >> (ClassLoader.java
> > >> :998)
> > >> >>>         at java/lang/ClassLoader.loadLibraryWithClassLoader (
> > >> ClassLoader.ja
> > >> >>>
> > >> >> va:970)
> > >> >>
> > >> >>>         at java/lang/System.loadLibrary ( System.java:706)
> > >> >>>         at com/ibm/icu4jni/common/ICU4JNILoader.loadLibrary
> > (Unknown
> > >> Source
> > >> >>>
> > >> >> )
> > >> >>
> > >> >>>         at com/ibm/icu4jni/converters/NativeConverter.<clinit>
> > >> (Unknown Sou
> > >> >>>
> > >> >> rce)
> > >> >>
> > >> >>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
> > >> >>>         at java/lang/J9VMInternals.initialize
> > >> (J9VMInternals.java:185)
> > >> >>>         at
> > com/ibm/icu4jni/charset/CharsetProviderICU.charsetForName
> > >> (Unkno
> > >> >>>
> > >> >> wn Source)
> > >> >>
> > >> >>>         at java/nio/charset/Charset.forNameInternal
> > >> (Charset.java:450)
> > >> >>>         at java/nio/charset/Charset.forName (Charset.java:513)
> > >> >>>         at java/lang/String.defaultCharset ( String.java:688)
> > >> >>>         at java/lang/String.<init> (String.java:252)
> > >> >>>         at org/apache/harmony/luni/util/Util.toString (Util.java
> > :74)
> > >> >>>         at java/lang/System.getPropertyList (Native Method)
> > >> >>>         at java/lang/System.ensureProperties (System.java:496)
> > >> >>>         at java/lang/System.<clinit> (System.java:55)
> > >> >>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
> > >> >>>         at java/lang/J9VMInternals.initialize
> > >> (J9VMInternals.java:185)
> > >> >>>         at java/lang/ClassLoader.initializeClassLoaders (
> > >> ClassLoader.java:6
> > >> >>>
> > >> >> 5)
> > >> >>
> > >> >>>         at java/lang/Thread.initialize (Thread.java:301)
> > >> >>>         at java/lang/Thread.<init> (Thread.java :120)
> > >> >>> JVMJ9VM015W Initialization error for library jclclear_23(14):
> > >> JVMJ9VM009E J
> > >> >>>
> > >> >> 9VMDllMain failed
> > >> >>
> > >> >>> HMYEXEL062E Internal VM error: Failed to create Java VM
> > >> >>> FAILED.
> > >> >>>
> > >> >>>
> > >> >>> On 1 June 2006 at 22:26, "Mikhail Loenko" < mloenko@gmail.com >
> > >> wrote:
> > >> >>>
> > >> >>>
> > >> >>>> 2006/6/1, Geir Magnusson Jr < geir@pobox.com>:
> > >> >>>>
> > >> >>>>
> > >> >>>>> Mikhail Loenko wrote:
> > >> >>>>>
> > >> >>>>>
> > >> >>>>>> 2006/6/1, Geir Magnusson Jr < geir@pobox.com >:
> > >> >>>>>>
> > >> >>>>>>
> > >> >>>>>>> I don't quite grok what you mean - could you post what you
> are
> > >> >>>>>>> suggesting in code here on -dev?
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>> Index:
> > >> modules/luni/src/test/java/tests/api/java/net/Inet6AddressTest.ja
> > >> >>>>>>
> > >> >> v
> > >> >>
> > >> >>>>>>
> > >> >>>>>>
> > >> >>>> a
> > >> >>>>
> > >> >>>>
> > >> >>>>>>
> > >> ===================================================================
> > >> >>>>>> ---
> > >> >>>>>>
> > >> modules/luni/src/test/java/tests/api/java/net/Inet6AddressTest.java
> > >> >>>>>> (revision
> > >> >>>>>> 410480)
> > >> >>>>>> +++
> > >> >>>>>>
> > >> modules/luni/src/test/java/tests/api/java/net/Inet6AddressTest.java
> > >>
> > >> >>>>>> (working
> > >> >>>>>> copy)
> > >> >>>>>> @@ -785,7 +785,7 @@
> > >> >>>>>>                 "FFFF:FFFF:FFFF:FFFF:FFFF:FFFF:
> 255.255.255.255
> > ",
> > >> >>>>>>                 "0:0:0:0:0:0:0:0", "0:0:0:0:0:0: 0.0.0.0" };
> > >> >>>>>>
> > >> >>>>>> -        String invalidIPAddresses[] = { ":", "FFFF:FFFF",
> > "1:1",
> > >> >>>>>> "::1.2.3.444" };
> > >> >>>>>> +        String invalidIPAddresses[] = { "FFFF:FFFF" };
> > >> >>>>>>
> > >> >>>>>>         for (int i = 0; i < validIPAddresses.length; i++) {
> > >> >>>>>>
> > >> >>>>>>
> > >> >>>>>>
> > >> >>>>>>
> > >> >>>>> Thx - why do you want to remove testing the others?  Because
> you
> > >> believe
> > >> >>>>> that it is in effect testing the resolver in the OS since each
> > >> returns
> > >> >>>>> the same answer,and thus it's testing the same thing over and
> > >> over?
> > >> >>>>>
> > >> >>>>>
> > >> >>>> I believe so, but I'd like to hear confirmation from code
> authors
> > >> >>>>
> > >> >>>> Thanks,
> > >> >>>> Mikhail
> > >> >>>>
> > >> >>>>
> > >> >>>
> > >> >>>
> > >> ---------------------------------------------------------------------
> > >> >>> Terms of use : http://incubator.apache.org/harmony/mailing.html
> > >> >>> To unsubscribe, e-mail: harmony-dev-
> unsubscribe@incubator.apache.org
> >
> > >> >>> For additional commands, e-mail:
> > >> harmony-dev-help@incubator.apache.org
> > >> >>>
> > >> >>>
> > >> >>>
> > >> >>>
> > >> >> --
> > >> >> Paulex Yang
> > >> >> China Software Development Lab
> > >> >> IBM
> > >> >>
> > >> >>
> > >> >>
> > >> >>
> > ---------------------------------------------------------------------
> > >> >> Terms of use : http://incubator.apache.org/harmony/mailing.html
> > >> >> To unsubscribe, e-mail: harmony-dev-
> unsubscribe@incubator.apache.org
> > >> >> For additional commands, e-mail:
> > >> harmony-dev-help@incubator.apache.org
> > >> >>
> > >> >
> > >> >
> > >> >
> > >> > -------------------------------------------------------------------
> --
> >
> > >> > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > >> > To unsubscribe, e-mail: harmony-dev-
> unsubscribe@incubator.apache.org
> > >> > For additional commands, e-mail:
> > harmony-dev-help@incubator.apache.org
> > >> >
> > >> >
> > >> >
> > >>
> > >>
> > >> --
> > >> Paulex Yang
> > >> China Software Development Lab
> > >> IBM
> > >>
> > >>
> > >>
> > >> ---------------------------------------------------------------------
> > >> Terms of use : http://incubator.apache.org/harmony/mailing.html
> > >> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > >> For additional commands, e-mail: harmony-dev-
> help@incubator.apache.org
> > >>
> > >>
> > >
> >
> >
> > --
> > Paulex Yang
> > China Software Development Lab
> > IBM
> >
> >
> >
> > ---------------------------------------------------------------------
> > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >
> >


---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org


Re: [classlib] [testing] Coverage (was Re: 37% of total test execution time is spent in a single test)

Posted by Tim Ellison <t....@gmail.com>.
Mikhail Loenko wrote:
> Great!

Yep, that's great.

> Is it possible to see uncovered classes/methods/lines?

There is a link at the top of the wiki page to show details.

> Thanks,
> Mikhail
> 
> P.S. I think java.awt does not currently belong to beans module,
> though who knows what the future may bring us :)

That's not funny ;-)

Regards,
Tim

-- 

Tim Ellison (t.p.ellison@gmail.com)
IBM Java technology centre, UK.

---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org


Re: [classlib] [testing] Coverage (was Re: 37% of total test execution time is spent in a single test)

Posted by Vladimir Ivanov <iv...@gmail.com>.
I've update the web site by information about auth, crypto and rmi modules.
The exclude list also was added.
I'm not sure that all tests were run correctly so I'm going to verify test
runs.

 Thanks,
  Vladimir


On 6/13/06, Vladimir Ivanov <iv...@gmail.com> wrote:
>
>   > One more thing I've missed: what is the exclude list you've finally
> ended with?
> I'm going to publish it ASAP.
>
>  > > P.S. I think java.awt does not currently belong to beans module,
> > > though who knows what the future may bring us :)
>
> In fact, coverage information is gathered for each jar, that, to some
> extend corresponds to modules content.
> Some awt classes are in beans.jar, that is why they appeared in coverage
> statistics for beans module.
>
>   Thanks,
>   Vladimir
>
> On 6/13/06, Mikhail Loenko <ml...@gmail.com> wrote:
>
> > Cool! Sorry, I've missed it
> >
> > One more thing I've missed: what is the exclude list you've finally
> > ended with?
> >
> > Thanks,
> > Mikhail
> >
> > 2006/6/13, Vladimir Ivanov <iv...@gmail.com>:
> > > The detailed info available at   http://viv.byethost15.com/ (this
> > address
> > > also pointed on wiki-page).
> > > Do you need additional information ?
> > >
> > > Thanks,
> > >  Vladimir
> > >
> > >
> > > On 6/13/06, Mikhail Loenko < mloenko@gmail.com > wrote:
> > > >
> > > > Great!
> > > >
> > > > Is it possible to see uncovered classes/methods/lines?
> > > >
> > > > Thanks,
> > > > Mikhail
> > > >
> > > > P.S. I think java.awt does not currently belong to beans module,
> > > > though who knows what the future may bring us :)
> > > >
> > > > 2006/6/13, Vladimir Ivanov < ivavladimir@gmail.com>:
> > > > > Latest Harmony API source coverage by Harmony API unit tests
> > results I
> > > > > stored at wiki page
> > > > >
> > > > > http://wiki.apache.org/harmony/Coverage_information
> > > > >
> > > > >
> > > > >
> > > > > I'm going to refresh it bi-weekly (seems, it is enough for
> > coverage).
> > > > >
> > > > >
> > > > >
> > > > > >I think we have got a agreement on the test naming convention[1],
> > but
> > > > > >for sure, there have been many (legacy) test cases before this
> > > > > >agreement, and I think the volunteer is highly welcome to provide
> > patch
> > > > > >for them.
> > > > > >
> > > > >
> > > >
> > >[1]http://incubator.apache.org/harmony/subcomponents/classlibrary/testing.html
> > > > >
> > > > > >
> > > > >
> > > > > If nobody objects I'm going to look through the unit tests to
> > correct
> > > > > package names according to the agreement (where needed).
> > > > >
> > > > >
> > > > >
> > > > >  Thanks,
> > > > >
> > > > >   Vladimir
> > > > >
> > > > >
> > > > >
> > > > > On 6/8/06, Paulex Yang < paulex.yang@gmail.com > wrote:
> > > > > >
> > > > > > Vladimir,
> > > > > >
> > > > > > Vladimir Ivanov wrote:
> > > > > > > Thanks Paulex!
> > > > > > >
> > > > > > > I did the same, but could not send results due to spam filter
> > J
> > > > > > > Observations:
> > > > > > >
> > > > > > >   1. Coverage results look pretty much similar.
> > > > > > >   2. Exclude list looks pretty much similar too, but, looks
> > like it
> > > > > > >   depends on the way of data collection (I didn't run ant task
> > and
> > > > the
> > > > > > > list is
> > > > > > >   a little bit different).
> > > > > > Great.
> > > > > > >
> > > > > > > In any case, I think, when we run harmony on another VM
> > exclude list
> > > > > > will
> > > > > > > have to be updated.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > May be we can start publishing the coverage information on
> > wiki
> > > > pages
> > > > > > and
> > > > > > > provide some updates time to time (I can do it)?
> > > > > > >
> > > > > > +1,  and of course, you can only if  no one in the mailing list
> > > > objects
> > > > > > and you'll have my welcome, and I think it will be even greater
> > if
> > > > these
> > > > > > reports can be generated regularly like what JAPI is doing:)
> > > > > > >
> > > > > > >
> > > > > > > One note:
> > > > > > >
> > > > > > > I noticed that different unit tests have very different
> > package
> > > > names
> > > > > > >
> > > > > > > Now the directory with all built tests copied to one place
> > looks
> > > > like:
> > > > > > I think we have got a agreement on the test naming
> > convention[1], but
> > > > > > for sure, there have been many (legacy) test cases before this
> > > > > > agreement, and I think the volunteer is highly welcome to
> > provide
> > > > patch
> > > > > > for them.
> > > > > >
> > > > > >
> > > >
> > [1]http://incubator.apache.org/harmony/subcomponents/classlibrary/testing.html
> >
> > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > C:\coverage\tests\test>ls
> > > > > > >
> > > > > > > GZIPOutClose2.txt        api           config         javax
> > > > > > > tests
> > > > > > >
> > > > > > > GZIPOutFinish.txt        api.injected  dazzle
> > > > org            xml
> > > > > >
> > > > > > >
> > > > > > > GZIPOutWrite.txt         binary        gif            prefs
> > > > > > >
> > > > > > > Inet6Address.golden.ser  bundles       impl
> > serialization
> > > > > > >
> > > > > > > JDK2-3gabba.zip          com           impl.injected
> > test.txt
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > I think, it would be good if tests had unified package names.
> > > > > > >
> > > > > > > Why? – so far, just common sense, just to have an order in
> > test
> > > > suite
> > > > > > >
> > > > > > > Organization (if consider all unit tests as solid test suite).
> >
> > > > > > >
> > > > > > > Thanks,
> > > > > > >  Vladimir
> > > > > > >
> > > > > > > For example, my exclude list for java.io is:
> > > > > > > -java.io.BufferedInputStream ,
> > > > > > > -java.io.BufferedOutputStream,
> > > > > > > -java.io.File ,
> > > > > > > -java.io.FileChannelFactory ,
> > > > > > > -java.io.FileDescriptor,
> > > > > > > -java.io.FileInputStream,
> > > > > > > -java.io.FileOutputStream,
> > > > > > > -java.io.FilterInputStream ,
> > > > > > > -java.io.FilterOutputStream,
> > > > > > > - java.io.InputStream,
> > > > > > > -java.io.OutputStream,
> > > > > > > -java.io.ObjectStreamField,
> > > > > > > - java.io.PrintStream
> > > > > > >
> > > > > > >
> > > > > > > On 6/6/06, Paulex Yang < paulex.yang@gmail.com > wrote:
> > > > > > >>
> > > > > > >> I've attach the scripts and excluded class lists to
> > JIRA,  please
> > > > refer
> > > > > > >> to https://issues.apache.org/jira/browse/HARMONY-564 .  Enjoy
> > it:).
> > > > > > >>
> > > > > > >> Mark Hindess wrote:
> > > > > > >> > On 2 June 2006 at 10:37, Paulex Yang <
> > paulex.yang@gmail.com>
> > > > wrote:
> > > > > > >> >
> > > > > > >> >> Mark,
> > > > > > >> >>
> > > > > > >> >> I'm glad that there is someone else has interest on emma,
> > I've
> > > > > > >> tried it
> > > > > > >> >> before. AFAIK, emma works by instrumentation, but
> > sometimes for
> > > > > > >> classes
> > > > > > >> >> in bootclasspath, the instrumentation cannot work, there
> > are two
> > > > > > >> cases:
> > > > > > >>
> > > > > > >> >> 1. Some instrumented classes cannot be loaded by VM.
> > > > > > >> >> 2. Some classes cannot be instrumented
> > > > > > >> >>
> > > > > > >> >> I have tried to look more inside to find some way to work
> > around
> > > > > > >> but I
> > > > > > >> >> haven't got enough time yet.
> > > > > > >> >>
> > > > > > >> >> Specifically for nio-channel module, I had a list for
> > these two
> > > > > > cases
> > > > > > >> (I
> > > > > > >> >> believe the data is a little outdated and should be
> > reevaluated)
> > > > > > >> >> case 1.
> > > > > > >> >> BaseByteBuffer.class
> > > > > > >> >> Buffer.class
> > > > > > >> >> BufferFactory.class
> > > > > > >> >> ByteBuffer.class
> > > > > > >> >> CharArrayBuffer.class
> > > > > > >> >> CharBuffer.class
> > > > > > >> >> HeapByteBuffer.class
> > > > > > >> >> ReadWriteCharArrayBuffer.class
> > > > > > >> >> ReadWriteHeapByteBuffer.class
> > > > > > >> >> FileChannel.class
> > > > > > >> >> AbstractInterruptibleChannel.class
> > > > > > >> >> FileChannelImpl.class
> > > > > > >> >> WriteOnlyFileChannel.class
> > > > > > >> >> LockManager.class
> > > > > > >> >> LockManager$1.class
> > > > > > >> >> ReadOnlyFileChannel.class
> > > > > > >> >>
> > > > > > >> >> case 2:
> > > > > > >> >> ByteChannel.class
> > > > > > >> >> Channel.class
> > > > > > >> >> GatheringByteChannel.class
> > > > > > >> >> InterruptibleChannel.class
> > > > > > >> >> WritableByteChannel.class
> > > > > > >> >>
> > > > > > >> >> And I have got some ant script and more excluded list for
> > emma,
> > > > if
> > > > > > >> >> anyone has interests, I can upload it to JIRA.
> > > > > > >> >>
> > > > > > >> >
> > > > > > >> > Yes!
> > > > > > >> >
> > > > > > >> > -Mark.
> > > > > > >> >
> > > > > > >> >
> > > > > > >> >> Mark Hindess wrote:
> > > > > > >> >>
> > > > > > >> >>> Anyone tried using emma (emma.sf.net) to look at test
> > coverage
> > > > > > >> for our
> > > > > > >> >>> tests?  That might help ensure we don't trim them down
> > too
> > > > much?
> > > > > > >> >>>
> > > > > > >> >>> I suspect we will soon reach a point where there are
> > > > significantly
> > > > > > >> fewer
> > > > > > >> >>> obvious opportunities for small contributions - e.g.
> > > > > > >> 1.5generification,
> > > > > > >> >>> etc from looking at JAPI reports.  Having test coverage
> > reports
> > > > > > >> might
> > > > > > >> >>> help by highlighting areas for improvement.
> > > > > > >> >>>
> > > > > > >> >>> I had a quick try with:
> > > > > > >> >>>
> > > > > > >> >>>   ant -f make/build.xml
> > > > > > >> >>>   ant -f modules/nio/make/build.xml test
> > > > > > >> >>>   java -cp emma.jar emma instr -m overwrite -cp
> > > > > > >> deploy/jdk/jre/lib/boot/nio
> > > > > > >> >>>
> > > > > > >> >> .jar
> > > > > > >> >>
> > > > > > >> >>> and then:
> > > > > > >> >>>
> > > > > > >> >>>   java -Xbootclasspath/p: emma.jar \
> > > > > > >> >>>        -cp build/tests:modules/nio/bin/test: junit.jar:.
> > \
> > > > > > >> >>>        junit.textui.TestRunner \
> > > > > > >> >>>
> > > > org.apache.harmony.tests.java.nio.channels.SocketChannelTest
> > > > > > >> >>>
> > > > > > >> >>> (Running all the tests is a little tricky because test
> > depends
> > > > on
> > > > > > >> build,
> > > > > > >> >>> which depends on clean which wipes out the instr'd jar.
> > ;-( )
> > > > > > >> >>>
> > > > > > >> >>> Sadly it dies horribly with the stack trace below.  I'll
> > take
> > > > > > >> another
> > > > > > >> >>> look when I have a little more time, but I thought I'd
> > mention
> > > > it
> > > > > > in
> > > > > > >> >>> case anyone else had any thoughts on the subject.
> > > > > > >> >>>
> > > > > > >> >>> Regards,
> > > > > > >> >>>
> > > > > > >> >>> Mark.
> > > > > > >> >>>
> > > > > > >> >>> Exception in thread "main"
> > > > java/lang/ExceptionInInitializerError
> > > > > > >> >>>         at java/lang/J9VMInternals.initialize
> > > > > > >> (J9VMInternals.java:203)
> > > > > > >> >>>         at java/nio/Buffer.$VRi ( Buffer.java:4294967295)
> > > > > > >> >>>         at java/nio/Buffer.<clinit> ( Buffer.java
> > :4294967295)
> > > > > > >> >>>         at java/lang/J9VMInternals.initializeImpl (Native
> >
> > > > Method)
> > > > > > >> >>>         at java/lang/J9VMInternals.initialize (
> > > > J9VMInternals.java
> > > > > > >> :185)
> > > > > > >> >>>         at java/lang/J9VMInternals.initialize
> > > > > > >> (J9VMInternals.java:154)
> > > > > > >> >>>         at java/lang/String.getBytes (String.java:783)
> > > > > > >> >>>         at org/apache/harmony/luni/util/Util.getBytes (
> > > > > > >> Util.java:54)
> > > > > > >> >>>         at java/lang/ClassLoader.loadLibraryWithPath
> > > > > > >> (ClassLoader.java
> > > > > > >> :998)
> > > > > > >> >>>         at
> > java/lang/ClassLoader.loadLibraryWithClassLoader (
> > > > > > >> ClassLoader.ja
> > > > > > >> >>>
> > > > > > >> >> va:970)
> > > > > > >> >>
> > > > > > >> >>>         at java/lang/System.loadLibrary (System.java:706)
> > > > > > >> >>>         at
> > com/ibm/icu4jni/common/ICU4JNILoader.loadLibrary
> > > > > > (Unknown
> > > > > > >> Source
> > > > > > >> >>>
> > > > > > >> >> )
> > > > > > >> >>
> > > > > > >> >>>         at
> > com/ibm/icu4jni/converters/NativeConverter.<clinit>
> > > > > > >> (Unknown Sou
> > > > > > >> >>>
> > > > > > >> >> rce)
> > > > > > >> >>
> > > > > > >> >>>         at java/lang/J9VMInternals.initializeImpl (Native
> >
> > > > Method)
> > > > > > >> >>>         at java/lang/J9VMInternals.initialize
> > > > > > >> ( J9VMInternals.java:185)
> > > > > > >> >>>         at
> > > > > > com/ibm/icu4jni/charset/CharsetProviderICU.charsetForName
> > > > > > >> (Unkno
> > > > > > >> >>>
> > > > > > >> >> wn Source)
> > > > > > >> >>
> > > > > > >> >>>         at java/nio/charset/Charset.forNameInternal
> > > > > > >> (Charset.java:450)
> > > > > > >> >>>         at java/nio/charset/Charset.forName (
> > Charset.java:513)
> > > > > > >> >>>         at java/lang/String.defaultCharset ( String.java
> > :688)
> > > > > > >> >>>         at java/lang/String.<init> ( String.java :252)
> > > > > > >> >>>         at org/apache/harmony/luni/util/Util.toString (
> > > > Util.java
> > > > > > :74)
> > > > > > >> >>>         at java/lang/System.getPropertyList (Native
> > Method)
> > > > > > >> >>>         at java/lang/System.ensureProperties (
> > System.java:496)
> > > > > > >> >>>         at java/lang/System.<clinit> (System.java:55)
> > > > > > >> >>>         at java/lang/J9VMInternals.initializeImpl (Native
> >
> > > > Method)
> > > > > > >> >>>         at java/lang/J9VMInternals.initialize
> > > > > > >> (J9VMInternals.java:185)
> > > > > > >> >>>         at java/lang/ClassLoader.initializeClassLoaders (
> >
> > > > > > >> ClassLoader.java:6
> > > > > > >> >>>
> > > > > > >> >> 5)
> > > > > > >> >>
> > > > > > >> >>>         at java/lang/Thread.initialize ( Thread.java:301)
> > > > > > >> >>>         at java/lang/Thread.<init> (Thread.java:120)
> > > > > > >> >>> java/lang/NullPointerException
> > > > > > >> >>>         at java/lang/Throwable.printStackTrace (
> > Throwable.java
> > > > > > :231)
> > > > > > >> >>>         at
> > > > com/vladium/emma/rt/RTExitHook.createClassLoaderClosure
> > > > > > >> (RTExitH
> > > > > > >> >>>
> > > > > > >> >> ook.java:63)
> > > > > > >> >>
> > > > > > >> >>>         at com/vladium/emma/rt/RT.reset ( RT.java:92)
> > > > > > >> >>>         at com/vladium/emma/rt/RT.<clinit> ( RT.java:238)
> > > > > > >> >>>         at java/lang/J9VMInternals.initializeImpl (Native
> > > > Method)
> > > > > > >> >>>         at java/lang/J9VMInternals.initialize (
> > > > J9VMInternals.java
> > > > > > >> :185)
> > > > > > >> >>>         at java/nio/Buffer.$VRi ( Buffer.java:4294967295)
> > > > > > >> >>>         at java/nio/Buffer.<clinit> ( Buffer.java
> > :4294967295)
> > > > > > >> >>>         at java/lang/J9VMInternals.initializeImpl (Native
> > > > Method)
> > > > > > >> >>>         at java/lang/J9VMInternals.initialize
> > > > > > >> (J9VMInternals.java:185)
> > > > > > >> >>>         at java/lang/J9VMInternals.initialize
> > > > > > >> (J9VMInternals.java:154)
> > > > > > >> >>>         at java/lang/String.getBytes ( String.java :783)
> > > > > > >> >>>         at org/apache/harmony/luni/util/Util.getBytes (
> > > > Util.java
> > > > > > :54)
> > > > > > >> >>>         at java/lang/ClassLoader.loadLibraryWithPath
> > > > > > >> (ClassLoader.java
> > > > > > >> :998)
> > > > > > >> >>>         at
> > java/lang/ClassLoader.loadLibraryWithClassLoader (
> > > > > > >> ClassLoader.ja
> > > > > > >> >>>
> > > > > > >> >> va:970)
> > > > > > >> >>
> > > > > > >> >>>         at java/lang/System.loadLibrary ( System.java
> > :706)
> > > > > > >> >>>         at
> > com/ibm/icu4jni/common/ICU4JNILoader.loadLibrary
> > > > > > (Unknown
> > > > > > >> Source
> > > > > > >> >>>
> > > > > > >> >> )
> > > > > > >> >>
> > > > > > >> >>>         at
> > com/ibm/icu4jni/converters/NativeConverter.<clinit>
> > > > > > >> (Unknown Sou
> > > > > > >> >>>
> > > > > > >> >> rce)
> > > > > > >> >>
> > > > > > >> >>>         at java/lang/J9VMInternals.initializeImpl (Native
> >
> > > > Method)
> > > > > > >> >>>         at java/lang/J9VMInternals.initialize
> > > > > > >> (J9VMInternals.java:185)
> > > > > > >> >>>         at
> > > > > > com/ibm/icu4jni/charset/CharsetProviderICU.charsetForName
> > > > > > >> (Unkno
> > > > > > >> >>>
> > > > > > >> >> wn Source)
> > > > > > >> >>
> > > > > > >> >>>         at java/nio/charset/Charset.forNameInternal
> > > > > > >> (Charset.java:450)
> > > > > > >> >>>         at java/nio/charset/Charset.forName (
> > Charset.java:513)
> > > > > > >> >>>         at java/lang/String.defaultCharset ( String.java
> > :688)
> > > > > > >> >>>         at java/lang/String.<init> (String.java :252)
> > > > > > >> >>>         at org/apache/harmony/luni/util/Util.toString (
> > > > Util.java
> > > > > > :74)
> > > > > > >> >>>         at java/lang/System.getPropertyList (Native
> > Method)
> > > > > > >> >>>         at java/lang/System.ensureProperties (System.java
> > :496)
> > > > > > >> >>>         at java/lang/System.<clinit> (System.java:55)
> > > > > > >> >>>         at java/lang/J9VMInternals.initializeImpl (Native
> >
> > > > Method)
> > > > > > >> >>>         at java/lang/J9VMInternals.initialize
> > > > > > >> (J9VMInternals.java:185)
> > > > > > >> >>>         at java/lang/ClassLoader.initializeClassLoaders (
> >
> > > > > > >> ClassLoader.java:6
> > > > > > >> >>>
> > > > > > >> >> 5)
> > > > > > >> >>
> > > > > > >> >>>         at java/lang/Thread.initialize ( Thread.java:301)
> > > > > > >> >>>         at java/lang/Thread.<init> (Thread.java :120)
> > > > > > >> >>> JVMJ9VM015W Initialization error for library
> > jclclear_23(14):
> > > > > > >> JVMJ9VM009E J
> > > > > > >> >>>
> > > > > > >> >> 9VMDllMain failed
> > > > > > >> >>
> > > > > > >> >>> HMYEXEL062E Internal VM error: Failed to create Java VM
> > > > > > >> >>> FAILED.
> > > > > > >> >>>
> > > > > > >> >>>
> > > > > > >> >>> On 1 June 2006 at 22:26, "Mikhail Loenko" <
> > mloenko@gmail.com >
> > > > > > >> wrote:
> > > > > > >> >>>
> > > > > > >> >>>
> > > > > > >> >>>> 2006/6/1, Geir Magnusson Jr < geir@pobox.com>:
> > > > > > >> >>>>
> > > > > > >> >>>>
> > > > > > >> >>>>> Mikhail Loenko wrote:
> > > > > > >> >>>>>
> > > > > > >> >>>>>
> > > > > > >> >>>>>> 2006/6/1, Geir Magnusson Jr < geir@pobox.com >:
> > > > > > >> >>>>>>
> > > > > > >> >>>>>>
> > > > > > >> >>>>>>> I don't quite grok what you mean - could you post
> > what you
> > > > are
> > > > > > >> >>>>>>> suggesting in code here on -dev?
> > > > > > >> >>>>>>>
> > > > > > >> >>>>>>>
> > > > > > >> >>>>>> Index:
> > > > > > >>
> > modules/luni/src/test/java/tests/api/java/net/Inet6AddressTest.ja
> > > > > > >> >>>>>>
> > > > > > >> >> v
> > > > > > >> >>
> > > > > > >> >>>>>>
> > > > > > >> >>>>>>
> > > > > > >> >>>> a
> > > > > > >> >>>>
> > > > > > >> >>>>
> > > > > > >> >>>>>>
> > > > > > >>
> > ===================================================================
> > > > > > >> >>>>>> ---
> > > > > > >> >>>>>>
> > > > > > >>
> > modules/luni/src/test/java/tests/api/java/net/Inet6AddressTest.java
> > > > > > >> >>>>>> (revision
> > > > > > >> >>>>>> 410480)
> > > > > > >> >>>>>> +++
> > > > > > >> >>>>>>
> > > > > > >>
> > modules/luni/src/test/java/tests/api/java/net/Inet6AddressTest.java
> > > > > > >>
> > > > > > >> >>>>>> (working
> > > > > > >> >>>>>> copy)
> > > > > > >> >>>>>> @@ -785,7 +785,7 @@
> > > > > > >> >>>>>>                 "FFFF:FFFF:FFFF:FFFF:FFFF:FFFF:
> > > > 255.255.255.255
> > > > > > ",
> > > > > > >> >>>>>>                 "0:0:0:0:0:0:0:0", "0:0:0:0:0:0:
> > 0.0.0.0" };
> > > > > > >> >>>>>>
> > > > > > >> >>>>>> -        String invalidIPAddresses[] = { ":",
> > "FFFF:FFFF",
> > > > > > "1:1",
> > > > > > >> >>>>>> "::1.2.3.444" };
> > > > > > >> >>>>>> +        String invalidIPAddresses[] = { "FFFF:FFFF"
> > };
> > > > > > >> >>>>>>
> > > > > > >> >>>>>>         for (int i = 0; i < validIPAddresses.length;
> > i++) {
> > > > > > >> >>>>>>
> > > > > > >> >>>>>>
> > > > > > >> >>>>>>
> > > > > > >> >>>>>>
> > > > > > >> >>>>> Thx - why do you want to remove testing the
> > others?  Because
> > > > you
> > > > > > >> believe
> > > > > > >> >>>>> that it is in effect testing the resolver in the OS
> > since
> > > > each
> > > > > > >> returns
> > > > > > >> >>>>> the same answer,and thus it's testing the same thing
> > over and
> > > > > > >> over?
> > > > > > >> >>>>>
> > > > > > >> >>>>>
> > > > > > >> >>>> I believe so, but I'd like to hear confirmation from
> > code
> > > > authors
> > > > > > >> >>>>
> > > > > > >> >>>> Thanks,
> > > > > > >> >>>> Mikhail
> > > > > > >> >>>>
> > > > > > >> >>>>
> > > > > > >> >>>
> > > > > > >> >>>
> > > > > > >>
> > > >
> > ---------------------------------------------------------------------
> > > > > > >> >>> Terms of use :
> > http://incubator.apache.org/harmony/mailing.html
> > > > > > >> >>> To unsubscribe, e-mail:
> > > > harmony-dev-unsubscribe@incubator.apache.org
> > > > > >
> > > > > > >> >>> For additional commands, e-mail:
> > > > > > >> harmony-dev-help@incubator.apache.org
> > > > > > >> >>>
> > > > > > >> >>>
> > > > > > >> >>>
> > > > > > >> >>>
> > > > > > >> >> --
> > > > > > >> >> Paulex Yang
> > > > > > >> >> China Software Development Lab
> > > > > > >> >> IBM
> > > > > > >> >>
> > > > > > >> >>
> > > > > > >> >>
> > > > > > >> >>
> > > > > >
> > ---------------------------------------------------------------------
> > > > > > >> >> Terms of use : http://incubator.apache.org/harmony/mailing.html
> >
> > > > > > >> >> To unsubscribe, e-mail:
> > > > harmony-dev-unsubscribe@incubator.apache.org
> > > > > > >> >> For additional commands, e-mail:
> > > > > > >> harmony-dev-help@incubator.apache.org
> > > > > > >> >>
> > > > > > >> >
> > > > > > >> >
> > > > > > >> >
> > > > > > >> >
> > > >
> > ---------------------------------------------------------------------
> > > > > >
> > > > > > >> > Terms of use :
> > http://incubator.apache.org/harmony/mailing.html
> > > > > > >> > To unsubscribe, e-mail:
> > > > harmony-dev-unsubscribe@incubator.apache.org
> > > > > > >> > For additional commands, e-mail:
> > > > > > harmony-dev-help@incubator.apache.org
> > > > > > >> >
> > > > > > >> >
> > > > > > >> >
> > > > > > >>
> > > > > > >>
> > > > > > >> --
> > > > > > >> Paulex Yang
> > > > > > >> China Software Development Lab
> > > > > > >> IBM
> > > > > > >>
> > > > > > >>
> > > > > > >>
> > > > > > >>
> > > >
> > ---------------------------------------------------------------------
> > > > > > >> Terms of use :
> > http://incubator.apache.org/harmony/mailing.html
> > > > > > >> To unsubscribe, e-mail:
> > > > harmony-dev-unsubscribe@incubator.apache.org
> > > > > > >> For additional commands, e-mail:
> > > > harmony-dev-help@incubator.apache.org
> > > > > > >>
> > > > > > >>
> > > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Paulex Yang
> > > > > > China Software Development Lab
> > > > > > IBM
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > ---------------------------------------------------------------------
> > > > > > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > > > > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> >
> > > > > > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > >
> > > >
> > ---------------------------------------------------------------------
> > > > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > > > For additional commands, e-mail:
> > harmony-dev-help@incubator.apache.org
> > > >
> > > >
> > >
> > >
> >
> > ---------------------------------------------------------------------
> > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >
> >
>

Re: [classlib] [testing] Coverage (was Re: 37% of total test execution time is spent in a single test)

Posted by Vladimir Ivanov <iv...@gmail.com>.
> One more thing I've missed: what is the exclude list you've finally ended
with?
I'm going to publish it ASAP.

> > P.S. I think java.awt does not currently belong to beans module,
> > though who knows what the future may bring us :)

In fact, coverage information is gathered for each jar, that, to some extend
corresponds to modules content.
Some awt classes are in beans.jar, that is why they appeared in coverage
statistics for beans module.

 Thanks,
  Vladimir

On 6/13/06, Mikhail Loenko <ml...@gmail.com> wrote:

> Cool! Sorry, I've missed it
>
> One more thing I've missed: what is the exclude list you've finally ended
> with?
>
> Thanks,
> Mikhail
>
> 2006/6/13, Vladimir Ivanov <iv...@gmail.com>:
> > The detailed info available at  http://viv.byethost15.com/ (this address
> > also pointed on wiki-page).
> > Do you need additional information ?
> >
> > Thanks,
> >  Vladimir
> >
> >
> > On 6/13/06, Mikhail Loenko < mloenko@gmail.com> wrote:
> > >
> > > Great!
> > >
> > > Is it possible to see uncovered classes/methods/lines?
> > >
> > > Thanks,
> > > Mikhail
> > >
> > > P.S. I think java.awt does not currently belong to beans module,
> > > though who knows what the future may bring us :)
> > >
> > > 2006/6/13, Vladimir Ivanov < ivavladimir@gmail.com>:
> > > > Latest Harmony API source coverage by Harmony API unit tests results
> I
> > > > stored at wiki page
> > > >
> > > > http://wiki.apache.org/harmony/Coverage_information
> > > >
> > > >
> > > >
> > > > I'm going to refresh it bi-weekly (seems, it is enough for
> coverage).
> > > >
> > > >
> > > >
> > > > >I think we have got a agreement on the test naming convention[1],
> but
> > > > >for sure, there have been many (legacy) test cases before this
> > > > >agreement, and I think the volunteer is highly welcome to provide
> patch
> > > > >for them.
> > > > >
> > > >
> > >
> >[1]http://incubator.apache.org/harmony/subcomponents/classlibrary/testing.html
> > > >
> > > > >
> > > >
> > > > If nobody objects I'm going to look through the unit tests to
> correct
> > > > package names according to the agreement (where needed).
> > > >
> > > >
> > > >
> > > >  Thanks,
> > > >
> > > >   Vladimir
> > > >
> > > >
> > > >
> > > > On 6/8/06, Paulex Yang <paulex.yang@gmail.com > wrote:
> > > > >
> > > > > Vladimir,
> > > > >
> > > > > Vladimir Ivanov wrote:
> > > > > > Thanks Paulex!
> > > > > >
> > > > > > I did the same, but could not send results due to spam filter J
> > > > > > Observations:
> > > > > >
> > > > > >   1. Coverage results look pretty much similar.
> > > > > >   2. Exclude list looks pretty much similar too, but, looks like
> it
> > > > > >   depends on the way of data collection (I didn't run ant task
> and
> > > the
> > > > > > list is
> > > > > >   a little bit different).
> > > > > Great.
> > > > > >
> > > > > > In any case, I think, when we run harmony on another VM exclude
> list
> > > > > will
> > > > > > have to be updated.
> > > > > >
> > > > > >
> > > > > >
> > > > > > May be we can start publishing the coverage information on wiki
> > > pages
> > > > > and
> > > > > > provide some updates time to time (I can do it)?
> > > > > >
> > > > > +1,  and of course, you can only if  no one in the mailing list
> > > objects
> > > > > and you'll have my welcome, and I think it will be even greater if
>
> > > these
> > > > > reports can be generated regularly like what JAPI is doing:)
> > > > > >
> > > > > >
> > > > > > One note:
> > > > > >
> > > > > > I noticed that different unit tests have very different package
> > > names
> > > > > >
> > > > > > Now the directory with all built tests copied to one place looks
>
> > > like:
> > > > > I think we have got a agreement on the test naming convention[1],
> but
> > > > > for sure, there have been many (legacy) test cases before this
> > > > > agreement, and I think the volunteer is highly welcome to provide
> > > patch
> > > > > for them.
> > > > >
> > > > >
> > >
> [1]http://incubator.apache.org/harmony/subcomponents/classlibrary/testing.html
> > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > C:\coverage\tests\test>ls
> > > > > >
> > > > > > GZIPOutClose2.txt        api           config         javax
> > > > > > tests
> > > > > >
> > > > > > GZIPOutFinish.txt        api.injected  dazzle
> > > org            xml
> > > > >
> > > > > >
> > > > > > GZIPOutWrite.txt         binary        gif            prefs
> > > > > >
> > > > > > Inet6Address.golden.ser  bundles       impl
> serialization
> > > > > >
> > > > > > JDK2-3gabba.zip          com           impl.injected   test.txt
> > > > > >
> > > > > >
> > > > > >
> > > > > > I think, it would be good if tests had unified package names.
> > > > > >
> > > > > > Why? – so far, just common sense, just to have an order in test
> > > suite
> > > > > >
> > > > > > Organization (if consider all unit tests as solid test suite).
> > > > > >
> > > > > > Thanks,
> > > > > >  Vladimir
> > > > > >
> > > > > > For example, my exclude list for java.io is:
> > > > > > -java.io.BufferedInputStream ,
> > > > > > -java.io.BufferedOutputStream,
> > > > > > -java.io.File ,
> > > > > > -java.io.FileChannelFactory ,
> > > > > > -java.io.FileDescriptor,
> > > > > > -java.io.FileInputStream,
> > > > > > -java.io.FileOutputStream,
> > > > > > -java.io.FilterInputStream ,
> > > > > > -java.io.FilterOutputStream,
> > > > > > - java.io.InputStream,
> > > > > > -java.io.OutputStream,
> > > > > > -java.io.ObjectStreamField,
> > > > > > - java.io.PrintStream
> > > > > >
> > > > > >
> > > > > > On 6/6/06, Paulex Yang < paulex.yang@gmail.com > wrote:
> > > > > >>
> > > > > >> I've attach the scripts and excluded class lists to
> JIRA,  please
> > > refer
> > > > > >> to https://issues.apache.org/jira/browse/HARMONY-564 .  Enjoy
> it:).
> > > > > >>
> > > > > >> Mark Hindess wrote:
> > > > > >> > On 2 June 2006 at 10:37, Paulex Yang < paulex.yang@gmail.com>
> > > wrote:
> > > > > >> >
> > > > > >> >> Mark,
> > > > > >> >>
> > > > > >> >> I'm glad that there is someone else has interest on emma,
> I've
> > > > > >> tried it
> > > > > >> >> before. AFAIK, emma works by instrumentation, but sometimes
> for
> > > > > >> classes
> > > > > >> >> in bootclasspath, the instrumentation cannot work, there are
> two
> > > > > >> cases:
> > > > > >>
> > > > > >> >> 1. Some instrumented classes cannot be loaded by VM.
> > > > > >> >> 2. Some classes cannot be instrumented
> > > > > >> >>
> > > > > >> >> I have tried to look more inside to find some way to work
> around
> > > > > >> but I
> > > > > >> >> haven't got enough time yet.
> > > > > >> >>
> > > > > >> >> Specifically for nio-channel module, I had a list for these
> two
> > > > > cases
> > > > > >> (I
> > > > > >> >> believe the data is a little outdated and should be
> reevaluated)
> > > > > >> >> case 1.
> > > > > >> >> BaseByteBuffer.class
> > > > > >> >> Buffer.class
> > > > > >> >> BufferFactory.class
> > > > > >> >> ByteBuffer.class
> > > > > >> >> CharArrayBuffer.class
> > > > > >> >> CharBuffer.class
> > > > > >> >> HeapByteBuffer.class
> > > > > >> >> ReadWriteCharArrayBuffer.class
> > > > > >> >> ReadWriteHeapByteBuffer.class
> > > > > >> >> FileChannel.class
> > > > > >> >> AbstractInterruptibleChannel.class
> > > > > >> >> FileChannelImpl.class
> > > > > >> >> WriteOnlyFileChannel.class
> > > > > >> >> LockManager.class
> > > > > >> >> LockManager$1.class
> > > > > >> >> ReadOnlyFileChannel.class
> > > > > >> >>
> > > > > >> >> case 2:
> > > > > >> >> ByteChannel.class
> > > > > >> >> Channel.class
> > > > > >> >> GatheringByteChannel.class
> > > > > >> >> InterruptibleChannel.class
> > > > > >> >> WritableByteChannel.class
> > > > > >> >>
> > > > > >> >> And I have got some ant script and more excluded list for
> emma,
> > > if
> > > > > >> >> anyone has interests, I can upload it to JIRA.
> > > > > >> >>
> > > > > >> >
> > > > > >> > Yes!
> > > > > >> >
> > > > > >> > -Mark.
> > > > > >> >
> > > > > >> >
> > > > > >> >> Mark Hindess wrote:
> > > > > >> >>
> > > > > >> >>> Anyone tried using emma (emma.sf.net) to look at test
> coverage
> > > > > >> for our
> > > > > >> >>> tests?  That might help ensure we don't trim them down too
> > > much?
> > > > > >> >>>
> > > > > >> >>> I suspect we will soon reach a point where there are
> > > significantly
> > > > > >> fewer
> > > > > >> >>> obvious opportunities for small contributions - e.g.
> > > > > >> 1.5generification,
> > > > > >> >>> etc from looking at JAPI reports.  Having test coverage
> reports
> > > > > >> might
> > > > > >> >>> help by highlighting areas for improvement.
> > > > > >> >>>
> > > > > >> >>> I had a quick try with:
> > > > > >> >>>
> > > > > >> >>>   ant -f make/build.xml
> > > > > >> >>>   ant -f modules/nio/make/build.xml test
> > > > > >> >>>   java -cp emma.jar emma instr -m overwrite -cp
> > > > > >> deploy/jdk/jre/lib/boot/nio
> > > > > >> >>>
> > > > > >> >> .jar
> > > > > >> >>
> > > > > >> >>> and then:
> > > > > >> >>>
> > > > > >> >>>   java -Xbootclasspath/p: emma.jar \
> > > > > >> >>>        -cp build/tests:modules/nio/bin/test: junit.jar:. \
> > > > > >> >>>        junit.textui.TestRunner \
> > > > > >> >>>
> > > org.apache.harmony.tests.java.nio.channels.SocketChannelTest
> > > > > >> >>>
> > > > > >> >>> (Running all the tests is a little tricky because test
> depends
> > > on
> > > > > >> build,
> > > > > >> >>> which depends on clean which wipes out the instr'd jar. ;-(
> )
> > > > > >> >>>
> > > > > >> >>> Sadly it dies horribly with the stack trace below.  I'll
> take
> > > > > >> another
> > > > > >> >>> look when I have a little more time, but I thought I'd
> mention
> > > it
> > > > > in
> > > > > >> >>> case anyone else had any thoughts on the subject.
> > > > > >> >>>
> > > > > >> >>> Regards,
> > > > > >> >>>
> > > > > >> >>> Mark.
> > > > > >> >>>
> > > > > >> >>> Exception in thread "main"
> > > java/lang/ExceptionInInitializerError
> > > > > >> >>>         at java/lang/J9VMInternals.initialize
> > > > > >> (J9VMInternals.java:203)
> > > > > >> >>>         at java/nio/Buffer.$VRi ( Buffer.java:4294967295)
> > > > > >> >>>         at java/nio/Buffer.<clinit> ( Buffer.java
> :4294967295)
> > > > > >> >>>         at java/lang/J9VMInternals.initializeImpl (Native
> > > Method)
> > > > > >> >>>         at java/lang/J9VMInternals.initialize (
> > > J9VMInternals.java
> > > > > >> :185)
> > > > > >> >>>         at java/lang/J9VMInternals.initialize
> > > > > >> (J9VMInternals.java:154)
> > > > > >> >>>         at java/lang/String.getBytes (String.java:783)
> > > > > >> >>>         at org/apache/harmony/luni/util/Util.getBytes (
> > > > > >> Util.java:54)
> > > > > >> >>>         at java/lang/ClassLoader.loadLibraryWithPath
> > > > > >> (ClassLoader.java
> > > > > >> :998)
> > > > > >> >>>         at java/lang/ClassLoader.loadLibraryWithClassLoader
> (
> > > > > >> ClassLoader.ja
> > > > > >> >>>
> > > > > >> >> va:970)
> > > > > >> >>
> > > > > >> >>>         at java/lang/System.loadLibrary (System.java:706)
> > > > > >> >>>         at com/ibm/icu4jni/common/ICU4JNILoader.loadLibrary
>
> > > > > (Unknown
> > > > > >> Source
> > > > > >> >>>
> > > > > >> >> )
> > > > > >> >>
> > > > > >> >>>         at
> com/ibm/icu4jni/converters/NativeConverter.<clinit>
> > > > > >> (Unknown Sou
> > > > > >> >>>
> > > > > >> >> rce)
> > > > > >> >>
> > > > > >> >>>         at java/lang/J9VMInternals.initializeImpl (Native
> > > Method)
> > > > > >> >>>         at java/lang/J9VMInternals.initialize
> > > > > >> ( J9VMInternals.java:185)
> > > > > >> >>>         at
> > > > > com/ibm/icu4jni/charset/CharsetProviderICU.charsetForName
> > > > > >> (Unkno
> > > > > >> >>>
> > > > > >> >> wn Source)
> > > > > >> >>
> > > > > >> >>>         at java/nio/charset/Charset.forNameInternal
> > > > > >> (Charset.java:450)
> > > > > >> >>>         at java/nio/charset/Charset.forName ( Charset.java
> :513)
> > > > > >> >>>         at java/lang/String.defaultCharset ( String.java
> :688)
> > > > > >> >>>         at java/lang/String.<init> ( String.java :252)
> > > > > >> >>>         at org/apache/harmony/luni/util/Util.toString (
> > > Util.java
> > > > > :74)
> > > > > >> >>>         at java/lang/System.getPropertyList (Native Method)
>
> > > > > >> >>>         at java/lang/System.ensureProperties ( System.java
> :496)
> > > > > >> >>>         at java/lang/System.<clinit> (System.java:55)
> > > > > >> >>>         at java/lang/J9VMInternals.initializeImpl (Native
> > > Method)
> > > > > >> >>>         at java/lang/J9VMInternals.initialize
> > > > > >> (J9VMInternals.java:185)
> > > > > >> >>>         at java/lang/ClassLoader.initializeClassLoaders (
> > > > > >> ClassLoader.java:6
> > > > > >> >>>
> > > > > >> >> 5)
> > > > > >> >>
> > > > > >> >>>         at java/lang/Thread.initialize ( Thread.java:301)
> > > > > >> >>>         at java/lang/Thread.<init> (Thread.java:120)
> > > > > >> >>> java/lang/NullPointerException
> > > > > >> >>>         at java/lang/Throwable.printStackTrace (
> Throwable.java
> > > > > :231)
> > > > > >> >>>         at
> > > com/vladium/emma/rt/RTExitHook.createClassLoaderClosure
> > > > > >> (RTExitH
> > > > > >> >>>
> > > > > >> >> ook.java:63)
> > > > > >> >>
> > > > > >> >>>         at com/vladium/emma/rt/RT.reset ( RT.java:92)
> > > > > >> >>>         at com/vladium/emma/rt/RT.<clinit> ( RT.java:238)
> > > > > >> >>>         at java/lang/J9VMInternals.initializeImpl (Native
> > > Method)
> > > > > >> >>>         at java/lang/J9VMInternals.initialize (
> > > J9VMInternals.java
> > > > > >> :185)
> > > > > >> >>>         at java/nio/Buffer.$VRi ( Buffer.java:4294967295)
> > > > > >> >>>         at java/nio/Buffer.<clinit> ( Buffer.java
> :4294967295)
> > > > > >> >>>         at java/lang/J9VMInternals.initializeImpl (Native
> > > Method)
> > > > > >> >>>         at java/lang/J9VMInternals.initialize
> > > > > >> (J9VMInternals.java:185)
> > > > > >> >>>         at java/lang/J9VMInternals.initialize
> > > > > >> (J9VMInternals.java:154)
> > > > > >> >>>         at java/lang/String.getBytes ( String.java :783)
> > > > > >> >>>         at org/apache/harmony/luni/util/Util.getBytes (
> > > Util.java
> > > > > :54)
> > > > > >> >>>         at java/lang/ClassLoader.loadLibraryWithPath
> > > > > >> (ClassLoader.java
> > > > > >> :998)
> > > > > >> >>>         at java/lang/ClassLoader.loadLibraryWithClassLoader
> (
> > > > > >> ClassLoader.ja
> > > > > >> >>>
> > > > > >> >> va:970)
> > > > > >> >>
> > > > > >> >>>         at java/lang/System.loadLibrary ( System.java:706)
> > > > > >> >>>         at com/ibm/icu4jni/common/ICU4JNILoader.loadLibrary
> > > > > (Unknown
> > > > > >> Source
> > > > > >> >>>
> > > > > >> >> )
> > > > > >> >>
> > > > > >> >>>         at
> com/ibm/icu4jni/converters/NativeConverter.<clinit>
> > > > > >> (Unknown Sou
> > > > > >> >>>
> > > > > >> >> rce)
> > > > > >> >>
> > > > > >> >>>         at java/lang/J9VMInternals.initializeImpl (Native
> > > Method)
> > > > > >> >>>         at java/lang/J9VMInternals.initialize
> > > > > >> (J9VMInternals.java:185)
> > > > > >> >>>         at
> > > > > com/ibm/icu4jni/charset/CharsetProviderICU.charsetForName
> > > > > >> (Unkno
> > > > > >> >>>
> > > > > >> >> wn Source)
> > > > > >> >>
> > > > > >> >>>         at java/nio/charset/Charset.forNameInternal
> > > > > >> (Charset.java:450)
> > > > > >> >>>         at java/nio/charset/Charset.forName ( Charset.java
> :513)
> > > > > >> >>>         at java/lang/String.defaultCharset ( String.java
> :688)
> > > > > >> >>>         at java/lang/String.<init> (String.java :252)
> > > > > >> >>>         at org/apache/harmony/luni/util/Util.toString (
> > > Util.java
> > > > > :74)
> > > > > >> >>>         at java/lang/System.getPropertyList (Native Method)
>
> > > > > >> >>>         at java/lang/System.ensureProperties (System.java
> :496)
> > > > > >> >>>         at java/lang/System.<clinit> (System.java:55)
> > > > > >> >>>         at java/lang/J9VMInternals.initializeImpl (Native
> > > Method)
> > > > > >> >>>         at java/lang/J9VMInternals.initialize
> > > > > >> (J9VMInternals.java:185)
> > > > > >> >>>         at java/lang/ClassLoader.initializeClassLoaders (
> > > > > >> ClassLoader.java:6
> > > > > >> >>>
> > > > > >> >> 5)
> > > > > >> >>
> > > > > >> >>>         at java/lang/Thread.initialize ( Thread.java:301)
> > > > > >> >>>         at java/lang/Thread.<init> (Thread.java :120)
> > > > > >> >>> JVMJ9VM015W Initialization error for library
> jclclear_23(14):
> > > > > >> JVMJ9VM009E J
> > > > > >> >>>
> > > > > >> >> 9VMDllMain failed
> > > > > >> >>
> > > > > >> >>> HMYEXEL062E Internal VM error: Failed to create Java VM
> > > > > >> >>> FAILED.
> > > > > >> >>>
> > > > > >> >>>
> > > > > >> >>> On 1 June 2006 at 22:26, "Mikhail Loenko" <
> mloenko@gmail.com >
> > > > > >> wrote:
> > > > > >> >>>
> > > > > >> >>>
> > > > > >> >>>> 2006/6/1, Geir Magnusson Jr < geir@pobox.com>:
> > > > > >> >>>>
> > > > > >> >>>>
> > > > > >> >>>>> Mikhail Loenko wrote:
> > > > > >> >>>>>
> > > > > >> >>>>>
> > > > > >> >>>>>> 2006/6/1, Geir Magnusson Jr < geir@pobox.com >:
> > > > > >> >>>>>>
> > > > > >> >>>>>>
> > > > > >> >>>>>>> I don't quite grok what you mean - could you post what
> you
> > > are
> > > > > >> >>>>>>> suggesting in code here on -dev?
> > > > > >> >>>>>>>
> > > > > >> >>>>>>>
> > > > > >> >>>>>> Index:
> > > > > >>
> modules/luni/src/test/java/tests/api/java/net/Inet6AddressTest.ja
> > > > > >> >>>>>>
> > > > > >> >> v
> > > > > >> >>
> > > > > >> >>>>>>
> > > > > >> >>>>>>
> > > > > >> >>>> a
> > > > > >> >>>>
> > > > > >> >>>>
> > > > > >> >>>>>>
> > > > > >>
> ===================================================================
> > > > > >> >>>>>> ---
> > > > > >> >>>>>>
> > > > > >>
> modules/luni/src/test/java/tests/api/java/net/Inet6AddressTest.java
> > > > > >> >>>>>> (revision
> > > > > >> >>>>>> 410480)
> > > > > >> >>>>>> +++
> > > > > >> >>>>>>
> > > > > >>
> modules/luni/src/test/java/tests/api/java/net/Inet6AddressTest.java
> > > > > >>
> > > > > >> >>>>>> (working
> > > > > >> >>>>>> copy)
> > > > > >> >>>>>> @@ -785,7 +785,7 @@
> > > > > >> >>>>>>                 "FFFF:FFFF:FFFF:FFFF:FFFF:FFFF:
> > > 255.255.255.255
> > > > > ",
> > > > > >> >>>>>>                 "0:0:0:0:0:0:0:0", "0:0:0:0:0:0: 0.0.0.0"
> };
> > > > > >> >>>>>>
> > > > > >> >>>>>> -        String invalidIPAddresses[] = { ":",
> "FFFF:FFFF",
> > > > > "1:1",
> > > > > >> >>>>>> "::1.2.3.444" };
> > > > > >> >>>>>> +        String invalidIPAddresses[] = { "FFFF:FFFF" };
> > > > > >> >>>>>>
> > > > > >> >>>>>>         for (int i = 0; i < validIPAddresses.length;
> i++) {
> > > > > >> >>>>>>
> > > > > >> >>>>>>
> > > > > >> >>>>>>
> > > > > >> >>>>>>
> > > > > >> >>>>> Thx - why do you want to remove testing the
> others?  Because
> > > you
> > > > > >> believe
> > > > > >> >>>>> that it is in effect testing the resolver in the OS since
> > > each
> > > > > >> returns
> > > > > >> >>>>> the same answer,and thus it's testing the same thing over
> and
> > > > > >> over?
> > > > > >> >>>>>
> > > > > >> >>>>>
> > > > > >> >>>> I believe so, but I'd like to hear confirmation from code
> > > authors
> > > > > >> >>>>
> > > > > >> >>>> Thanks,
> > > > > >> >>>> Mikhail
> > > > > >> >>>>
> > > > > >> >>>>
> > > > > >> >>>
> > > > > >> >>>
> > > > > >>
> > > ---------------------------------------------------------------------
> > > > > >> >>> Terms of use :
> http://incubator.apache.org/harmony/mailing.html
> > > > > >> >>> To unsubscribe, e-mail:
> > > harmony-dev-unsubscribe@incubator.apache.org
> > > > >
> > > > > >> >>> For additional commands, e-mail:
> > > > > >> harmony-dev-help@incubator.apache.org
> > > > > >> >>>
> > > > > >> >>>
> > > > > >> >>>
> > > > > >> >>>
> > > > > >> >> --
> > > > > >> >> Paulex Yang
> > > > > >> >> China Software Development Lab
> > > > > >> >> IBM
> > > > > >> >>
> > > > > >> >>
> > > > > >> >>
> > > > > >> >>
> > > > >
> ---------------------------------------------------------------------
> > > > > >> >> Terms of use :
> http://incubator.apache.org/harmony/mailing.html
> > > > > >> >> To unsubscribe, e-mail:
> > > harmony-dev-unsubscribe@incubator.apache.org
> > > > > >> >> For additional commands, e-mail:
> > > > > >> harmony-dev-help@incubator.apache.org
> > > > > >> >>
> > > > > >> >
> > > > > >> >
> > > > > >> >
> > > > > >> >
> > > ---------------------------------------------------------------------
> > > > >
> > > > > >> > Terms of use :
> http://incubator.apache.org/harmony/mailing.html
> > > > > >> > To unsubscribe, e-mail:
> > > harmony-dev-unsubscribe@incubator.apache.org
> > > > > >> > For additional commands, e-mail:
> > > > > harmony-dev-help@incubator.apache.org
> > > > > >> >
> > > > > >> >
> > > > > >> >
> > > > > >>
> > > > > >>
> > > > > >> --
> > > > > >> Paulex Yang
> > > > > >> China Software Development Lab
> > > > > >> IBM
> > > > > >>
> > > > > >>
> > > > > >>
> > > > > >>
> > > ---------------------------------------------------------------------
> > > > > >> Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > > > >> To unsubscribe, e-mail:
> > > harmony-dev-unsubscribe@incubator.apache.org
> > > > > >> For additional commands, e-mail:
> > > harmony-dev-help@incubator.apache.org
> > > > > >>
> > > > > >>
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Paulex Yang
> > > > > China Software Development Lab
> > > > > IBM
> > > > >
> > > > >
> > > > >
> > > > >
> ---------------------------------------------------------------------
> > > > > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > > > To unsubscribe, e-mail:
> harmony-dev-unsubscribe@incubator.apache.org
> > > > > For additional commands, e-mail:
> harmony-dev-help@incubator.apache.org
> > > > >
> > > > >
> > > >
> > > >
> > >
> > > ---------------------------------------------------------------------
> > > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> > >
> > >
> >
> >
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>

Re: [classlib] [testing] Coverage (was Re: 37% of total test execution time is spent in a single test)

Posted by Mikhail Loenko <ml...@gmail.com>.
Cool! Sorry, I've missed it

One more thing I've missed: what is the exclude list you've finally ended with?

Thanks,
Mikhail

2006/6/13, Vladimir Ivanov <iv...@gmail.com>:
> The detailed info available at  http://viv.byethost15.com/ (this address
> also pointed on wiki-page).
> Do you need additional information ?
>
> Thanks,
>  Vladimir
>
>
> On 6/13/06, Mikhail Loenko <ml...@gmail.com> wrote:
> >
> > Great!
> >
> > Is it possible to see uncovered classes/methods/lines?
> >
> > Thanks,
> > Mikhail
> >
> > P.S. I think java.awt does not currently belong to beans module,
> > though who knows what the future may bring us :)
> >
> > 2006/6/13, Vladimir Ivanov <iv...@gmail.com>:
> > > Latest Harmony API source coverage by Harmony API unit tests results I
> > > stored at wiki page
> > >
> > > http://wiki.apache.org/harmony/Coverage_information
> > >
> > >
> > >
> > > I'm going to refresh it bi-weekly (seems, it is enough for coverage).
> > >
> > >
> > >
> > > >I think we have got a agreement on the test naming convention[1], but
> > > >for sure, there have been many (legacy) test cases before this
> > > >agreement, and I think the volunteer is highly welcome to provide patch
> > > >for them.
> > > >
> > >
> > >[1]http://incubator.apache.org/harmony/subcomponents/classlibrary/testing.html
> > >
> > > >
> > >
> > > If nobody objects I'm going to look through the unit tests to correct
> > > package names according to the agreement (where needed).
> > >
> > >
> > >
> > >  Thanks,
> > >
> > >   Vladimir
> > >
> > >
> > >
> > > On 6/8/06, Paulex Yang <paulex.yang@gmail.com > wrote:
> > > >
> > > > Vladimir,
> > > >
> > > > Vladimir Ivanov wrote:
> > > > > Thanks Paulex!
> > > > >
> > > > > I did the same, but could not send results due to spam filter J
> > > > > Observations:
> > > > >
> > > > >   1. Coverage results look pretty much similar.
> > > > >   2. Exclude list looks pretty much similar too, but, looks like it
> > > > >   depends on the way of data collection (I didn't run ant task and
> > the
> > > > > list is
> > > > >   a little bit different).
> > > > Great.
> > > > >
> > > > > In any case, I think, when we run harmony on another VM exclude list
> > > > will
> > > > > have to be updated.
> > > > >
> > > > >
> > > > >
> > > > > May be we can start publishing the coverage information on wiki
> > pages
> > > > and
> > > > > provide some updates time to time (I can do it)?
> > > > >
> > > > +1,  and of course, you can only if  no one in the mailing list
> > objects
> > > > and you'll have my welcome, and I think it will be even greater if
> > these
> > > > reports can be generated regularly like what JAPI is doing:)
> > > > >
> > > > >
> > > > > One note:
> > > > >
> > > > > I noticed that different unit tests have very different package
> > names
> > > > >
> > > > > Now the directory with all built tests copied to one place looks
> > like:
> > > > I think we have got a agreement on the test naming convention[1], but
> > > > for sure, there have been many (legacy) test cases before this
> > > > agreement, and I think the volunteer is highly welcome to provide
> > patch
> > > > for them.
> > > >
> > > >
> > [1]http://incubator.apache.org/harmony/subcomponents/classlibrary/testing.html
> > > >
> > > > >
> > > > >
> > > > >
> > > > > C:\coverage\tests\test>ls
> > > > >
> > > > > GZIPOutClose2.txt        api           config         javax
> > > > > tests
> > > > >
> > > > > GZIPOutFinish.txt        api.injected  dazzle
> > org            xml
> > > >
> > > > >
> > > > > GZIPOutWrite.txt         binary        gif            prefs
> > > > >
> > > > > Inet6Address.golden.ser  bundles       impl           serialization
> > > > >
> > > > > JDK2-3gabba.zip          com           impl.injected   test.txt
> > > > >
> > > > >
> > > > >
> > > > > I think, it would be good if tests had unified package names.
> > > > >
> > > > > Why? – so far, just common sense, just to have an order in test
> > suite
> > > > >
> > > > > Organization (if consider all unit tests as solid test suite).
> > > > >
> > > > > Thanks,
> > > > >  Vladimir
> > > > >
> > > > > For example, my exclude list for java.io is:
> > > > > -java.io.BufferedInputStream ,
> > > > > -java.io.BufferedOutputStream,
> > > > > -java.io.File ,
> > > > > -java.io.FileChannelFactory,
> > > > > -java.io.FileDescriptor,
> > > > > -java.io.FileInputStream,
> > > > > -java.io.FileOutputStream,
> > > > > -java.io.FilterInputStream ,
> > > > > -java.io.FilterOutputStream,
> > > > > - java.io.InputStream,
> > > > > -java.io.OutputStream,
> > > > > -java.io.ObjectStreamField,
> > > > > -java.io.PrintStream
> > > > >
> > > > >
> > > > > On 6/6/06, Paulex Yang < paulex.yang@gmail.com > wrote:
> > > > >>
> > > > >> I've attach the scripts and excluded class lists to JIRA,  please
> > refer
> > > > >> to https://issues.apache.org/jira/browse/HARMONY-564 .  Enjoy it:).
> > > > >>
> > > > >> Mark Hindess wrote:
> > > > >> > On 2 June 2006 at 10:37, Paulex Yang < paulex.yang@gmail.com>
> > wrote:
> > > > >> >
> > > > >> >> Mark,
> > > > >> >>
> > > > >> >> I'm glad that there is someone else has interest on emma, I've
> > > > >> tried it
> > > > >> >> before. AFAIK, emma works by instrumentation, but sometimes for
> > > > >> classes
> > > > >> >> in bootclasspath, the instrumentation cannot work, there are two
> > > > >> cases:
> > > > >>
> > > > >> >> 1. Some instrumented classes cannot be loaded by VM.
> > > > >> >> 2. Some classes cannot be instrumented
> > > > >> >>
> > > > >> >> I have tried to look more inside to find some way to work around
> > > > >> but I
> > > > >> >> haven't got enough time yet.
> > > > >> >>
> > > > >> >> Specifically for nio-channel module, I had a list for these two
> > > > cases
> > > > >> (I
> > > > >> >> believe the data is a little outdated and should be reevaluated)
> > > > >> >> case 1.
> > > > >> >> BaseByteBuffer.class
> > > > >> >> Buffer.class
> > > > >> >> BufferFactory.class
> > > > >> >> ByteBuffer.class
> > > > >> >> CharArrayBuffer.class
> > > > >> >> CharBuffer.class
> > > > >> >> HeapByteBuffer.class
> > > > >> >> ReadWriteCharArrayBuffer.class
> > > > >> >> ReadWriteHeapByteBuffer.class
> > > > >> >> FileChannel.class
> > > > >> >> AbstractInterruptibleChannel.class
> > > > >> >> FileChannelImpl.class
> > > > >> >> WriteOnlyFileChannel.class
> > > > >> >> LockManager.class
> > > > >> >> LockManager$1.class
> > > > >> >> ReadOnlyFileChannel.class
> > > > >> >>
> > > > >> >> case 2:
> > > > >> >> ByteChannel.class
> > > > >> >> Channel.class
> > > > >> >> GatheringByteChannel.class
> > > > >> >> InterruptibleChannel.class
> > > > >> >> WritableByteChannel.class
> > > > >> >>
> > > > >> >> And I have got some ant script and more excluded list for emma,
> > if
> > > > >> >> anyone has interests, I can upload it to JIRA.
> > > > >> >>
> > > > >> >
> > > > >> > Yes!
> > > > >> >
> > > > >> > -Mark.
> > > > >> >
> > > > >> >
> > > > >> >> Mark Hindess wrote:
> > > > >> >>
> > > > >> >>> Anyone tried using emma (emma.sf.net) to look at test coverage
> > > > >> for our
> > > > >> >>> tests?  That might help ensure we don't trim them down too
> > much?
> > > > >> >>>
> > > > >> >>> I suspect we will soon reach a point where there are
> > significantly
> > > > >> fewer
> > > > >> >>> obvious opportunities for small contributions - e.g.
> > > > >> 1.5generification,
> > > > >> >>> etc from looking at JAPI reports.  Having test coverage reports
> > > > >> might
> > > > >> >>> help by highlighting areas for improvement.
> > > > >> >>>
> > > > >> >>> I had a quick try with:
> > > > >> >>>
> > > > >> >>>   ant -f make/build.xml
> > > > >> >>>   ant -f modules/nio/make/build.xml test
> > > > >> >>>   java -cp emma.jar emma instr -m overwrite -cp
> > > > >> deploy/jdk/jre/lib/boot/nio
> > > > >> >>>
> > > > >> >> .jar
> > > > >> >>
> > > > >> >>> and then:
> > > > >> >>>
> > > > >> >>>   java -Xbootclasspath/p: emma.jar \
> > > > >> >>>        -cp build/tests:modules/nio/bin/test:junit.jar:. \
> > > > >> >>>        junit.textui.TestRunner \
> > > > >> >>>
> > org.apache.harmony.tests.java.nio.channels.SocketChannelTest
> > > > >> >>>
> > > > >> >>> (Running all the tests is a little tricky because test depends
> > on
> > > > >> build,
> > > > >> >>> which depends on clean which wipes out the instr'd jar. ;-( )
> > > > >> >>>
> > > > >> >>> Sadly it dies horribly with the stack trace below.  I'll take
> > > > >> another
> > > > >> >>> look when I have a little more time, but I thought I'd mention
> > it
> > > > in
> > > > >> >>> case anyone else had any thoughts on the subject.
> > > > >> >>>
> > > > >> >>> Regards,
> > > > >> >>>
> > > > >> >>> Mark.
> > > > >> >>>
> > > > >> >>> Exception in thread "main"
> > java/lang/ExceptionInInitializerError
> > > > >> >>>         at java/lang/J9VMInternals.initialize
> > > > >> (J9VMInternals.java:203)
> > > > >> >>>         at java/nio/Buffer.$VRi ( Buffer.java:4294967295)
> > > > >> >>>         at java/nio/Buffer.<clinit> ( Buffer.java:4294967295)
> > > > >> >>>         at java/lang/J9VMInternals.initializeImpl (Native
> > Method)
> > > > >> >>>         at java/lang/J9VMInternals.initialize (
> > J9VMInternals.java
> > > > >> :185)
> > > > >> >>>         at java/lang/J9VMInternals.initialize
> > > > >> (J9VMInternals.java:154)
> > > > >> >>>         at java/lang/String.getBytes (String.java:783)
> > > > >> >>>         at org/apache/harmony/luni/util/Util.getBytes (
> > > > >> Util.java:54)
> > > > >> >>>         at java/lang/ClassLoader.loadLibraryWithPath
> > > > >> (ClassLoader.java
> > > > >> :998)
> > > > >> >>>         at java/lang/ClassLoader.loadLibraryWithClassLoader (
> > > > >> ClassLoader.ja
> > > > >> >>>
> > > > >> >> va:970)
> > > > >> >>
> > > > >> >>>         at java/lang/System.loadLibrary (System.java:706)
> > > > >> >>>         at com/ibm/icu4jni/common/ICU4JNILoader.loadLibrary
> > > > (Unknown
> > > > >> Source
> > > > >> >>>
> > > > >> >> )
> > > > >> >>
> > > > >> >>>         at com/ibm/icu4jni/converters/NativeConverter.<clinit>
> > > > >> (Unknown Sou
> > > > >> >>>
> > > > >> >> rce)
> > > > >> >>
> > > > >> >>>         at java/lang/J9VMInternals.initializeImpl (Native
> > Method)
> > > > >> >>>         at java/lang/J9VMInternals.initialize
> > > > >> ( J9VMInternals.java:185)
> > > > >> >>>         at
> > > > com/ibm/icu4jni/charset/CharsetProviderICU.charsetForName
> > > > >> (Unkno
> > > > >> >>>
> > > > >> >> wn Source)
> > > > >> >>
> > > > >> >>>         at java/nio/charset/Charset.forNameInternal
> > > > >> (Charset.java:450)
> > > > >> >>>         at java/nio/charset/Charset.forName (Charset.java:513)
> > > > >> >>>         at java/lang/String.defaultCharset ( String.java:688)
> > > > >> >>>         at java/lang/String.<init> ( String.java:252)
> > > > >> >>>         at org/apache/harmony/luni/util/Util.toString (
> > Util.java
> > > > :74)
> > > > >> >>>         at java/lang/System.getPropertyList (Native Method)
> > > > >> >>>         at java/lang/System.ensureProperties ( System.java:496)
> > > > >> >>>         at java/lang/System.<clinit> (System.java:55)
> > > > >> >>>         at java/lang/J9VMInternals.initializeImpl (Native
> > Method)
> > > > >> >>>         at java/lang/J9VMInternals.initialize
> > > > >> (J9VMInternals.java:185)
> > > > >> >>>         at java/lang/ClassLoader.initializeClassLoaders (
> > > > >> ClassLoader.java:6
> > > > >> >>>
> > > > >> >> 5)
> > > > >> >>
> > > > >> >>>         at java/lang/Thread.initialize (Thread.java:301)
> > > > >> >>>         at java/lang/Thread.<init> (Thread.java:120)
> > > > >> >>> java/lang/NullPointerException
> > > > >> >>>         at java/lang/Throwable.printStackTrace ( Throwable.java
> > > > :231)
> > > > >> >>>         at
> > com/vladium/emma/rt/RTExitHook.createClassLoaderClosure
> > > > >> (RTExitH
> > > > >> >>>
> > > > >> >> ook.java:63)
> > > > >> >>
> > > > >> >>>         at com/vladium/emma/rt/RT.reset ( RT.java:92)
> > > > >> >>>         at com/vladium/emma/rt/RT.<clinit> (RT.java:238)
> > > > >> >>>         at java/lang/J9VMInternals.initializeImpl (Native
> > Method)
> > > > >> >>>         at java/lang/J9VMInternals.initialize (
> > J9VMInternals.java
> > > > >> :185)
> > > > >> >>>         at java/nio/Buffer.$VRi ( Buffer.java:4294967295)
> > > > >> >>>         at java/nio/Buffer.<clinit> (Buffer.java:4294967295)
> > > > >> >>>         at java/lang/J9VMInternals.initializeImpl (Native
> > Method)
> > > > >> >>>         at java/lang/J9VMInternals.initialize
> > > > >> (J9VMInternals.java:185)
> > > > >> >>>         at java/lang/J9VMInternals.initialize
> > > > >> (J9VMInternals.java:154)
> > > > >> >>>         at java/lang/String.getBytes ( String.java :783)
> > > > >> >>>         at org/apache/harmony/luni/util/Util.getBytes (
> > Util.java
> > > > :54)
> > > > >> >>>         at java/lang/ClassLoader.loadLibraryWithPath
> > > > >> (ClassLoader.java
> > > > >> :998)
> > > > >> >>>         at java/lang/ClassLoader.loadLibraryWithClassLoader (
> > > > >> ClassLoader.ja
> > > > >> >>>
> > > > >> >> va:970)
> > > > >> >>
> > > > >> >>>         at java/lang/System.loadLibrary ( System.java:706)
> > > > >> >>>         at com/ibm/icu4jni/common/ICU4JNILoader.loadLibrary
> > > > (Unknown
> > > > >> Source
> > > > >> >>>
> > > > >> >> )
> > > > >> >>
> > > > >> >>>         at com/ibm/icu4jni/converters/NativeConverter.<clinit>
> > > > >> (Unknown Sou
> > > > >> >>>
> > > > >> >> rce)
> > > > >> >>
> > > > >> >>>         at java/lang/J9VMInternals.initializeImpl (Native
> > Method)
> > > > >> >>>         at java/lang/J9VMInternals.initialize
> > > > >> (J9VMInternals.java:185)
> > > > >> >>>         at
> > > > com/ibm/icu4jni/charset/CharsetProviderICU.charsetForName
> > > > >> (Unkno
> > > > >> >>>
> > > > >> >> wn Source)
> > > > >> >>
> > > > >> >>>         at java/nio/charset/Charset.forNameInternal
> > > > >> (Charset.java:450)
> > > > >> >>>         at java/nio/charset/Charset.forName (Charset.java:513)
> > > > >> >>>         at java/lang/String.defaultCharset ( String.java:688)
> > > > >> >>>         at java/lang/String.<init> (String.java:252)
> > > > >> >>>         at org/apache/harmony/luni/util/Util.toString (
> > Util.java
> > > > :74)
> > > > >> >>>         at java/lang/System.getPropertyList (Native Method)
> > > > >> >>>         at java/lang/System.ensureProperties (System.java:496)
> > > > >> >>>         at java/lang/System.<clinit> (System.java:55)
> > > > >> >>>         at java/lang/J9VMInternals.initializeImpl (Native
> > Method)
> > > > >> >>>         at java/lang/J9VMInternals.initialize
> > > > >> (J9VMInternals.java:185)
> > > > >> >>>         at java/lang/ClassLoader.initializeClassLoaders (
> > > > >> ClassLoader.java:6
> > > > >> >>>
> > > > >> >> 5)
> > > > >> >>
> > > > >> >>>         at java/lang/Thread.initialize (Thread.java:301)
> > > > >> >>>         at java/lang/Thread.<init> (Thread.java :120)
> > > > >> >>> JVMJ9VM015W Initialization error for library jclclear_23(14):
> > > > >> JVMJ9VM009E J
> > > > >> >>>
> > > > >> >> 9VMDllMain failed
> > > > >> >>
> > > > >> >>> HMYEXEL062E Internal VM error: Failed to create Java VM
> > > > >> >>> FAILED.
> > > > >> >>>
> > > > >> >>>
> > > > >> >>> On 1 June 2006 at 22:26, "Mikhail Loenko" < mloenko@gmail.com >
> > > > >> wrote:
> > > > >> >>>
> > > > >> >>>
> > > > >> >>>> 2006/6/1, Geir Magnusson Jr < geir@pobox.com>:
> > > > >> >>>>
> > > > >> >>>>
> > > > >> >>>>> Mikhail Loenko wrote:
> > > > >> >>>>>
> > > > >> >>>>>
> > > > >> >>>>>> 2006/6/1, Geir Magnusson Jr < geir@pobox.com >:
> > > > >> >>>>>>
> > > > >> >>>>>>
> > > > >> >>>>>>> I don't quite grok what you mean - could you post what you
> > are
> > > > >> >>>>>>> suggesting in code here on -dev?
> > > > >> >>>>>>>
> > > > >> >>>>>>>
> > > > >> >>>>>> Index:
> > > > >> modules/luni/src/test/java/tests/api/java/net/Inet6AddressTest.ja
> > > > >> >>>>>>
> > > > >> >> v
> > > > >> >>
> > > > >> >>>>>>
> > > > >> >>>>>>
> > > > >> >>>> a
> > > > >> >>>>
> > > > >> >>>>
> > > > >> >>>>>>
> > > > >> ===================================================================
> > > > >> >>>>>> ---
> > > > >> >>>>>>
> > > > >> modules/luni/src/test/java/tests/api/java/net/Inet6AddressTest.java
> > > > >> >>>>>> (revision
> > > > >> >>>>>> 410480)
> > > > >> >>>>>> +++
> > > > >> >>>>>>
> > > > >> modules/luni/src/test/java/tests/api/java/net/Inet6AddressTest.java
> > > > >>
> > > > >> >>>>>> (working
> > > > >> >>>>>> copy)
> > > > >> >>>>>> @@ -785,7 +785,7 @@
> > > > >> >>>>>>                 "FFFF:FFFF:FFFF:FFFF:FFFF:FFFF:
> > 255.255.255.255
> > > > ",
> > > > >> >>>>>>                 "0:0:0:0:0:0:0:0", "0:0:0:0:0:0: 0.0.0.0" };
> > > > >> >>>>>>
> > > > >> >>>>>> -        String invalidIPAddresses[] = { ":", "FFFF:FFFF",
> > > > "1:1",
> > > > >> >>>>>> "::1.2.3.444" };
> > > > >> >>>>>> +        String invalidIPAddresses[] = { "FFFF:FFFF" };
> > > > >> >>>>>>
> > > > >> >>>>>>         for (int i = 0; i < validIPAddresses.length; i++) {
> > > > >> >>>>>>
> > > > >> >>>>>>
> > > > >> >>>>>>
> > > > >> >>>>>>
> > > > >> >>>>> Thx - why do you want to remove testing the others?  Because
> > you
> > > > >> believe
> > > > >> >>>>> that it is in effect testing the resolver in the OS since
> > each
> > > > >> returns
> > > > >> >>>>> the same answer,and thus it's testing the same thing over and
> > > > >> over?
> > > > >> >>>>>
> > > > >> >>>>>
> > > > >> >>>> I believe so, but I'd like to hear confirmation from code
> > authors
> > > > >> >>>>
> > > > >> >>>> Thanks,
> > > > >> >>>> Mikhail
> > > > >> >>>>
> > > > >> >>>>
> > > > >> >>>
> > > > >> >>>
> > > > >>
> > ---------------------------------------------------------------------
> > > > >> >>> Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > > >> >>> To unsubscribe, e-mail:
> > harmony-dev-unsubscribe@incubator.apache.org
> > > >
> > > > >> >>> For additional commands, e-mail:
> > > > >> harmony-dev-help@incubator.apache.org
> > > > >> >>>
> > > > >> >>>
> > > > >> >>>
> > > > >> >>>
> > > > >> >> --
> > > > >> >> Paulex Yang
> > > > >> >> China Software Development Lab
> > > > >> >> IBM
> > > > >> >>
> > > > >> >>
> > > > >> >>
> > > > >> >>
> > > > ---------------------------------------------------------------------
> > > > >> >> Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > > >> >> To unsubscribe, e-mail:
> > harmony-dev-unsubscribe@incubator.apache.org
> > > > >> >> For additional commands, e-mail:
> > > > >> harmony-dev-help@incubator.apache.org
> > > > >> >>
> > > > >> >
> > > > >> >
> > > > >> >
> > > > >> >
> > ---------------------------------------------------------------------
> > > >
> > > > >> > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > > >> > To unsubscribe, e-mail:
> > harmony-dev-unsubscribe@incubator.apache.org
> > > > >> > For additional commands, e-mail:
> > > > harmony-dev-help@incubator.apache.org
> > > > >> >
> > > > >> >
> > > > >> >
> > > > >>
> > > > >>
> > > > >> --
> > > > >> Paulex Yang
> > > > >> China Software Development Lab
> > > > >> IBM
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > ---------------------------------------------------------------------
> > > > >> Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > > >> To unsubscribe, e-mail:
> > harmony-dev-unsubscribe@incubator.apache.org
> > > > >> For additional commands, e-mail:
> > harmony-dev-help@incubator.apache.org
> > > > >>
> > > > >>
> > > > >
> > > >
> > > >
> > > > --
> > > > Paulex Yang
> > > > China Software Development Lab
> > > > IBM
> > > >
> > > >
> > > >
> > > > ---------------------------------------------------------------------
> > > > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > > > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> > > >
> > > >
> > >
> > >
> >
> > ---------------------------------------------------------------------
> > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >
> >
>
>

---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org


Re: [classlib] [testing] Coverage (was Re: 37% of total test execution time is spent in a single test)

Posted by Vladimir Ivanov <iv...@gmail.com>.
The detailed info available at  http://viv.byethost15.com/ (this address
also pointed on wiki-page).
Do you need additional information ?

Thanks,
  Vladimir


On 6/13/06, Mikhail Loenko <ml...@gmail.com> wrote:
>
> Great!
>
> Is it possible to see uncovered classes/methods/lines?
>
> Thanks,
> Mikhail
>
> P.S. I think java.awt does not currently belong to beans module,
> though who knows what the future may bring us :)
>
> 2006/6/13, Vladimir Ivanov <iv...@gmail.com>:
> > Latest Harmony API source coverage by Harmony API unit tests results I
> > stored at wiki page
> >
> > http://wiki.apache.org/harmony/Coverage_information
> >
> >
> >
> > I'm going to refresh it bi-weekly (seems, it is enough for coverage).
> >
> >
> >
> > >I think we have got a agreement on the test naming convention[1], but
> > >for sure, there have been many (legacy) test cases before this
> > >agreement, and I think the volunteer is highly welcome to provide patch
> > >for them.
> > >
> >
> >[1]http://incubator.apache.org/harmony/subcomponents/classlibrary/testing.html
> >
> > >
> >
> > If nobody objects I'm going to look through the unit tests to correct
> > package names according to the agreement (where needed).
> >
> >
> >
> >  Thanks,
> >
> >   Vladimir
> >
> >
> >
> > On 6/8/06, Paulex Yang <paulex.yang@gmail.com > wrote:
> > >
> > > Vladimir,
> > >
> > > Vladimir Ivanov wrote:
> > > > Thanks Paulex!
> > > >
> > > > I did the same, but could not send results due to spam filter J
> > > > Observations:
> > > >
> > > >   1. Coverage results look pretty much similar.
> > > >   2. Exclude list looks pretty much similar too, but, looks like it
> > > >   depends on the way of data collection (I didn't run ant task and
> the
> > > > list is
> > > >   a little bit different).
> > > Great.
> > > >
> > > > In any case, I think, when we run harmony on another VM exclude list
> > > will
> > > > have to be updated.
> > > >
> > > >
> > > >
> > > > May be we can start publishing the coverage information on wiki
> pages
> > > and
> > > > provide some updates time to time (I can do it)?
> > > >
> > > +1,  and of course, you can only if  no one in the mailing list
> objects
> > > and you'll have my welcome, and I think it will be even greater if
> these
> > > reports can be generated regularly like what JAPI is doing:)
> > > >
> > > >
> > > > One note:
> > > >
> > > > I noticed that different unit tests have very different package
> names
> > > >
> > > > Now the directory with all built tests copied to one place looks
> like:
> > > I think we have got a agreement on the test naming convention[1], but
> > > for sure, there have been many (legacy) test cases before this
> > > agreement, and I think the volunteer is highly welcome to provide
> patch
> > > for them.
> > >
> > >
> [1]http://incubator.apache.org/harmony/subcomponents/classlibrary/testing.html
> > >
> > > >
> > > >
> > > >
> > > > C:\coverage\tests\test>ls
> > > >
> > > > GZIPOutClose2.txt        api           config         javax
> > > > tests
> > > >
> > > > GZIPOutFinish.txt        api.injected  dazzle
> org            xml
> > >
> > > >
> > > > GZIPOutWrite.txt         binary        gif            prefs
> > > >
> > > > Inet6Address.golden.ser  bundles       impl           serialization
> > > >
> > > > JDK2-3gabba.zip          com           impl.injected   test.txt
> > > >
> > > >
> > > >
> > > > I think, it would be good if tests had unified package names.
> > > >
> > > > Why? – so far, just common sense, just to have an order in test
> suite
> > > >
> > > > Organization (if consider all unit tests as solid test suite).
> > > >
> > > > Thanks,
> > > >  Vladimir
> > > >
> > > > For example, my exclude list for java.io is:
> > > > -java.io.BufferedInputStream ,
> > > > -java.io.BufferedOutputStream,
> > > > -java.io.File ,
> > > > -java.io.FileChannelFactory,
> > > > -java.io.FileDescriptor,
> > > > -java.io.FileInputStream,
> > > > -java.io.FileOutputStream,
> > > > -java.io.FilterInputStream ,
> > > > -java.io.FilterOutputStream,
> > > > - java.io.InputStream,
> > > > -java.io.OutputStream,
> > > > -java.io.ObjectStreamField,
> > > > -java.io.PrintStream
> > > >
> > > >
> > > > On 6/6/06, Paulex Yang < paulex.yang@gmail.com > wrote:
> > > >>
> > > >> I've attach the scripts and excluded class lists to JIRA,  please
> refer
> > > >> to https://issues.apache.org/jira/browse/HARMONY-564 .  Enjoy it:).
> > > >>
> > > >> Mark Hindess wrote:
> > > >> > On 2 June 2006 at 10:37, Paulex Yang < paulex.yang@gmail.com>
> wrote:
> > > >> >
> > > >> >> Mark,
> > > >> >>
> > > >> >> I'm glad that there is someone else has interest on emma, I've
> > > >> tried it
> > > >> >> before. AFAIK, emma works by instrumentation, but sometimes for
> > > >> classes
> > > >> >> in bootclasspath, the instrumentation cannot work, there are two
> > > >> cases:
> > > >>
> > > >> >> 1. Some instrumented classes cannot be loaded by VM.
> > > >> >> 2. Some classes cannot be instrumented
> > > >> >>
> > > >> >> I have tried to look more inside to find some way to work around
> > > >> but I
> > > >> >> haven't got enough time yet.
> > > >> >>
> > > >> >> Specifically for nio-channel module, I had a list for these two
> > > cases
> > > >> (I
> > > >> >> believe the data is a little outdated and should be reevaluated)
> > > >> >> case 1.
> > > >> >> BaseByteBuffer.class
> > > >> >> Buffer.class
> > > >> >> BufferFactory.class
> > > >> >> ByteBuffer.class
> > > >> >> CharArrayBuffer.class
> > > >> >> CharBuffer.class
> > > >> >> HeapByteBuffer.class
> > > >> >> ReadWriteCharArrayBuffer.class
> > > >> >> ReadWriteHeapByteBuffer.class
> > > >> >> FileChannel.class
> > > >> >> AbstractInterruptibleChannel.class
> > > >> >> FileChannelImpl.class
> > > >> >> WriteOnlyFileChannel.class
> > > >> >> LockManager.class
> > > >> >> LockManager$1.class
> > > >> >> ReadOnlyFileChannel.class
> > > >> >>
> > > >> >> case 2:
> > > >> >> ByteChannel.class
> > > >> >> Channel.class
> > > >> >> GatheringByteChannel.class
> > > >> >> InterruptibleChannel.class
> > > >> >> WritableByteChannel.class
> > > >> >>
> > > >> >> And I have got some ant script and more excluded list for emma,
> if
> > > >> >> anyone has interests, I can upload it to JIRA.
> > > >> >>
> > > >> >
> > > >> > Yes!
> > > >> >
> > > >> > -Mark.
> > > >> >
> > > >> >
> > > >> >> Mark Hindess wrote:
> > > >> >>
> > > >> >>> Anyone tried using emma (emma.sf.net) to look at test coverage
> > > >> for our
> > > >> >>> tests?  That might help ensure we don't trim them down too
> much?
> > > >> >>>
> > > >> >>> I suspect we will soon reach a point where there are
> significantly
> > > >> fewer
> > > >> >>> obvious opportunities for small contributions - e.g.
> > > >> 1.5generification,
> > > >> >>> etc from looking at JAPI reports.  Having test coverage reports
> > > >> might
> > > >> >>> help by highlighting areas for improvement.
> > > >> >>>
> > > >> >>> I had a quick try with:
> > > >> >>>
> > > >> >>>   ant -f make/build.xml
> > > >> >>>   ant -f modules/nio/make/build.xml test
> > > >> >>>   java -cp emma.jar emma instr -m overwrite -cp
> > > >> deploy/jdk/jre/lib/boot/nio
> > > >> >>>
> > > >> >> .jar
> > > >> >>
> > > >> >>> and then:
> > > >> >>>
> > > >> >>>   java -Xbootclasspath/p: emma.jar \
> > > >> >>>        -cp build/tests:modules/nio/bin/test:junit.jar:. \
> > > >> >>>        junit.textui.TestRunner \
> > > >> >>>
> org.apache.harmony.tests.java.nio.channels.SocketChannelTest
> > > >> >>>
> > > >> >>> (Running all the tests is a little tricky because test depends
> on
> > > >> build,
> > > >> >>> which depends on clean which wipes out the instr'd jar. ;-( )
> > > >> >>>
> > > >> >>> Sadly it dies horribly with the stack trace below.  I'll take
> > > >> another
> > > >> >>> look when I have a little more time, but I thought I'd mention
> it
> > > in
> > > >> >>> case anyone else had any thoughts on the subject.
> > > >> >>>
> > > >> >>> Regards,
> > > >> >>>
> > > >> >>> Mark.
> > > >> >>>
> > > >> >>> Exception in thread "main"
> java/lang/ExceptionInInitializerError
> > > >> >>>         at java/lang/J9VMInternals.initialize
> > > >> (J9VMInternals.java:203)
> > > >> >>>         at java/nio/Buffer.$VRi ( Buffer.java:4294967295)
> > > >> >>>         at java/nio/Buffer.<clinit> ( Buffer.java:4294967295)
> > > >> >>>         at java/lang/J9VMInternals.initializeImpl (Native
> Method)
> > > >> >>>         at java/lang/J9VMInternals.initialize (
> J9VMInternals.java
> > > >> :185)
> > > >> >>>         at java/lang/J9VMInternals.initialize
> > > >> (J9VMInternals.java:154)
> > > >> >>>         at java/lang/String.getBytes (String.java:783)
> > > >> >>>         at org/apache/harmony/luni/util/Util.getBytes (
> > > >> Util.java:54)
> > > >> >>>         at java/lang/ClassLoader.loadLibraryWithPath
> > > >> (ClassLoader.java
> > > >> :998)
> > > >> >>>         at java/lang/ClassLoader.loadLibraryWithClassLoader (
> > > >> ClassLoader.ja
> > > >> >>>
> > > >> >> va:970)
> > > >> >>
> > > >> >>>         at java/lang/System.loadLibrary (System.java:706)
> > > >> >>>         at com/ibm/icu4jni/common/ICU4JNILoader.loadLibrary
> > > (Unknown
> > > >> Source
> > > >> >>>
> > > >> >> )
> > > >> >>
> > > >> >>>         at com/ibm/icu4jni/converters/NativeConverter.<clinit>
> > > >> (Unknown Sou
> > > >> >>>
> > > >> >> rce)
> > > >> >>
> > > >> >>>         at java/lang/J9VMInternals.initializeImpl (Native
> Method)
> > > >> >>>         at java/lang/J9VMInternals.initialize
> > > >> ( J9VMInternals.java:185)
> > > >> >>>         at
> > > com/ibm/icu4jni/charset/CharsetProviderICU.charsetForName
> > > >> (Unkno
> > > >> >>>
> > > >> >> wn Source)
> > > >> >>
> > > >> >>>         at java/nio/charset/Charset.forNameInternal
> > > >> (Charset.java:450)
> > > >> >>>         at java/nio/charset/Charset.forName (Charset.java:513)
> > > >> >>>         at java/lang/String.defaultCharset ( String.java:688)
> > > >> >>>         at java/lang/String.<init> ( String.java:252)
> > > >> >>>         at org/apache/harmony/luni/util/Util.toString (
> Util.java
> > > :74)
> > > >> >>>         at java/lang/System.getPropertyList (Native Method)
> > > >> >>>         at java/lang/System.ensureProperties ( System.java:496)
> > > >> >>>         at java/lang/System.<clinit> (System.java:55)
> > > >> >>>         at java/lang/J9VMInternals.initializeImpl (Native
> Method)
> > > >> >>>         at java/lang/J9VMInternals.initialize
> > > >> (J9VMInternals.java:185)
> > > >> >>>         at java/lang/ClassLoader.initializeClassLoaders (
> > > >> ClassLoader.java:6
> > > >> >>>
> > > >> >> 5)
> > > >> >>
> > > >> >>>         at java/lang/Thread.initialize (Thread.java:301)
> > > >> >>>         at java/lang/Thread.<init> (Thread.java:120)
> > > >> >>> java/lang/NullPointerException
> > > >> >>>         at java/lang/Throwable.printStackTrace ( Throwable.java
> > > :231)
> > > >> >>>         at
> com/vladium/emma/rt/RTExitHook.createClassLoaderClosure
> > > >> (RTExitH
> > > >> >>>
> > > >> >> ook.java:63)
> > > >> >>
> > > >> >>>         at com/vladium/emma/rt/RT.reset ( RT.java:92)
> > > >> >>>         at com/vladium/emma/rt/RT.<clinit> (RT.java:238)
> > > >> >>>         at java/lang/J9VMInternals.initializeImpl (Native
> Method)
> > > >> >>>         at java/lang/J9VMInternals.initialize (
> J9VMInternals.java
> > > >> :185)
> > > >> >>>         at java/nio/Buffer.$VRi ( Buffer.java:4294967295)
> > > >> >>>         at java/nio/Buffer.<clinit> (Buffer.java:4294967295)
> > > >> >>>         at java/lang/J9VMInternals.initializeImpl (Native
> Method)
> > > >> >>>         at java/lang/J9VMInternals.initialize
> > > >> (J9VMInternals.java:185)
> > > >> >>>         at java/lang/J9VMInternals.initialize
> > > >> (J9VMInternals.java:154)
> > > >> >>>         at java/lang/String.getBytes ( String.java :783)
> > > >> >>>         at org/apache/harmony/luni/util/Util.getBytes (
> Util.java
> > > :54)
> > > >> >>>         at java/lang/ClassLoader.loadLibraryWithPath
> > > >> (ClassLoader.java
> > > >> :998)
> > > >> >>>         at java/lang/ClassLoader.loadLibraryWithClassLoader (
> > > >> ClassLoader.ja
> > > >> >>>
> > > >> >> va:970)
> > > >> >>
> > > >> >>>         at java/lang/System.loadLibrary ( System.java:706)
> > > >> >>>         at com/ibm/icu4jni/common/ICU4JNILoader.loadLibrary
> > > (Unknown
> > > >> Source
> > > >> >>>
> > > >> >> )
> > > >> >>
> > > >> >>>         at com/ibm/icu4jni/converters/NativeConverter.<clinit>
> > > >> (Unknown Sou
> > > >> >>>
> > > >> >> rce)
> > > >> >>
> > > >> >>>         at java/lang/J9VMInternals.initializeImpl (Native
> Method)
> > > >> >>>         at java/lang/J9VMInternals.initialize
> > > >> (J9VMInternals.java:185)
> > > >> >>>         at
> > > com/ibm/icu4jni/charset/CharsetProviderICU.charsetForName
> > > >> (Unkno
> > > >> >>>
> > > >> >> wn Source)
> > > >> >>
> > > >> >>>         at java/nio/charset/Charset.forNameInternal
> > > >> (Charset.java:450)
> > > >> >>>         at java/nio/charset/Charset.forName (Charset.java:513)
> > > >> >>>         at java/lang/String.defaultCharset ( String.java:688)
> > > >> >>>         at java/lang/String.<init> (String.java:252)
> > > >> >>>         at org/apache/harmony/luni/util/Util.toString (
> Util.java
> > > :74)
> > > >> >>>         at java/lang/System.getPropertyList (Native Method)
> > > >> >>>         at java/lang/System.ensureProperties (System.java:496)
> > > >> >>>         at java/lang/System.<clinit> (System.java:55)
> > > >> >>>         at java/lang/J9VMInternals.initializeImpl (Native
> Method)
> > > >> >>>         at java/lang/J9VMInternals.initialize
> > > >> (J9VMInternals.java:185)
> > > >> >>>         at java/lang/ClassLoader.initializeClassLoaders (
> > > >> ClassLoader.java:6
> > > >> >>>
> > > >> >> 5)
> > > >> >>
> > > >> >>>         at java/lang/Thread.initialize (Thread.java:301)
> > > >> >>>         at java/lang/Thread.<init> (Thread.java :120)
> > > >> >>> JVMJ9VM015W Initialization error for library jclclear_23(14):
> > > >> JVMJ9VM009E J
> > > >> >>>
> > > >> >> 9VMDllMain failed
> > > >> >>
> > > >> >>> HMYEXEL062E Internal VM error: Failed to create Java VM
> > > >> >>> FAILED.
> > > >> >>>
> > > >> >>>
> > > >> >>> On 1 June 2006 at 22:26, "Mikhail Loenko" < mloenko@gmail.com >
> > > >> wrote:
> > > >> >>>
> > > >> >>>
> > > >> >>>> 2006/6/1, Geir Magnusson Jr < geir@pobox.com>:
> > > >> >>>>
> > > >> >>>>
> > > >> >>>>> Mikhail Loenko wrote:
> > > >> >>>>>
> > > >> >>>>>
> > > >> >>>>>> 2006/6/1, Geir Magnusson Jr < geir@pobox.com >:
> > > >> >>>>>>
> > > >> >>>>>>
> > > >> >>>>>>> I don't quite grok what you mean - could you post what you
> are
> > > >> >>>>>>> suggesting in code here on -dev?
> > > >> >>>>>>>
> > > >> >>>>>>>
> > > >> >>>>>> Index:
> > > >> modules/luni/src/test/java/tests/api/java/net/Inet6AddressTest.ja
> > > >> >>>>>>
> > > >> >> v
> > > >> >>
> > > >> >>>>>>
> > > >> >>>>>>
> > > >> >>>> a
> > > >> >>>>
> > > >> >>>>
> > > >> >>>>>>
> > > >> ===================================================================
> > > >> >>>>>> ---
> > > >> >>>>>>
> > > >> modules/luni/src/test/java/tests/api/java/net/Inet6AddressTest.java
> > > >> >>>>>> (revision
> > > >> >>>>>> 410480)
> > > >> >>>>>> +++
> > > >> >>>>>>
> > > >> modules/luni/src/test/java/tests/api/java/net/Inet6AddressTest.java
> > > >>
> > > >> >>>>>> (working
> > > >> >>>>>> copy)
> > > >> >>>>>> @@ -785,7 +785,7 @@
> > > >> >>>>>>                 "FFFF:FFFF:FFFF:FFFF:FFFF:FFFF:
> 255.255.255.255
> > > ",
> > > >> >>>>>>                 "0:0:0:0:0:0:0:0", "0:0:0:0:0:0: 0.0.0.0" };
> > > >> >>>>>>
> > > >> >>>>>> -        String invalidIPAddresses[] = { ":", "FFFF:FFFF",
> > > "1:1",
> > > >> >>>>>> "::1.2.3.444" };
> > > >> >>>>>> +        String invalidIPAddresses[] = { "FFFF:FFFF" };
> > > >> >>>>>>
> > > >> >>>>>>         for (int i = 0; i < validIPAddresses.length; i++) {
> > > >> >>>>>>
> > > >> >>>>>>
> > > >> >>>>>>
> > > >> >>>>>>
> > > >> >>>>> Thx - why do you want to remove testing the others?  Because
> you
> > > >> believe
> > > >> >>>>> that it is in effect testing the resolver in the OS since
> each
> > > >> returns
> > > >> >>>>> the same answer,and thus it's testing the same thing over and
> > > >> over?
> > > >> >>>>>
> > > >> >>>>>
> > > >> >>>> I believe so, but I'd like to hear confirmation from code
> authors
> > > >> >>>>
> > > >> >>>> Thanks,
> > > >> >>>> Mikhail
> > > >> >>>>
> > > >> >>>>
> > > >> >>>
> > > >> >>>
> > > >>
> ---------------------------------------------------------------------
> > > >> >>> Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > >> >>> To unsubscribe, e-mail:
> harmony-dev-unsubscribe@incubator.apache.org
> > >
> > > >> >>> For additional commands, e-mail:
> > > >> harmony-dev-help@incubator.apache.org
> > > >> >>>
> > > >> >>>
> > > >> >>>
> > > >> >>>
> > > >> >> --
> > > >> >> Paulex Yang
> > > >> >> China Software Development Lab
> > > >> >> IBM
> > > >> >>
> > > >> >>
> > > >> >>
> > > >> >>
> > > ---------------------------------------------------------------------
> > > >> >> Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > >> >> To unsubscribe, e-mail:
> harmony-dev-unsubscribe@incubator.apache.org
> > > >> >> For additional commands, e-mail:
> > > >> harmony-dev-help@incubator.apache.org
> > > >> >>
> > > >> >
> > > >> >
> > > >> >
> > > >> >
> ---------------------------------------------------------------------
> > >
> > > >> > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > >> > To unsubscribe, e-mail:
> harmony-dev-unsubscribe@incubator.apache.org
> > > >> > For additional commands, e-mail:
> > > harmony-dev-help@incubator.apache.org
> > > >> >
> > > >> >
> > > >> >
> > > >>
> > > >>
> > > >> --
> > > >> Paulex Yang
> > > >> China Software Development Lab
> > > >> IBM
> > > >>
> > > >>
> > > >>
> > > >>
> ---------------------------------------------------------------------
> > > >> Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > >> To unsubscribe, e-mail:
> harmony-dev-unsubscribe@incubator.apache.org
> > > >> For additional commands, e-mail:
> harmony-dev-help@incubator.apache.org
> > > >>
> > > >>
> > > >
> > >
> > >
> > > --
> > > Paulex Yang
> > > China Software Development Lab
> > > IBM
> > >
> > >
> > >
> > > ---------------------------------------------------------------------
> > > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> > >
> > >
> >
> >
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>

Re: [classlib] [testing] Coverage (was Re: 37% of total test execution time is spent in a single test)

Posted by Mikhail Loenko <ml...@gmail.com>.
Great!

Is it possible to see uncovered classes/methods/lines?

Thanks,
Mikhail

P.S. I think java.awt does not currently belong to beans module,
though who knows what the future may bring us :)

2006/6/13, Vladimir Ivanov <iv...@gmail.com>:
> Latest Harmony API source coverage by Harmony API unit tests results I
> stored at wiki page
>
> http://wiki.apache.org/harmony/Coverage_information
>
>
>
> I'm going to refresh it bi-weekly (seems, it is enough for coverage).
>
>
>
> >I think we have got a agreement on the test naming convention[1], but
> >for sure, there have been many (legacy) test cases before this
> >agreement, and I think the volunteer is highly welcome to provide patch
> >for them.
> >
> >[1]http://incubator.apache.org/harmony/subcomponents/classlibrary/testing.html
>
> >
>
> If nobody objects I'm going to look through the unit tests to correct
> package names according to the agreement (where needed).
>
>
>
>  Thanks,
>
>   Vladimir
>
>
>
> On 6/8/06, Paulex Yang <paulex.yang@gmail.com > wrote:
> >
> > Vladimir,
> >
> > Vladimir Ivanov wrote:
> > > Thanks Paulex!
> > >
> > > I did the same, but could not send results due to spam filter J
> > > Observations:
> > >
> > >   1. Coverage results look pretty much similar.
> > >   2. Exclude list looks pretty much similar too, but, looks like it
> > >   depends on the way of data collection (I didn't run ant task and the
> > > list is
> > >   a little bit different).
> > Great.
> > >
> > > In any case, I think, when we run harmony on another VM exclude list
> > will
> > > have to be updated.
> > >
> > >
> > >
> > > May be we can start publishing the coverage information on wiki pages
> > and
> > > provide some updates time to time (I can do it)?
> > >
> > +1,  and of course, you can only if  no one in the mailing list objects
> > and you'll have my welcome, and I think it will be even greater if these
> > reports can be generated regularly like what JAPI is doing:)
> > >
> > >
> > > One note:
> > >
> > > I noticed that different unit tests have very different package names
> > >
> > > Now the directory with all built tests copied to one place looks like:
> > I think we have got a agreement on the test naming convention[1], but
> > for sure, there have been many (legacy) test cases before this
> > agreement, and I think the volunteer is highly welcome to provide patch
> > for them.
> >
> > [1]http://incubator.apache.org/harmony/subcomponents/classlibrary/testing.html
> >
> > >
> > >
> > >
> > > C:\coverage\tests\test>ls
> > >
> > > GZIPOutClose2.txt        api           config         javax
> > > tests
> > >
> > > GZIPOutFinish.txt        api.injected  dazzle         org            xml
> >
> > >
> > > GZIPOutWrite.txt         binary        gif            prefs
> > >
> > > Inet6Address.golden.ser  bundles       impl           serialization
> > >
> > > JDK2-3gabba.zip          com           impl.injected   test.txt
> > >
> > >
> > >
> > > I think, it would be good if tests had unified package names.
> > >
> > > Why? – so far, just common sense, just to have an order in test suite
> > >
> > > Organization (if consider all unit tests as solid test suite).
> > >
> > > Thanks,
> > >  Vladimir
> > >
> > > For example, my exclude list for java.io is:
> > > -java.io.BufferedInputStream ,
> > > -java.io.BufferedOutputStream,
> > > -java.io.File ,
> > > -java.io.FileChannelFactory,
> > > -java.io.FileDescriptor,
> > > -java.io.FileInputStream,
> > > -java.io.FileOutputStream,
> > > -java.io.FilterInputStream ,
> > > -java.io.FilterOutputStream,
> > > - java.io.InputStream,
> > > -java.io.OutputStream,
> > > -java.io.ObjectStreamField,
> > > -java.io.PrintStream
> > >
> > >
> > > On 6/6/06, Paulex Yang < paulex.yang@gmail.com > wrote:
> > >>
> > >> I've attach the scripts and excluded class lists to JIRA,  please refer
> > >> to https://issues.apache.org/jira/browse/HARMONY-564 .  Enjoy it:).
> > >>
> > >> Mark Hindess wrote:
> > >> > On 2 June 2006 at 10:37, Paulex Yang < paulex.yang@gmail.com> wrote:
> > >> >
> > >> >> Mark,
> > >> >>
> > >> >> I'm glad that there is someone else has interest on emma, I've
> > >> tried it
> > >> >> before. AFAIK, emma works by instrumentation, but sometimes for
> > >> classes
> > >> >> in bootclasspath, the instrumentation cannot work, there are two
> > >> cases:
> > >>
> > >> >> 1. Some instrumented classes cannot be loaded by VM.
> > >> >> 2. Some classes cannot be instrumented
> > >> >>
> > >> >> I have tried to look more inside to find some way to work around
> > >> but I
> > >> >> haven't got enough time yet.
> > >> >>
> > >> >> Specifically for nio-channel module, I had a list for these two
> > cases
> > >> (I
> > >> >> believe the data is a little outdated and should be reevaluated)
> > >> >> case 1.
> > >> >> BaseByteBuffer.class
> > >> >> Buffer.class
> > >> >> BufferFactory.class
> > >> >> ByteBuffer.class
> > >> >> CharArrayBuffer.class
> > >> >> CharBuffer.class
> > >> >> HeapByteBuffer.class
> > >> >> ReadWriteCharArrayBuffer.class
> > >> >> ReadWriteHeapByteBuffer.class
> > >> >> FileChannel.class
> > >> >> AbstractInterruptibleChannel.class
> > >> >> FileChannelImpl.class
> > >> >> WriteOnlyFileChannel.class
> > >> >> LockManager.class
> > >> >> LockManager$1.class
> > >> >> ReadOnlyFileChannel.class
> > >> >>
> > >> >> case 2:
> > >> >> ByteChannel.class
> > >> >> Channel.class
> > >> >> GatheringByteChannel.class
> > >> >> InterruptibleChannel.class
> > >> >> WritableByteChannel.class
> > >> >>
> > >> >> And I have got some ant script and more excluded list for emma, if
> > >> >> anyone has interests, I can upload it to JIRA.
> > >> >>
> > >> >
> > >> > Yes!
> > >> >
> > >> > -Mark.
> > >> >
> > >> >
> > >> >> Mark Hindess wrote:
> > >> >>
> > >> >>> Anyone tried using emma (emma.sf.net) to look at test coverage
> > >> for our
> > >> >>> tests?  That might help ensure we don't trim them down too much?
> > >> >>>
> > >> >>> I suspect we will soon reach a point where there are significantly
> > >> fewer
> > >> >>> obvious opportunities for small contributions - e.g.
> > >> 1.5generification,
> > >> >>> etc from looking at JAPI reports.  Having test coverage reports
> > >> might
> > >> >>> help by highlighting areas for improvement.
> > >> >>>
> > >> >>> I had a quick try with:
> > >> >>>
> > >> >>>   ant -f make/build.xml
> > >> >>>   ant -f modules/nio/make/build.xml test
> > >> >>>   java -cp emma.jar emma instr -m overwrite -cp
> > >> deploy/jdk/jre/lib/boot/nio
> > >> >>>
> > >> >> .jar
> > >> >>
> > >> >>> and then:
> > >> >>>
> > >> >>>   java -Xbootclasspath/p: emma.jar \
> > >> >>>        -cp build/tests:modules/nio/bin/test:junit.jar:. \
> > >> >>>        junit.textui.TestRunner \
> > >> >>>        org.apache.harmony.tests.java.nio.channels.SocketChannelTest
> > >> >>>
> > >> >>> (Running all the tests is a little tricky because test depends on
> > >> build,
> > >> >>> which depends on clean which wipes out the instr'd jar. ;-( )
> > >> >>>
> > >> >>> Sadly it dies horribly with the stack trace below.  I'll take
> > >> another
> > >> >>> look when I have a little more time, but I thought I'd mention it
> > in
> > >> >>> case anyone else had any thoughts on the subject.
> > >> >>>
> > >> >>> Regards,
> > >> >>>
> > >> >>> Mark.
> > >> >>>
> > >> >>> Exception in thread "main" java/lang/ExceptionInInitializerError
> > >> >>>         at java/lang/J9VMInternals.initialize
> > >> (J9VMInternals.java:203)
> > >> >>>         at java/nio/Buffer.$VRi ( Buffer.java:4294967295)
> > >> >>>         at java/nio/Buffer.<clinit> ( Buffer.java:4294967295)
> > >> >>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
> > >> >>>         at java/lang/J9VMInternals.initialize ( J9VMInternals.java
> > >> :185)
> > >> >>>         at java/lang/J9VMInternals.initialize
> > >> (J9VMInternals.java:154)
> > >> >>>         at java/lang/String.getBytes (String.java:783)
> > >> >>>         at org/apache/harmony/luni/util/Util.getBytes (
> > >> Util.java:54)
> > >> >>>         at java/lang/ClassLoader.loadLibraryWithPath
> > >> (ClassLoader.java
> > >> :998)
> > >> >>>         at java/lang/ClassLoader.loadLibraryWithClassLoader (
> > >> ClassLoader.ja
> > >> >>>
> > >> >> va:970)
> > >> >>
> > >> >>>         at java/lang/System.loadLibrary (System.java:706)
> > >> >>>         at com/ibm/icu4jni/common/ICU4JNILoader.loadLibrary
> > (Unknown
> > >> Source
> > >> >>>
> > >> >> )
> > >> >>
> > >> >>>         at com/ibm/icu4jni/converters/NativeConverter.<clinit>
> > >> (Unknown Sou
> > >> >>>
> > >> >> rce)
> > >> >>
> > >> >>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
> > >> >>>         at java/lang/J9VMInternals.initialize
> > >> ( J9VMInternals.java:185)
> > >> >>>         at
> > com/ibm/icu4jni/charset/CharsetProviderICU.charsetForName
> > >> (Unkno
> > >> >>>
> > >> >> wn Source)
> > >> >>
> > >> >>>         at java/nio/charset/Charset.forNameInternal
> > >> (Charset.java:450)
> > >> >>>         at java/nio/charset/Charset.forName (Charset.java:513)
> > >> >>>         at java/lang/String.defaultCharset ( String.java:688)
> > >> >>>         at java/lang/String.<init> ( String.java:252)
> > >> >>>         at org/apache/harmony/luni/util/Util.toString (Util.java
> > :74)
> > >> >>>         at java/lang/System.getPropertyList (Native Method)
> > >> >>>         at java/lang/System.ensureProperties ( System.java:496)
> > >> >>>         at java/lang/System.<clinit> (System.java:55)
> > >> >>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
> > >> >>>         at java/lang/J9VMInternals.initialize
> > >> (J9VMInternals.java:185)
> > >> >>>         at java/lang/ClassLoader.initializeClassLoaders (
> > >> ClassLoader.java:6
> > >> >>>
> > >> >> 5)
> > >> >>
> > >> >>>         at java/lang/Thread.initialize (Thread.java:301)
> > >> >>>         at java/lang/Thread.<init> (Thread.java:120)
> > >> >>> java/lang/NullPointerException
> > >> >>>         at java/lang/Throwable.printStackTrace ( Throwable.java
> > :231)
> > >> >>>         at com/vladium/emma/rt/RTExitHook.createClassLoaderClosure
> > >> (RTExitH
> > >> >>>
> > >> >> ook.java:63)
> > >> >>
> > >> >>>         at com/vladium/emma/rt/RT.reset ( RT.java:92)
> > >> >>>         at com/vladium/emma/rt/RT.<clinit> (RT.java:238)
> > >> >>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
> > >> >>>         at java/lang/J9VMInternals.initialize ( J9VMInternals.java
> > >> :185)
> > >> >>>         at java/nio/Buffer.$VRi ( Buffer.java:4294967295)
> > >> >>>         at java/nio/Buffer.<clinit> (Buffer.java:4294967295)
> > >> >>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
> > >> >>>         at java/lang/J9VMInternals.initialize
> > >> (J9VMInternals.java:185)
> > >> >>>         at java/lang/J9VMInternals.initialize
> > >> (J9VMInternals.java:154)
> > >> >>>         at java/lang/String.getBytes ( String.java :783)
> > >> >>>         at org/apache/harmony/luni/util/Util.getBytes (Util.java
> > :54)
> > >> >>>         at java/lang/ClassLoader.loadLibraryWithPath
> > >> (ClassLoader.java
> > >> :998)
> > >> >>>         at java/lang/ClassLoader.loadLibraryWithClassLoader (
> > >> ClassLoader.ja
> > >> >>>
> > >> >> va:970)
> > >> >>
> > >> >>>         at java/lang/System.loadLibrary ( System.java:706)
> > >> >>>         at com/ibm/icu4jni/common/ICU4JNILoader.loadLibrary
> > (Unknown
> > >> Source
> > >> >>>
> > >> >> )
> > >> >>
> > >> >>>         at com/ibm/icu4jni/converters/NativeConverter.<clinit>
> > >> (Unknown Sou
> > >> >>>
> > >> >> rce)
> > >> >>
> > >> >>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
> > >> >>>         at java/lang/J9VMInternals.initialize
> > >> (J9VMInternals.java:185)
> > >> >>>         at
> > com/ibm/icu4jni/charset/CharsetProviderICU.charsetForName
> > >> (Unkno
> > >> >>>
> > >> >> wn Source)
> > >> >>
> > >> >>>         at java/nio/charset/Charset.forNameInternal
> > >> (Charset.java:450)
> > >> >>>         at java/nio/charset/Charset.forName (Charset.java:513)
> > >> >>>         at java/lang/String.defaultCharset ( String.java:688)
> > >> >>>         at java/lang/String.<init> (String.java:252)
> > >> >>>         at org/apache/harmony/luni/util/Util.toString (Util.java
> > :74)
> > >> >>>         at java/lang/System.getPropertyList (Native Method)
> > >> >>>         at java/lang/System.ensureProperties (System.java:496)
> > >> >>>         at java/lang/System.<clinit> (System.java:55)
> > >> >>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
> > >> >>>         at java/lang/J9VMInternals.initialize
> > >> (J9VMInternals.java:185)
> > >> >>>         at java/lang/ClassLoader.initializeClassLoaders (
> > >> ClassLoader.java:6
> > >> >>>
> > >> >> 5)
> > >> >>
> > >> >>>         at java/lang/Thread.initialize (Thread.java:301)
> > >> >>>         at java/lang/Thread.<init> (Thread.java :120)
> > >> >>> JVMJ9VM015W Initialization error for library jclclear_23(14):
> > >> JVMJ9VM009E J
> > >> >>>
> > >> >> 9VMDllMain failed
> > >> >>
> > >> >>> HMYEXEL062E Internal VM error: Failed to create Java VM
> > >> >>> FAILED.
> > >> >>>
> > >> >>>
> > >> >>> On 1 June 2006 at 22:26, "Mikhail Loenko" < mloenko@gmail.com >
> > >> wrote:
> > >> >>>
> > >> >>>
> > >> >>>> 2006/6/1, Geir Magnusson Jr < geir@pobox.com>:
> > >> >>>>
> > >> >>>>
> > >> >>>>> Mikhail Loenko wrote:
> > >> >>>>>
> > >> >>>>>
> > >> >>>>>> 2006/6/1, Geir Magnusson Jr < geir@pobox.com >:
> > >> >>>>>>
> > >> >>>>>>
> > >> >>>>>>> I don't quite grok what you mean - could you post what you are
> > >> >>>>>>> suggesting in code here on -dev?
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>> Index:
> > >> modules/luni/src/test/java/tests/api/java/net/Inet6AddressTest.ja
> > >> >>>>>>
> > >> >> v
> > >> >>
> > >> >>>>>>
> > >> >>>>>>
> > >> >>>> a
> > >> >>>>
> > >> >>>>
> > >> >>>>>>
> > >> ===================================================================
> > >> >>>>>> ---
> > >> >>>>>>
> > >> modules/luni/src/test/java/tests/api/java/net/Inet6AddressTest.java
> > >> >>>>>> (revision
> > >> >>>>>> 410480)
> > >> >>>>>> +++
> > >> >>>>>>
> > >> modules/luni/src/test/java/tests/api/java/net/Inet6AddressTest.java
> > >>
> > >> >>>>>> (working
> > >> >>>>>> copy)
> > >> >>>>>> @@ -785,7 +785,7 @@
> > >> >>>>>>                 "FFFF:FFFF:FFFF:FFFF:FFFF:FFFF: 255.255.255.255
> > ",
> > >> >>>>>>                 "0:0:0:0:0:0:0:0", "0:0:0:0:0:0: 0.0.0.0" };
> > >> >>>>>>
> > >> >>>>>> -        String invalidIPAddresses[] = { ":", "FFFF:FFFF",
> > "1:1",
> > >> >>>>>> "::1.2.3.444" };
> > >> >>>>>> +        String invalidIPAddresses[] = { "FFFF:FFFF" };
> > >> >>>>>>
> > >> >>>>>>         for (int i = 0; i < validIPAddresses.length; i++) {
> > >> >>>>>>
> > >> >>>>>>
> > >> >>>>>>
> > >> >>>>>>
> > >> >>>>> Thx - why do you want to remove testing the others?  Because you
> > >> believe
> > >> >>>>> that it is in effect testing the resolver in the OS since each
> > >> returns
> > >> >>>>> the same answer,and thus it's testing the same thing over and
> > >> over?
> > >> >>>>>
> > >> >>>>>
> > >> >>>> I believe so, but I'd like to hear confirmation from code authors
> > >> >>>>
> > >> >>>> Thanks,
> > >> >>>> Mikhail
> > >> >>>>
> > >> >>>>
> > >> >>>
> > >> >>>
> > >> ---------------------------------------------------------------------
> > >> >>> Terms of use : http://incubator.apache.org/harmony/mailing.html
> > >> >>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> >
> > >> >>> For additional commands, e-mail:
> > >> harmony-dev-help@incubator.apache.org
> > >> >>>
> > >> >>>
> > >> >>>
> > >> >>>
> > >> >> --
> > >> >> Paulex Yang
> > >> >> China Software Development Lab
> > >> >> IBM
> > >> >>
> > >> >>
> > >> >>
> > >> >>
> > ---------------------------------------------------------------------
> > >> >> Terms of use : http://incubator.apache.org/harmony/mailing.html
> > >> >> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > >> >> For additional commands, e-mail:
> > >> harmony-dev-help@incubator.apache.org
> > >> >>
> > >> >
> > >> >
> > >> >
> > >> > ---------------------------------------------------------------------
> >
> > >> > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > >> > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > >> > For additional commands, e-mail:
> > harmony-dev-help@incubator.apache.org
> > >> >
> > >> >
> > >> >
> > >>
> > >>
> > >> --
> > >> Paulex Yang
> > >> China Software Development Lab
> > >> IBM
> > >>
> > >>
> > >>
> > >> ---------------------------------------------------------------------
> > >> Terms of use : http://incubator.apache.org/harmony/mailing.html
> > >> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > >> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> > >>
> > >>
> > >
> >
> >
> > --
> > Paulex Yang
> > China Software Development Lab
> > IBM
> >
> >
> >
> > ---------------------------------------------------------------------
> > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >
> >
>
>

---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org


Re: [classlib] [testing] Coverage (was Re: 37% of total test execution time is spent in a single test)

Posted by Paulex Yang <pa...@gmail.com>.
Vladimir Ivanov wrote:
> Latest Harmony API source coverage by Harmony API unit tests results I
> stored at wiki page
>
> http://wiki.apache.org/harmony/Coverage_information
Cool! Thank you, Vladimir.

Would you please also add the excluded class list, so that we can 
differentiate them from the classes actually without unit tests?
>
>
>
> I'm going to refresh it bi-weekly (seems, it is enough for coverage).
>
>
>
>> I think we have got a agreement on the test naming convention[1], but
>> for sure, there have been many (legacy) test cases before this
>> agreement, and I think the volunteer is highly welcome to provide patch
>> for them.
>>
>> [1]http://incubator.apache.org/harmony/subcomponents/classlibrary/testing.html 
>>
>
>>
>
> If nobody objects I'm going to look through the unit tests to correct
> package names according to the agreement (where needed).
>
>
>
> Thanks,
>
>   Vladimir
>
>
>
> On 6/8/06, Paulex Yang <paulex.yang@gmail.com > wrote:
>>
>> Vladimir,
>>
>> Vladimir Ivanov wrote:
>> > Thanks Paulex!
>> >
>> > I did the same, but could not send results due to spam filter J
>> > Observations:
>> >
>> >   1. Coverage results look pretty much similar.
>> >   2. Exclude list looks pretty much similar too, but, looks like it
>> >   depends on the way of data collection (I didn't run ant task and the
>> > list is
>> >   a little bit different).
>> Great.
>> >
>> > In any case, I think, when we run harmony on another VM exclude list
>> will
>> > have to be updated.
>> >
>> >
>> >
>> > May be we can start publishing the coverage information on wiki pages
>> and
>> > provide some updates time to time (I can do it)?
>> >
>> +1,  and of course, you can only if  no one in the mailing list objects
>> and you'll have my welcome, and I think it will be even greater if these
>> reports can be generated regularly like what JAPI is doing:)
>> >
>> >
>> > One note:
>> >
>> > I noticed that different unit tests have very different package names
>> >
>> > Now the directory with all built tests copied to one place looks like:
>> I think we have got a agreement on the test naming convention[1], but
>> for sure, there have been many (legacy) test cases before this
>> agreement, and I think the volunteer is highly welcome to provide patch
>> for them.
>>
>> [1]http://incubator.apache.org/harmony/subcomponents/classlibrary/testing.html 
>>
>>
>> >
>> >
>> >
>> > C:\coverage\tests\test>ls
>> >
>> > GZIPOutClose2.txt        api           config         javax
>> > tests
>> >
>> > GZIPOutFinish.txt        api.injected  dazzle         
>> org            xml
>>
>> >
>> > GZIPOutWrite.txt         binary        gif            prefs
>> >
>> > Inet6Address.golden.ser  bundles       impl           serialization
>> >
>> > JDK2-3gabba.zip          com           impl.injected   test.txt
>> >
>> >
>> >
>> > I think, it would be good if tests had unified package names.
>> >
>> > Why? – so far, just common sense, just to have an order in test suite
>> >
>> > Organization (if consider all unit tests as solid test suite).
>> >
>> > Thanks,
>> >  Vladimir
>> >
>> > For example, my exclude list for java.io is:
>> > -java.io.BufferedInputStream ,
>> > -java.io.BufferedOutputStream,
>> > -java.io.File ,
>> > -java.io.FileChannelFactory,
>> > -java.io.FileDescriptor,
>> > -java.io.FileInputStream,
>> > -java.io.FileOutputStream,
>> > -java.io.FilterInputStream ,
>> > -java.io.FilterOutputStream,
>> > - java.io.InputStream,
>> > -java.io.OutputStream,
>> > -java.io.ObjectStreamField,
>> > -java.io.PrintStream
>> >
>> >
>> > On 6/6/06, Paulex Yang < paulex.yang@gmail.com > wrote:
>> >>
>> >> I've attach the scripts and excluded class lists to JIRA,  please 
>> refer
>> >> to https://issues.apache.org/jira/browse/HARMONY-564 .  Enjoy it:).
>> >>
>> >> Mark Hindess wrote:
>> >> > On 2 June 2006 at 10:37, Paulex Yang < paulex.yang@gmail.com> 
>> wrote:
>> >> >
>> >> >> Mark,
>> >> >>
>> >> >> I'm glad that there is someone else has interest on emma, I've
>> >> tried it
>> >> >> before. AFAIK, emma works by instrumentation, but sometimes for
>> >> classes
>> >> >> in bootclasspath, the instrumentation cannot work, there are two
>> >> cases:
>> >>
>> >> >> 1. Some instrumented classes cannot be loaded by VM.
>> >> >> 2. Some classes cannot be instrumented
>> >> >>
>> >> >> I have tried to look more inside to find some way to work around
>> >> but I
>> >> >> haven't got enough time yet.
>> >> >>
>> >> >> Specifically for nio-channel module, I had a list for these two
>> cases
>> >> (I
>> >> >> believe the data is a little outdated and should be reevaluated)
>> >> >> case 1.
>> >> >> BaseByteBuffer.class
>> >> >> Buffer.class
>> >> >> BufferFactory.class
>> >> >> ByteBuffer.class
>> >> >> CharArrayBuffer.class
>> >> >> CharBuffer.class
>> >> >> HeapByteBuffer.class
>> >> >> ReadWriteCharArrayBuffer.class
>> >> >> ReadWriteHeapByteBuffer.class
>> >> >> FileChannel.class
>> >> >> AbstractInterruptibleChannel.class
>> >> >> FileChannelImpl.class
>> >> >> WriteOnlyFileChannel.class
>> >> >> LockManager.class
>> >> >> LockManager$1.class
>> >> >> ReadOnlyFileChannel.class
>> >> >>
>> >> >> case 2:
>> >> >> ByteChannel.class
>> >> >> Channel.class
>> >> >> GatheringByteChannel.class
>> >> >> InterruptibleChannel.class
>> >> >> WritableByteChannel.class
>> >> >>
>> >> >> And I have got some ant script and more excluded list for emma, if
>> >> >> anyone has interests, I can upload it to JIRA.
>> >> >>
>> >> >
>> >> > Yes!
>> >> >
>> >> > -Mark.
>> >> >
>> >> >
>> >> >> Mark Hindess wrote:
>> >> >>
>> >> >>> Anyone tried using emma (emma.sf.net) to look at test coverage
>> >> for our
>> >> >>> tests?  That might help ensure we don't trim them down too much?
>> >> >>>
>> >> >>> I suspect we will soon reach a point where there are 
>> significantly
>> >> fewer
>> >> >>> obvious opportunities for small contributions - e.g.
>> >> 1.5generification,
>> >> >>> etc from looking at JAPI reports.  Having test coverage reports
>> >> might
>> >> >>> help by highlighting areas for improvement.
>> >> >>>
>> >> >>> I had a quick try with:
>> >> >>>
>> >> >>>   ant -f make/build.xml
>> >> >>>   ant -f modules/nio/make/build.xml test
>> >> >>>   java -cp emma.jar emma instr -m overwrite -cp
>> >> deploy/jdk/jre/lib/boot/nio
>> >> >>>
>> >> >> .jar
>> >> >>
>> >> >>> and then:
>> >> >>>
>> >> >>>   java -Xbootclasspath/p: emma.jar \
>> >> >>>        -cp build/tests:modules/nio/bin/test:junit.jar:. \
>> >> >>>        junit.textui.TestRunner \
>> >> >>>        
>> org.apache.harmony.tests.java.nio.channels.SocketChannelTest
>> >> >>>
>> >> >>> (Running all the tests is a little tricky because test depends on
>> >> build,
>> >> >>> which depends on clean which wipes out the instr'd jar. ;-( )
>> >> >>>
>> >> >>> Sadly it dies horribly with the stack trace below.  I'll take
>> >> another
>> >> >>> look when I have a little more time, but I thought I'd mention it
>> in
>> >> >>> case anyone else had any thoughts on the subject.
>> >> >>>
>> >> >>> Regards,
>> >> >>>
>> >> >>> Mark.
>> >> >>>
>> >> >>> Exception in thread "main" java/lang/ExceptionInInitializerError
>> >> >>>         at java/lang/J9VMInternals.initialize
>> >> (J9VMInternals.java:203)
>> >> >>>         at java/nio/Buffer.$VRi ( Buffer.java:4294967295)
>> >> >>>         at java/nio/Buffer.<clinit> ( Buffer.java:4294967295)
>> >> >>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
>> >> >>>         at java/lang/J9VMInternals.initialize ( 
>> J9VMInternals.java
>> >> :185)
>> >> >>>         at java/lang/J9VMInternals.initialize
>> >> (J9VMInternals.java:154)
>> >> >>>         at java/lang/String.getBytes (String.java:783)
>> >> >>>         at org/apache/harmony/luni/util/Util.getBytes (
>> >> Util.java:54)
>> >> >>>         at java/lang/ClassLoader.loadLibraryWithPath
>> >> (ClassLoader.java
>> >> :998)
>> >> >>>         at java/lang/ClassLoader.loadLibraryWithClassLoader (
>> >> ClassLoader.ja
>> >> >>>
>> >> >> va:970)
>> >> >>
>> >> >>>         at java/lang/System.loadLibrary (System.java:706)
>> >> >>>         at com/ibm/icu4jni/common/ICU4JNILoader.loadLibrary
>> (Unknown
>> >> Source
>> >> >>>
>> >> >> )
>> >> >>
>> >> >>>         at com/ibm/icu4jni/converters/NativeConverter.<clinit>
>> >> (Unknown Sou
>> >> >>>
>> >> >> rce)
>> >> >>
>> >> >>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
>> >> >>>         at java/lang/J9VMInternals.initialize
>> >> ( J9VMInternals.java:185)
>> >> >>>         at
>> com/ibm/icu4jni/charset/CharsetProviderICU.charsetForName
>> >> (Unkno
>> >> >>>
>> >> >> wn Source)
>> >> >>
>> >> >>>         at java/nio/charset/Charset.forNameInternal
>> >> (Charset.java:450)
>> >> >>>         at java/nio/charset/Charset.forName (Charset.java:513)
>> >> >>>         at java/lang/String.defaultCharset ( String.java:688)
>> >> >>>         at java/lang/String.<init> ( String.java:252)
>> >> >>>         at org/apache/harmony/luni/util/Util.toString (Util.java
>> :74)
>> >> >>>         at java/lang/System.getPropertyList (Native Method)
>> >> >>>         at java/lang/System.ensureProperties ( System.java:496)
>> >> >>>         at java/lang/System.<clinit> (System.java:55)
>> >> >>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
>> >> >>>         at java/lang/J9VMInternals.initialize
>> >> (J9VMInternals.java:185)
>> >> >>>         at java/lang/ClassLoader.initializeClassLoaders (
>> >> ClassLoader.java:6
>> >> >>>
>> >> >> 5)
>> >> >>
>> >> >>>         at java/lang/Thread.initialize (Thread.java:301)
>> >> >>>         at java/lang/Thread.<init> (Thread.java:120)
>> >> >>> java/lang/NullPointerException
>> >> >>>         at java/lang/Throwable.printStackTrace ( Throwable.java
>> :231)
>> >> >>>         at 
>> com/vladium/emma/rt/RTExitHook.createClassLoaderClosure
>> >> (RTExitH
>> >> >>>
>> >> >> ook.java:63)
>> >> >>
>> >> >>>         at com/vladium/emma/rt/RT.reset ( RT.java:92)
>> >> >>>         at com/vladium/emma/rt/RT.<clinit> (RT.java:238)
>> >> >>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
>> >> >>>         at java/lang/J9VMInternals.initialize ( 
>> J9VMInternals.java
>> >> :185)
>> >> >>>         at java/nio/Buffer.$VRi ( Buffer.java:4294967295)
>> >> >>>         at java/nio/Buffer.<clinit> (Buffer.java:4294967295)
>> >> >>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
>> >> >>>         at java/lang/J9VMInternals.initialize
>> >> (J9VMInternals.java:185)
>> >> >>>         at java/lang/J9VMInternals.initialize
>> >> (J9VMInternals.java:154)
>> >> >>>         at java/lang/String.getBytes ( String.java :783)
>> >> >>>         at org/apache/harmony/luni/util/Util.getBytes (Util.java
>> :54)
>> >> >>>         at java/lang/ClassLoader.loadLibraryWithPath
>> >> (ClassLoader.java
>> >> :998)
>> >> >>>         at java/lang/ClassLoader.loadLibraryWithClassLoader (
>> >> ClassLoader.ja
>> >> >>>
>> >> >> va:970)
>> >> >>
>> >> >>>         at java/lang/System.loadLibrary ( System.java:706)
>> >> >>>         at com/ibm/icu4jni/common/ICU4JNILoader.loadLibrary
>> (Unknown
>> >> Source
>> >> >>>
>> >> >> )
>> >> >>
>> >> >>>         at com/ibm/icu4jni/converters/NativeConverter.<clinit>
>> >> (Unknown Sou
>> >> >>>
>> >> >> rce)
>> >> >>
>> >> >>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
>> >> >>>         at java/lang/J9VMInternals.initialize
>> >> (J9VMInternals.java:185)
>> >> >>>         at
>> com/ibm/icu4jni/charset/CharsetProviderICU.charsetForName
>> >> (Unkno
>> >> >>>
>> >> >> wn Source)
>> >> >>
>> >> >>>         at java/nio/charset/Charset.forNameInternal
>> >> (Charset.java:450)
>> >> >>>         at java/nio/charset/Charset.forName (Charset.java:513)
>> >> >>>         at java/lang/String.defaultCharset ( String.java:688)
>> >> >>>         at java/lang/String.<init> (String.java:252)
>> >> >>>         at org/apache/harmony/luni/util/Util.toString (Util.java
>> :74)
>> >> >>>         at java/lang/System.getPropertyList (Native Method)
>> >> >>>         at java/lang/System.ensureProperties (System.java:496)
>> >> >>>         at java/lang/System.<clinit> (System.java:55)
>> >> >>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
>> >> >>>         at java/lang/J9VMInternals.initialize
>> >> (J9VMInternals.java:185)
>> >> >>>         at java/lang/ClassLoader.initializeClassLoaders (
>> >> ClassLoader.java:6
>> >> >>>
>> >> >> 5)
>> >> >>
>> >> >>>         at java/lang/Thread.initialize (Thread.java:301)
>> >> >>>         at java/lang/Thread.<init> (Thread.java :120)
>> >> >>> JVMJ9VM015W Initialization error for library jclclear_23(14):
>> >> JVMJ9VM009E J
>> >> >>>
>> >> >> 9VMDllMain failed
>> >> >>
>> >> >>> HMYEXEL062E Internal VM error: Failed to create Java VM
>> >> >>> FAILED.
>> >> >>>
>> >> >>>
>> >> >>> On 1 June 2006 at 22:26, "Mikhail Loenko" < mloenko@gmail.com >
>> >> wrote:
>> >> >>>
>> >> >>>
>> >> >>>> 2006/6/1, Geir Magnusson Jr < geir@pobox.com>:
>> >> >>>>
>> >> >>>>
>> >> >>>>> Mikhail Loenko wrote:
>> >> >>>>>
>> >> >>>>>
>> >> >>>>>> 2006/6/1, Geir Magnusson Jr < geir@pobox.com >:
>> >> >>>>>>
>> >> >>>>>>
>> >> >>>>>>> I don't quite grok what you mean - could you post what you 
>> are
>> >> >>>>>>> suggesting in code here on -dev?
>> >> >>>>>>>
>> >> >>>>>>>
>> >> >>>>>> Index:
>> >> modules/luni/src/test/java/tests/api/java/net/Inet6AddressTest.ja
>> >> >>>>>>
>> >> >> v
>> >> >>
>> >> >>>>>>
>> >> >>>>>>
>> >> >>>> a
>> >> >>>>
>> >> >>>>
>> >> >>>>>>
>> >> ===================================================================
>> >> >>>>>> ---
>> >> >>>>>>
>> >> modules/luni/src/test/java/tests/api/java/net/Inet6AddressTest.java
>> >> >>>>>> (revision
>> >> >>>>>> 410480)
>> >> >>>>>> +++
>> >> >>>>>>
>> >> modules/luni/src/test/java/tests/api/java/net/Inet6AddressTest.java
>> >>
>> >> >>>>>> (working
>> >> >>>>>> copy)
>> >> >>>>>> @@ -785,7 +785,7 @@
>> >> >>>>>>                 "FFFF:FFFF:FFFF:FFFF:FFFF:FFFF: 
>> 255.255.255.255
>> ",
>> >> >>>>>>                 "0:0:0:0:0:0:0:0", "0:0:0:0:0:0: 0.0.0.0" };
>> >> >>>>>>
>> >> >>>>>> -        String invalidIPAddresses[] = { ":", "FFFF:FFFF",
>> "1:1",
>> >> >>>>>> "::1.2.3.444" };
>> >> >>>>>> +        String invalidIPAddresses[] = { "FFFF:FFFF" };
>> >> >>>>>>
>> >> >>>>>>         for (int i = 0; i < validIPAddresses.length; i++) {
>> >> >>>>>>
>> >> >>>>>>
>> >> >>>>>>
>> >> >>>>>>
>> >> >>>>> Thx - why do you want to remove testing the others?  Because 
>> you
>> >> believe
>> >> >>>>> that it is in effect testing the resolver in the OS since each
>> >> returns
>> >> >>>>> the same answer,and thus it's testing the same thing over and
>> >> over?
>> >> >>>>>
>> >> >>>>>
>> >> >>>> I believe so, but I'd like to hear confirmation from code 
>> authors
>> >> >>>>
>> >> >>>> Thanks,
>> >> >>>> Mikhail
>> >> >>>>
>> >> >>>>
>> >> >>>
>> >> >>>
>> >> ---------------------------------------------------------------------
>> >> >>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>> >> >>> To unsubscribe, e-mail: 
>> harmony-dev-unsubscribe@incubator.apache.org
>>
>> >> >>> For additional commands, e-mail:
>> >> harmony-dev-help@incubator.apache.org
>> >> >>>
>> >> >>>
>> >> >>>
>> >> >>>
>> >> >> --
>> >> >> Paulex Yang
>> >> >> China Software Development Lab
>> >> >> IBM
>> >> >>
>> >> >>
>> >> >>
>> >> >>
>> ---------------------------------------------------------------------
>> >> >> Terms of use : http://incubator.apache.org/harmony/mailing.html
>> >> >> To unsubscribe, e-mail: 
>> harmony-dev-unsubscribe@incubator.apache.org
>> >> >> For additional commands, e-mail:
>> >> harmony-dev-help@incubator.apache.org
>> >> >>
>> >> >
>> >> >
>> >> >
>> >> > 
>> ---------------------------------------------------------------------
>>
>> >> > Terms of use : http://incubator.apache.org/harmony/mailing.html
>> >> > To unsubscribe, e-mail: 
>> harmony-dev-unsubscribe@incubator.apache.org
>> >> > For additional commands, e-mail:
>> harmony-dev-help@incubator.apache.org
>> >> >
>> >> >
>> >> >
>> >>
>> >>
>> >> --
>> >> Paulex Yang
>> >> China Software Development Lab
>> >> IBM
>> >>
>> >>
>> >>
>> >> ---------------------------------------------------------------------
>> >> Terms of use : http://incubator.apache.org/harmony/mailing.html
>> >> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>> >> For additional commands, e-mail: 
>> harmony-dev-help@incubator.apache.org
>> >>
>> >>
>> >
>>
>>
>> -- 
>> Paulex Yang
>> China Software Development Lab
>> IBM
>>
>>
>>
>> ---------------------------------------------------------------------
>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>
>>
>


-- 
Paulex Yang
China Software Development Lab
IBM



---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org


Re: [classlib] [testing] Coverage (was Re: 37% of total test execution time is spent in a single test)

Posted by Vladimir Ivanov <iv...@gmail.com>.
Latest Harmony API source coverage by Harmony API unit tests results I
stored at wiki page

http://wiki.apache.org/harmony/Coverage_information



I'm going to refresh it bi-weekly (seems, it is enough for coverage).



>I think we have got a agreement on the test naming convention[1], but
>for sure, there have been many (legacy) test cases before this
>agreement, and I think the volunteer is highly welcome to provide patch
>for them.
>
>[1]http://incubator.apache.org/harmony/subcomponents/classlibrary/testing.html

>

If nobody objects I'm going to look through the unit tests to correct
package names according to the agreement (where needed).



 Thanks,

   Vladimir



On 6/8/06, Paulex Yang <paulex.yang@gmail.com > wrote:
>
> Vladimir,
>
> Vladimir Ivanov wrote:
> > Thanks Paulex!
> >
> > I did the same, but could not send results due to spam filter J
> > Observations:
> >
> >   1. Coverage results look pretty much similar.
> >   2. Exclude list looks pretty much similar too, but, looks like it
> >   depends on the way of data collection (I didn't run ant task and the
> > list is
> >   a little bit different).
> Great.
> >
> > In any case, I think, when we run harmony on another VM exclude list
> will
> > have to be updated.
> >
> >
> >
> > May be we can start publishing the coverage information on wiki pages
> and
> > provide some updates time to time (I can do it)?
> >
> +1,  and of course, you can only if  no one in the mailing list objects
> and you'll have my welcome, and I think it will be even greater if these
> reports can be generated regularly like what JAPI is doing:)
> >
> >
> > One note:
> >
> > I noticed that different unit tests have very different package names
> >
> > Now the directory with all built tests copied to one place looks like:
> I think we have got a agreement on the test naming convention[1], but
> for sure, there have been many (legacy) test cases before this
> agreement, and I think the volunteer is highly welcome to provide patch
> for them.
>
> [1]http://incubator.apache.org/harmony/subcomponents/classlibrary/testing.html
>
> >
> >
> >
> > C:\coverage\tests\test>ls
> >
> > GZIPOutClose2.txt        api           config         javax
> > tests
> >
> > GZIPOutFinish.txt        api.injected  dazzle         org            xml
>
> >
> > GZIPOutWrite.txt         binary        gif            prefs
> >
> > Inet6Address.golden.ser  bundles       impl           serialization
> >
> > JDK2-3gabba.zip          com           impl.injected   test.txt
> >
> >
> >
> > I think, it would be good if tests had unified package names.
> >
> > Why? – so far, just common sense, just to have an order in test suite
> >
> > Organization (if consider all unit tests as solid test suite).
> >
> > Thanks,
> >  Vladimir
> >
> > For example, my exclude list for java.io is:
> > -java.io.BufferedInputStream ,
> > -java.io.BufferedOutputStream,
> > -java.io.File ,
> > -java.io.FileChannelFactory,
> > -java.io.FileDescriptor,
> > -java.io.FileInputStream,
> > -java.io.FileOutputStream,
> > -java.io.FilterInputStream ,
> > -java.io.FilterOutputStream,
> > - java.io.InputStream,
> > -java.io.OutputStream,
> > -java.io.ObjectStreamField,
> > -java.io.PrintStream
> >
> >
> > On 6/6/06, Paulex Yang < paulex.yang@gmail.com > wrote:
> >>
> >> I've attach the scripts and excluded class lists to JIRA,  please refer
> >> to https://issues.apache.org/jira/browse/HARMONY-564 .  Enjoy it:).
> >>
> >> Mark Hindess wrote:
> >> > On 2 June 2006 at 10:37, Paulex Yang < paulex.yang@gmail.com> wrote:
> >> >
> >> >> Mark,
> >> >>
> >> >> I'm glad that there is someone else has interest on emma, I've
> >> tried it
> >> >> before. AFAIK, emma works by instrumentation, but sometimes for
> >> classes
> >> >> in bootclasspath, the instrumentation cannot work, there are two
> >> cases:
> >>
> >> >> 1. Some instrumented classes cannot be loaded by VM.
> >> >> 2. Some classes cannot be instrumented
> >> >>
> >> >> I have tried to look more inside to find some way to work around
> >> but I
> >> >> haven't got enough time yet.
> >> >>
> >> >> Specifically for nio-channel module, I had a list for these two
> cases
> >> (I
> >> >> believe the data is a little outdated and should be reevaluated)
> >> >> case 1.
> >> >> BaseByteBuffer.class
> >> >> Buffer.class
> >> >> BufferFactory.class
> >> >> ByteBuffer.class
> >> >> CharArrayBuffer.class
> >> >> CharBuffer.class
> >> >> HeapByteBuffer.class
> >> >> ReadWriteCharArrayBuffer.class
> >> >> ReadWriteHeapByteBuffer.class
> >> >> FileChannel.class
> >> >> AbstractInterruptibleChannel.class
> >> >> FileChannelImpl.class
> >> >> WriteOnlyFileChannel.class
> >> >> LockManager.class
> >> >> LockManager$1.class
> >> >> ReadOnlyFileChannel.class
> >> >>
> >> >> case 2:
> >> >> ByteChannel.class
> >> >> Channel.class
> >> >> GatheringByteChannel.class
> >> >> InterruptibleChannel.class
> >> >> WritableByteChannel.class
> >> >>
> >> >> And I have got some ant script and more excluded list for emma, if
> >> >> anyone has interests, I can upload it to JIRA.
> >> >>
> >> >
> >> > Yes!
> >> >
> >> > -Mark.
> >> >
> >> >
> >> >> Mark Hindess wrote:
> >> >>
> >> >>> Anyone tried using emma (emma.sf.net) to look at test coverage
> >> for our
> >> >>> tests?  That might help ensure we don't trim them down too much?
> >> >>>
> >> >>> I suspect we will soon reach a point where there are significantly
> >> fewer
> >> >>> obvious opportunities for small contributions - e.g.
> >> 1.5generification,
> >> >>> etc from looking at JAPI reports.  Having test coverage reports
> >> might
> >> >>> help by highlighting areas for improvement.
> >> >>>
> >> >>> I had a quick try with:
> >> >>>
> >> >>>   ant -f make/build.xml
> >> >>>   ant -f modules/nio/make/build.xml test
> >> >>>   java -cp emma.jar emma instr -m overwrite -cp
> >> deploy/jdk/jre/lib/boot/nio
> >> >>>
> >> >> .jar
> >> >>
> >> >>> and then:
> >> >>>
> >> >>>   java -Xbootclasspath/p: emma.jar \
> >> >>>        -cp build/tests:modules/nio/bin/test:junit.jar:. \
> >> >>>        junit.textui.TestRunner \
> >> >>>        org.apache.harmony.tests.java.nio.channels.SocketChannelTest
> >> >>>
> >> >>> (Running all the tests is a little tricky because test depends on
> >> build,
> >> >>> which depends on clean which wipes out the instr'd jar. ;-( )
> >> >>>
> >> >>> Sadly it dies horribly with the stack trace below.  I'll take
> >> another
> >> >>> look when I have a little more time, but I thought I'd mention it
> in
> >> >>> case anyone else had any thoughts on the subject.
> >> >>>
> >> >>> Regards,
> >> >>>
> >> >>> Mark.
> >> >>>
> >> >>> Exception in thread "main" java/lang/ExceptionInInitializerError
> >> >>>         at java/lang/J9VMInternals.initialize
> >> (J9VMInternals.java:203)
> >> >>>         at java/nio/Buffer.$VRi ( Buffer.java:4294967295)
> >> >>>         at java/nio/Buffer.<clinit> ( Buffer.java:4294967295)
> >> >>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
> >> >>>         at java/lang/J9VMInternals.initialize ( J9VMInternals.java
> >> :185)
> >> >>>         at java/lang/J9VMInternals.initialize
> >> (J9VMInternals.java:154)
> >> >>>         at java/lang/String.getBytes (String.java:783)
> >> >>>         at org/apache/harmony/luni/util/Util.getBytes (
> >> Util.java:54)
> >> >>>         at java/lang/ClassLoader.loadLibraryWithPath
> >> (ClassLoader.java
> >> :998)
> >> >>>         at java/lang/ClassLoader.loadLibraryWithClassLoader (
> >> ClassLoader.ja
> >> >>>
> >> >> va:970)
> >> >>
> >> >>>         at java/lang/System.loadLibrary (System.java:706)
> >> >>>         at com/ibm/icu4jni/common/ICU4JNILoader.loadLibrary
> (Unknown
> >> Source
> >> >>>
> >> >> )
> >> >>
> >> >>>         at com/ibm/icu4jni/converters/NativeConverter.<clinit>
> >> (Unknown Sou
> >> >>>
> >> >> rce)
> >> >>
> >> >>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
> >> >>>         at java/lang/J9VMInternals.initialize
> >> ( J9VMInternals.java:185)
> >> >>>         at
> com/ibm/icu4jni/charset/CharsetProviderICU.charsetForName
> >> (Unkno
> >> >>>
> >> >> wn Source)
> >> >>
> >> >>>         at java/nio/charset/Charset.forNameInternal
> >> (Charset.java:450)
> >> >>>         at java/nio/charset/Charset.forName (Charset.java:513)
> >> >>>         at java/lang/String.defaultCharset ( String.java:688)
> >> >>>         at java/lang/String.<init> ( String.java:252)
> >> >>>         at org/apache/harmony/luni/util/Util.toString (Util.java
> :74)
> >> >>>         at java/lang/System.getPropertyList (Native Method)
> >> >>>         at java/lang/System.ensureProperties ( System.java:496)
> >> >>>         at java/lang/System.<clinit> (System.java:55)
> >> >>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
> >> >>>         at java/lang/J9VMInternals.initialize
> >> (J9VMInternals.java:185)
> >> >>>         at java/lang/ClassLoader.initializeClassLoaders (
> >> ClassLoader.java:6
> >> >>>
> >> >> 5)
> >> >>
> >> >>>         at java/lang/Thread.initialize (Thread.java:301)
> >> >>>         at java/lang/Thread.<init> (Thread.java:120)
> >> >>> java/lang/NullPointerException
> >> >>>         at java/lang/Throwable.printStackTrace ( Throwable.java
> :231)
> >> >>>         at com/vladium/emma/rt/RTExitHook.createClassLoaderClosure
> >> (RTExitH
> >> >>>
> >> >> ook.java:63)
> >> >>
> >> >>>         at com/vladium/emma/rt/RT.reset ( RT.java:92)
> >> >>>         at com/vladium/emma/rt/RT.<clinit> (RT.java:238)
> >> >>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
> >> >>>         at java/lang/J9VMInternals.initialize ( J9VMInternals.java
> >> :185)
> >> >>>         at java/nio/Buffer.$VRi ( Buffer.java:4294967295)
> >> >>>         at java/nio/Buffer.<clinit> (Buffer.java:4294967295)
> >> >>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
> >> >>>         at java/lang/J9VMInternals.initialize
> >> (J9VMInternals.java:185)
> >> >>>         at java/lang/J9VMInternals.initialize
> >> (J9VMInternals.java:154)
> >> >>>         at java/lang/String.getBytes ( String.java :783)
> >> >>>         at org/apache/harmony/luni/util/Util.getBytes (Util.java
> :54)
> >> >>>         at java/lang/ClassLoader.loadLibraryWithPath
> >> (ClassLoader.java
> >> :998)
> >> >>>         at java/lang/ClassLoader.loadLibraryWithClassLoader (
> >> ClassLoader.ja
> >> >>>
> >> >> va:970)
> >> >>
> >> >>>         at java/lang/System.loadLibrary ( System.java:706)
> >> >>>         at com/ibm/icu4jni/common/ICU4JNILoader.loadLibrary
> (Unknown
> >> Source
> >> >>>
> >> >> )
> >> >>
> >> >>>         at com/ibm/icu4jni/converters/NativeConverter.<clinit>
> >> (Unknown Sou
> >> >>>
> >> >> rce)
> >> >>
> >> >>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
> >> >>>         at java/lang/J9VMInternals.initialize
> >> (J9VMInternals.java:185)
> >> >>>         at
> com/ibm/icu4jni/charset/CharsetProviderICU.charsetForName
> >> (Unkno
> >> >>>
> >> >> wn Source)
> >> >>
> >> >>>         at java/nio/charset/Charset.forNameInternal
> >> (Charset.java:450)
> >> >>>         at java/nio/charset/Charset.forName (Charset.java:513)
> >> >>>         at java/lang/String.defaultCharset ( String.java:688)
> >> >>>         at java/lang/String.<init> (String.java:252)
> >> >>>         at org/apache/harmony/luni/util/Util.toString (Util.java
> :74)
> >> >>>         at java/lang/System.getPropertyList (Native Method)
> >> >>>         at java/lang/System.ensureProperties (System.java:496)
> >> >>>         at java/lang/System.<clinit> (System.java:55)
> >> >>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
> >> >>>         at java/lang/J9VMInternals.initialize
> >> (J9VMInternals.java:185)
> >> >>>         at java/lang/ClassLoader.initializeClassLoaders (
> >> ClassLoader.java:6
> >> >>>
> >> >> 5)
> >> >>
> >> >>>         at java/lang/Thread.initialize (Thread.java:301)
> >> >>>         at java/lang/Thread.<init> (Thread.java :120)
> >> >>> JVMJ9VM015W Initialization error for library jclclear_23(14):
> >> JVMJ9VM009E J
> >> >>>
> >> >> 9VMDllMain failed
> >> >>
> >> >>> HMYEXEL062E Internal VM error: Failed to create Java VM
> >> >>> FAILED.
> >> >>>
> >> >>>
> >> >>> On 1 June 2006 at 22:26, "Mikhail Loenko" < mloenko@gmail.com >
> >> wrote:
> >> >>>
> >> >>>
> >> >>>> 2006/6/1, Geir Magnusson Jr < geir@pobox.com>:
> >> >>>>
> >> >>>>
> >> >>>>> Mikhail Loenko wrote:
> >> >>>>>
> >> >>>>>
> >> >>>>>> 2006/6/1, Geir Magnusson Jr < geir@pobox.com >:
> >> >>>>>>
> >> >>>>>>
> >> >>>>>>> I don't quite grok what you mean - could you post what you are
> >> >>>>>>> suggesting in code here on -dev?
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>> Index:
> >> modules/luni/src/test/java/tests/api/java/net/Inet6AddressTest.ja
> >> >>>>>>
> >> >> v
> >> >>
> >> >>>>>>
> >> >>>>>>
> >> >>>> a
> >> >>>>
> >> >>>>
> >> >>>>>>
> >> ===================================================================
> >> >>>>>> ---
> >> >>>>>>
> >> modules/luni/src/test/java/tests/api/java/net/Inet6AddressTest.java
> >> >>>>>> (revision
> >> >>>>>> 410480)
> >> >>>>>> +++
> >> >>>>>>
> >> modules/luni/src/test/java/tests/api/java/net/Inet6AddressTest.java
> >>
> >> >>>>>> (working
> >> >>>>>> copy)
> >> >>>>>> @@ -785,7 +785,7 @@
> >> >>>>>>                 "FFFF:FFFF:FFFF:FFFF:FFFF:FFFF: 255.255.255.255
> ",
> >> >>>>>>                 "0:0:0:0:0:0:0:0", "0:0:0:0:0:0: 0.0.0.0" };
> >> >>>>>>
> >> >>>>>> -        String invalidIPAddresses[] = { ":", "FFFF:FFFF",
> "1:1",
> >> >>>>>> "::1.2.3.444" };
> >> >>>>>> +        String invalidIPAddresses[] = { "FFFF:FFFF" };
> >> >>>>>>
> >> >>>>>>         for (int i = 0; i < validIPAddresses.length; i++) {
> >> >>>>>>
> >> >>>>>>
> >> >>>>>>
> >> >>>>>>
> >> >>>>> Thx - why do you want to remove testing the others?  Because you
> >> believe
> >> >>>>> that it is in effect testing the resolver in the OS since each
> >> returns
> >> >>>>> the same answer,and thus it's testing the same thing over and
> >> over?
> >> >>>>>
> >> >>>>>
> >> >>>> I believe so, but I'd like to hear confirmation from code authors
> >> >>>>
> >> >>>> Thanks,
> >> >>>> Mikhail
> >> >>>>
> >> >>>>
> >> >>>
> >> >>>
> >> ---------------------------------------------------------------------
> >> >>> Terms of use : http://incubator.apache.org/harmony/mailing.html
> >> >>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>
> >> >>> For additional commands, e-mail:
> >> harmony-dev-help@incubator.apache.org
> >> >>>
> >> >>>
> >> >>>
> >> >>>
> >> >> --
> >> >> Paulex Yang
> >> >> China Software Development Lab
> >> >> IBM
> >> >>
> >> >>
> >> >>
> >> >>
> ---------------------------------------------------------------------
> >> >> Terms of use : http://incubator.apache.org/harmony/mailing.html
> >> >> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> >> >> For additional commands, e-mail:
> >> harmony-dev-help@incubator.apache.org
> >> >>
> >> >
> >> >
> >> >
> >> > ---------------------------------------------------------------------
>
> >> > Terms of use : http://incubator.apache.org/harmony/mailing.html
> >> > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> >> > For additional commands, e-mail:
> harmony-dev-help@incubator.apache.org
> >> >
> >> >
> >> >
> >>
> >>
> >> --
> >> Paulex Yang
> >> China Software Development Lab
> >> IBM
> >>
> >>
> >>
> >> ---------------------------------------------------------------------
> >> Terms of use : http://incubator.apache.org/harmony/mailing.html
> >> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> >> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >>
> >>
> >
>
>
> --
> Paulex Yang
> China Software Development Lab
> IBM
>
>
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>

Re: [classlib] [testing] Coverage (was Re: 37% of total test execution time is spent in a single test)

Posted by Paulex Yang <pa...@gmail.com>.
Vladimir,

Vladimir Ivanov wrote:
> Thanks Paulex!
>
> I did the same, but could not send results due to spam filter J
> Observations:
>
>   1. Coverage results look pretty much similar.
>   2. Exclude list looks pretty much similar too, but, looks like it
>   depends on the way of data collection (I didn't run ant task and the 
> list is
>   a little bit different).
Great.
>
> In any case, I think, when we run harmony on another VM exclude list will
> have to be updated.
>
>
>
> May be we can start publishing the coverage information on wiki pages and
> provide some updates time to time (I can do it)?
>
+1,  and of course, you can only if  no one in the mailing list objects 
and you'll have my welcome, and I think it will be even greater if these 
reports can be generated regularly like what JAPI is doing:)
>
>
> One note:
>
> I noticed that different unit tests have very different package names
>
> Now the directory with all built tests copied to one place looks like:
I think we have got a agreement on the test naming convention[1], but 
for sure, there have been many (legacy) test cases before this 
agreement, and I think the volunteer is highly welcome to provide patch 
for them.

[1]http://incubator.apache.org/harmony/subcomponents/classlibrary/testing.html
>
>
>
> C:\coverage\tests\test>ls
>
> GZIPOutClose2.txt        api           config         javax          
> tests
>
> GZIPOutFinish.txt        api.injected  dazzle         org            xml
>
> GZIPOutWrite.txt         binary        gif            prefs
>
> Inet6Address.golden.ser  bundles       impl           serialization
>
> JDK2-3gabba.zip          com           impl.injected  test.txt
>
>
>
> I think, it would be good if tests had unified package names.
>
> Why? – so far, just common sense, just to have an order in test suite
>
> Organization (if consider all unit tests as solid test suite).
>
> Thanks,
>  Vladimir
>
> For example, my exclude list for java.io is:
> -java.io.BufferedInputStream,
> -java.io.BufferedOutputStream,
> -java.io.File,
> -java.io.FileChannelFactory,
> -java.io.FileDescriptor,
> -java.io.FileInputStream,
> -java.io.FileOutputStream,
> -java.io.FilterInputStream ,
> -java.io.FilterOutputStream,
> -java.io.InputStream,
> -java.io.OutputStream,
> -java.io.ObjectStreamField,
> -java.io.PrintStream
>
>
> On 6/6/06, Paulex Yang <pa...@gmail.com> wrote:
>>
>> I've attach the scripts and excluded class lists to JIRA,  please refer
>> to https://issues.apache.org/jira/browse/HARMONY-564.  Enjoy it:).
>>
>> Mark Hindess wrote:
>> > On 2 June 2006 at 10:37, Paulex Yang <pa...@gmail.com> wrote:
>> >
>> >> Mark,
>> >>
>> >> I'm glad that there is someone else has interest on emma, I've 
>> tried it
>> >> before. AFAIK, emma works by instrumentation, but sometimes for 
>> classes
>> >> in bootclasspath, the instrumentation cannot work, there are two 
>> cases:
>>
>> >> 1. Some instrumented classes cannot be loaded by VM.
>> >> 2. Some classes cannot be instrumented
>> >>
>> >> I have tried to look more inside to find some way to work around 
>> but I
>> >> haven't got enough time yet.
>> >>
>> >> Specifically for nio-channel module, I had a list for these two cases
>> (I
>> >> believe the data is a little outdated and should be reevaluated)
>> >> case 1.
>> >> BaseByteBuffer.class
>> >> Buffer.class
>> >> BufferFactory.class
>> >> ByteBuffer.class
>> >> CharArrayBuffer.class
>> >> CharBuffer.class
>> >> HeapByteBuffer.class
>> >> ReadWriteCharArrayBuffer.class
>> >> ReadWriteHeapByteBuffer.class
>> >> FileChannel.class
>> >> AbstractInterruptibleChannel.class
>> >> FileChannelImpl.class
>> >> WriteOnlyFileChannel.class
>> >> LockManager.class
>> >> LockManager$1.class
>> >> ReadOnlyFileChannel.class
>> >>
>> >> case 2:
>> >> ByteChannel.class
>> >> Channel.class
>> >> GatheringByteChannel.class
>> >> InterruptibleChannel.class
>> >> WritableByteChannel.class
>> >>
>> >> And I have got some ant script and more excluded list for emma, if
>> >> anyone has interests, I can upload it to JIRA.
>> >>
>> >
>> > Yes!
>> >
>> > -Mark.
>> >
>> >
>> >> Mark Hindess wrote:
>> >>
>> >>> Anyone tried using emma (emma.sf.net) to look at test coverage 
>> for our
>> >>> tests?  That might help ensure we don't trim them down too much?
>> >>>
>> >>> I suspect we will soon reach a point where there are significantly
>> fewer
>> >>> obvious opportunities for small contributions - e.g. 
>> 1.5generification,
>> >>> etc from looking at JAPI reports.  Having test coverage reports 
>> might
>> >>> help by highlighting areas for improvement.
>> >>>
>> >>> I had a quick try with:
>> >>>
>> >>>   ant -f make/build.xml
>> >>>   ant -f modules/nio/make/build.xml test
>> >>>   java -cp emma.jar emma instr -m overwrite -cp
>> deploy/jdk/jre/lib/boot/nio
>> >>>
>> >> .jar
>> >>
>> >>> and then:
>> >>>
>> >>>   java -Xbootclasspath/p: emma.jar \
>> >>>        -cp build/tests:modules/nio/bin/test:junit.jar:. \
>> >>>        junit.textui.TestRunner \
>> >>>        org.apache.harmony.tests.java.nio.channels.SocketChannelTest
>> >>>
>> >>> (Running all the tests is a little tricky because test depends on
>> build,
>> >>> which depends on clean which wipes out the instr'd jar. ;-( )
>> >>>
>> >>> Sadly it dies horribly with the stack trace below.  I'll take 
>> another
>> >>> look when I have a little more time, but I thought I'd mention it in
>> >>> case anyone else had any thoughts on the subject.
>> >>>
>> >>> Regards,
>> >>>
>> >>> Mark.
>> >>>
>> >>> Exception in thread "main" java/lang/ExceptionInInitializerError
>> >>>         at java/lang/J9VMInternals.initialize 
>> (J9VMInternals.java:203)
>> >>>         at java/nio/Buffer.$VRi ( Buffer.java:4294967295)
>> >>>         at java/nio/Buffer.<clinit> (Buffer.java:4294967295)
>> >>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
>> >>>         at java/lang/J9VMInternals.initialize ( J9VMInternals.java
>> :185)
>> >>>         at java/lang/J9VMInternals.initialize 
>> (J9VMInternals.java:154)
>> >>>         at java/lang/String.getBytes (String.java:783)
>> >>>         at org/apache/harmony/luni/util/Util.getBytes ( 
>> Util.java:54)
>> >>>         at java/lang/ClassLoader.loadLibraryWithPath 
>> (ClassLoader.java
>> :998)
>> >>>         at java/lang/ClassLoader.loadLibraryWithClassLoader (
>> ClassLoader.ja
>> >>>
>> >> va:970)
>> >>
>> >>>         at java/lang/System.loadLibrary (System.java:706)
>> >>>         at com/ibm/icu4jni/common/ICU4JNILoader.loadLibrary (Unknown
>> Source
>> >>>
>> >> )
>> >>
>> >>>         at com/ibm/icu4jni/converters/NativeConverter.<clinit>
>> (Unknown Sou
>> >>>
>> >> rce)
>> >>
>> >>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
>> >>>         at java/lang/J9VMInternals.initialize 
>> (J9VMInternals.java:185)
>> >>>         at com/ibm/icu4jni/charset/CharsetProviderICU.charsetForName
>> (Unkno
>> >>>
>> >> wn Source)
>> >>
>> >>>         at java/nio/charset/Charset.forNameInternal 
>> (Charset.java:450)
>> >>>         at java/nio/charset/Charset.forName (Charset.java:513)
>> >>>         at java/lang/String.defaultCharset ( String.java:688)
>> >>>         at java/lang/String.<init> (String.java:252)
>> >>>         at org/apache/harmony/luni/util/Util.toString (Util.java:74)
>> >>>         at java/lang/System.getPropertyList (Native Method)
>> >>>         at java/lang/System.ensureProperties (System.java:496)
>> >>>         at java/lang/System.<clinit> (System.java:55)
>> >>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
>> >>>         at java/lang/J9VMInternals.initialize 
>> (J9VMInternals.java:185)
>> >>>         at java/lang/ClassLoader.initializeClassLoaders (
>> ClassLoader.java:6
>> >>>
>> >> 5)
>> >>
>> >>>         at java/lang/Thread.initialize (Thread.java:301)
>> >>>         at java/lang/Thread.<init> (Thread.java:120)
>> >>> java/lang/NullPointerException
>> >>>         at java/lang/Throwable.printStackTrace ( Throwable.java:231)
>> >>>         at com/vladium/emma/rt/RTExitHook.createClassLoaderClosure
>> (RTExitH
>> >>>
>> >> ook.java:63)
>> >>
>> >>>         at com/vladium/emma/rt/RT.reset ( RT.java:92)
>> >>>         at com/vladium/emma/rt/RT.<clinit> (RT.java:238)
>> >>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
>> >>>         at java/lang/J9VMInternals.initialize ( J9VMInternals.java
>> :185)
>> >>>         at java/nio/Buffer.$VRi (Buffer.java:4294967295)
>> >>>         at java/nio/Buffer.<clinit> (Buffer.java:4294967295)
>> >>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
>> >>>         at java/lang/J9VMInternals.initialize 
>> (J9VMInternals.java:185)
>> >>>         at java/lang/J9VMInternals.initialize 
>> (J9VMInternals.java:154)
>> >>>         at java/lang/String.getBytes ( String.java:783)
>> >>>         at org/apache/harmony/luni/util/Util.getBytes (Util.java:54)
>> >>>         at java/lang/ClassLoader.loadLibraryWithPath 
>> (ClassLoader.java
>> :998)
>> >>>         at java/lang/ClassLoader.loadLibraryWithClassLoader (
>> ClassLoader.ja
>> >>>
>> >> va:970)
>> >>
>> >>>         at java/lang/System.loadLibrary (System.java:706)
>> >>>         at com/ibm/icu4jni/common/ICU4JNILoader.loadLibrary (Unknown
>> Source
>> >>>
>> >> )
>> >>
>> >>>         at com/ibm/icu4jni/converters/NativeConverter.<clinit>
>> (Unknown Sou
>> >>>
>> >> rce)
>> >>
>> >>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
>> >>>         at java/lang/J9VMInternals.initialize 
>> (J9VMInternals.java:185)
>> >>>         at com/ibm/icu4jni/charset/CharsetProviderICU.charsetForName
>> (Unkno
>> >>>
>> >> wn Source)
>> >>
>> >>>         at java/nio/charset/Charset.forNameInternal 
>> (Charset.java:450)
>> >>>         at java/nio/charset/Charset.forName (Charset.java:513)
>> >>>         at java/lang/String.defaultCharset ( String.java:688)
>> >>>         at java/lang/String.<init> (String.java:252)
>> >>>         at org/apache/harmony/luni/util/Util.toString (Util.java:74)
>> >>>         at java/lang/System.getPropertyList (Native Method)
>> >>>         at java/lang/System.ensureProperties (System.java:496)
>> >>>         at java/lang/System.<clinit> (System.java:55)
>> >>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
>> >>>         at java/lang/J9VMInternals.initialize 
>> (J9VMInternals.java:185)
>> >>>         at java/lang/ClassLoader.initializeClassLoaders (
>> ClassLoader.java:6
>> >>>
>> >> 5)
>> >>
>> >>>         at java/lang/Thread.initialize (Thread.java:301)
>> >>>         at java/lang/Thread.<init> (Thread.java:120)
>> >>> JVMJ9VM015W Initialization error for library jclclear_23(14):
>> JVMJ9VM009E J
>> >>>
>> >> 9VMDllMain failed
>> >>
>> >>> HMYEXEL062E Internal VM error: Failed to create Java VM
>> >>> FAILED.
>> >>>
>> >>>
>> >>> On 1 June 2006 at 22:26, "Mikhail Loenko" < mloenko@gmail.com> 
>> wrote:
>> >>>
>> >>>
>> >>>> 2006/6/1, Geir Magnusson Jr <ge...@pobox.com>:
>> >>>>
>> >>>>
>> >>>>> Mikhail Loenko wrote:
>> >>>>>
>> >>>>>
>> >>>>>> 2006/6/1, Geir Magnusson Jr <geir@pobox.com >:
>> >>>>>>
>> >>>>>>
>> >>>>>>> I don't quite grok what you mean - could you post what you are
>> >>>>>>> suggesting in code here on -dev?
>> >>>>>>>
>> >>>>>>>
>> >>>>>> Index:
>> modules/luni/src/test/java/tests/api/java/net/Inet6AddressTest.ja
>> >>>>>>
>> >> v
>> >>
>> >>>>>>
>> >>>>>>
>> >>>> a
>> >>>>
>> >>>>
>> >>>>>> 
>> ===================================================================
>> >>>>>> ---
>> >>>>>> 
>> modules/luni/src/test/java/tests/api/java/net/Inet6AddressTest.java
>> >>>>>> (revision
>> >>>>>> 410480)
>> >>>>>> +++
>> >>>>>> 
>> modules/luni/src/test/java/tests/api/java/net/Inet6AddressTest.java
>>
>> >>>>>> (working
>> >>>>>> copy)
>> >>>>>> @@ -785,7 +785,7 @@
>> >>>>>>                 "FFFF:FFFF:FFFF:FFFF:FFFF:FFFF: 255.255.255.255",
>> >>>>>>                 "0:0:0:0:0:0:0:0", "0:0:0:0:0:0:0.0.0.0" };
>> >>>>>>
>> >>>>>> -        String invalidIPAddresses[] = { ":", "FFFF:FFFF", "1:1",
>> >>>>>> "::1.2.3.444" };
>> >>>>>> +        String invalidIPAddresses[] = { "FFFF:FFFF" };
>> >>>>>>
>> >>>>>>         for (int i = 0; i < validIPAddresses.length; i++) {
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>> Thx - why do you want to remove testing the others?  Because you
>> believe
>> >>>>> that it is in effect testing the resolver in the OS since each
>> returns
>> >>>>> the same answer,and thus it's testing the same thing over and 
>> over?
>> >>>>>
>> >>>>>
>> >>>> I believe so, but I'd like to hear confirmation from code authors
>> >>>>
>> >>>> Thanks,
>> >>>> Mikhail
>> >>>>
>> >>>>
>> >>>
>> >>> 
>> ---------------------------------------------------------------------
>> >>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>> >>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>> >>> For additional commands, e-mail: 
>> harmony-dev-help@incubator.apache.org
>> >>>
>> >>>
>> >>>
>> >>>
>> >> --
>> >> Paulex Yang
>> >> China Software Development Lab
>> >> IBM
>> >>
>> >>
>> >>
>> >> ---------------------------------------------------------------------
>> >> Terms of use : http://incubator.apache.org/harmony/mailing.html
>> >> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>> >> For additional commands, e-mail: 
>> harmony-dev-help@incubator.apache.org
>> >>
>> >
>> >
>> >
>> > ---------------------------------------------------------------------
>> > Terms of use : http://incubator.apache.org/harmony/mailing.html
>> > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>> > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>> >
>> >
>> >
>>
>>
>> -- 
>> Paulex Yang
>> China Software Development Lab
>> IBM
>>
>>
>>
>> ---------------------------------------------------------------------
>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>
>>
>


-- 
Paulex Yang
China Software Development Lab
IBM



---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org


Re: [classlib] [testing] Coverage (was Re: 37% of total test execution time is spent in a single test)

Posted by Vladimir Ivanov <iv...@gmail.com>.
Thanks Paulex!

I did the same, but could not send results due to spam filter J
Observations:

   1. Coverage results look pretty much similar.
   2. Exclude list looks pretty much similar too, but, looks like it
   depends on the way of data collection (I didn't run ant task and the list is
   a little bit different).

 In any case, I think, when we run harmony on another VM exclude list will
have to be updated.



May be we can start publishing the coverage information on wiki pages and
provide some updates time to time (I can do it)?



One note:

I noticed that different unit tests have very different package names

Now the directory with all built tests copied to one place looks like:



C:\coverage\tests\test>ls

GZIPOutClose2.txt        api           config         javax          tests

GZIPOutFinish.txt        api.injected  dazzle         org            xml

GZIPOutWrite.txt         binary        gif            prefs

Inet6Address.golden.ser  bundles       impl           serialization

JDK2-3gabba.zip          com           impl.injected  test.txt



I think, it would be good if tests had unified package names.

Why? – so far, just common sense, just to have an order in test suite

Organization (if consider all unit tests as solid test suite).

 Thanks,
  Vladimir

For example, my exclude list for java.io is:
-java.io.BufferedInputStream,
-java.io.BufferedOutputStream,
-java.io.File,
-java.io.FileChannelFactory,
-java.io.FileDescriptor,
-java.io.FileInputStream,
-java.io.FileOutputStream,
-java.io.FilterInputStream ,
-java.io.FilterOutputStream,
-java.io.InputStream,
-java.io.OutputStream,
-java.io.ObjectStreamField,
-java.io.PrintStream


On 6/6/06, Paulex Yang <pa...@gmail.com> wrote:
>
> I've attach the scripts and excluded class lists to JIRA,  please refer
> to https://issues.apache.org/jira/browse/HARMONY-564.  Enjoy it:).
>
> Mark Hindess wrote:
> > On 2 June 2006 at 10:37, Paulex Yang <pa...@gmail.com> wrote:
> >
> >> Mark,
> >>
> >> I'm glad that there is someone else has interest on emma, I've tried it
> >> before. AFAIK, emma works by instrumentation, but sometimes for classes
> >> in bootclasspath, the instrumentation cannot work, there are two cases:
>
> >> 1. Some instrumented classes cannot be loaded by VM.
> >> 2. Some classes cannot be instrumented
> >>
> >> I have tried to look more inside to find some way to work around but I
> >> haven't got enough time yet.
> >>
> >> Specifically for nio-channel module, I had a list for these two cases
> (I
> >> believe the data is a little outdated and should be reevaluated)
> >> case 1.
> >> BaseByteBuffer.class
> >> Buffer.class
> >> BufferFactory.class
> >> ByteBuffer.class
> >> CharArrayBuffer.class
> >> CharBuffer.class
> >> HeapByteBuffer.class
> >> ReadWriteCharArrayBuffer.class
> >> ReadWriteHeapByteBuffer.class
> >> FileChannel.class
> >> AbstractInterruptibleChannel.class
> >> FileChannelImpl.class
> >> WriteOnlyFileChannel.class
> >> LockManager.class
> >> LockManager$1.class
> >> ReadOnlyFileChannel.class
> >>
> >> case 2:
> >> ByteChannel.class
> >> Channel.class
> >> GatheringByteChannel.class
> >> InterruptibleChannel.class
> >> WritableByteChannel.class
> >>
> >> And I have got some ant script and more excluded list for emma, if
> >> anyone has interests, I can upload it to JIRA.
> >>
> >
> > Yes!
> >
> > -Mark.
> >
> >
> >> Mark Hindess wrote:
> >>
> >>> Anyone tried using emma (emma.sf.net) to look at test coverage for our
> >>> tests?  That might help ensure we don't trim them down too much?
> >>>
> >>> I suspect we will soon reach a point where there are significantly
> fewer
> >>> obvious opportunities for small contributions - e.g. 1.5generification,
> >>> etc from looking at JAPI reports.  Having test coverage reports might
> >>> help by highlighting areas for improvement.
> >>>
> >>> I had a quick try with:
> >>>
> >>>   ant -f make/build.xml
> >>>   ant -f modules/nio/make/build.xml test
> >>>   java -cp emma.jar emma instr -m overwrite -cp
> deploy/jdk/jre/lib/boot/nio
> >>>
> >> .jar
> >>
> >>> and then:
> >>>
> >>>   java -Xbootclasspath/p: emma.jar \
> >>>        -cp build/tests:modules/nio/bin/test:junit.jar:. \
> >>>        junit.textui.TestRunner \
> >>>        org.apache.harmony.tests.java.nio.channels.SocketChannelTest
> >>>
> >>> (Running all the tests is a little tricky because test depends on
> build,
> >>> which depends on clean which wipes out the instr'd jar. ;-( )
> >>>
> >>> Sadly it dies horribly with the stack trace below.  I'll take another
> >>> look when I have a little more time, but I thought I'd mention it in
> >>> case anyone else had any thoughts on the subject.
> >>>
> >>> Regards,
> >>>
> >>> Mark.
> >>>
> >>> Exception in thread "main" java/lang/ExceptionInInitializerError
> >>>         at java/lang/J9VMInternals.initialize (J9VMInternals.java:203)
> >>>         at java/nio/Buffer.$VRi ( Buffer.java:4294967295)
> >>>         at java/nio/Buffer.<clinit> (Buffer.java:4294967295)
> >>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
> >>>         at java/lang/J9VMInternals.initialize ( J9VMInternals.java
> :185)
> >>>         at java/lang/J9VMInternals.initialize (J9VMInternals.java:154)
> >>>         at java/lang/String.getBytes (String.java:783)
> >>>         at org/apache/harmony/luni/util/Util.getBytes ( Util.java:54)
> >>>         at java/lang/ClassLoader.loadLibraryWithPath (ClassLoader.java
> :998)
> >>>         at java/lang/ClassLoader.loadLibraryWithClassLoader (
> ClassLoader.ja
> >>>
> >> va:970)
> >>
> >>>         at java/lang/System.loadLibrary (System.java:706)
> >>>         at com/ibm/icu4jni/common/ICU4JNILoader.loadLibrary (Unknown
> Source
> >>>
> >> )
> >>
> >>>         at com/ibm/icu4jni/converters/NativeConverter.<clinit>
> (Unknown Sou
> >>>
> >> rce)
> >>
> >>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
> >>>         at java/lang/J9VMInternals.initialize (J9VMInternals.java:185)
> >>>         at com/ibm/icu4jni/charset/CharsetProviderICU.charsetForName
> (Unkno
> >>>
> >> wn Source)
> >>
> >>>         at java/nio/charset/Charset.forNameInternal (Charset.java:450)
> >>>         at java/nio/charset/Charset.forName (Charset.java:513)
> >>>         at java/lang/String.defaultCharset ( String.java:688)
> >>>         at java/lang/String.<init> (String.java:252)
> >>>         at org/apache/harmony/luni/util/Util.toString (Util.java:74)
> >>>         at java/lang/System.getPropertyList (Native Method)
> >>>         at java/lang/System.ensureProperties (System.java:496)
> >>>         at java/lang/System.<clinit> (System.java:55)
> >>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
> >>>         at java/lang/J9VMInternals.initialize (J9VMInternals.java:185)
> >>>         at java/lang/ClassLoader.initializeClassLoaders (
> ClassLoader.java:6
> >>>
> >> 5)
> >>
> >>>         at java/lang/Thread.initialize (Thread.java:301)
> >>>         at java/lang/Thread.<init> (Thread.java:120)
> >>> java/lang/NullPointerException
> >>>         at java/lang/Throwable.printStackTrace ( Throwable.java:231)
> >>>         at com/vladium/emma/rt/RTExitHook.createClassLoaderClosure
> (RTExitH
> >>>
> >> ook.java:63)
> >>
> >>>         at com/vladium/emma/rt/RT.reset ( RT.java:92)
> >>>         at com/vladium/emma/rt/RT.<clinit> (RT.java:238)
> >>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
> >>>         at java/lang/J9VMInternals.initialize ( J9VMInternals.java
> :185)
> >>>         at java/nio/Buffer.$VRi (Buffer.java:4294967295)
> >>>         at java/nio/Buffer.<clinit> (Buffer.java:4294967295)
> >>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
> >>>         at java/lang/J9VMInternals.initialize (J9VMInternals.java:185)
> >>>         at java/lang/J9VMInternals.initialize (J9VMInternals.java:154)
> >>>         at java/lang/String.getBytes ( String.java:783)
> >>>         at org/apache/harmony/luni/util/Util.getBytes (Util.java:54)
> >>>         at java/lang/ClassLoader.loadLibraryWithPath (ClassLoader.java
> :998)
> >>>         at java/lang/ClassLoader.loadLibraryWithClassLoader (
> ClassLoader.ja
> >>>
> >> va:970)
> >>
> >>>         at java/lang/System.loadLibrary (System.java:706)
> >>>         at com/ibm/icu4jni/common/ICU4JNILoader.loadLibrary (Unknown
> Source
> >>>
> >> )
> >>
> >>>         at com/ibm/icu4jni/converters/NativeConverter.<clinit>
> (Unknown Sou
> >>>
> >> rce)
> >>
> >>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
> >>>         at java/lang/J9VMInternals.initialize (J9VMInternals.java:185)
> >>>         at com/ibm/icu4jni/charset/CharsetProviderICU.charsetForName
> (Unkno
> >>>
> >> wn Source)
> >>
> >>>         at java/nio/charset/Charset.forNameInternal (Charset.java:450)
> >>>         at java/nio/charset/Charset.forName (Charset.java:513)
> >>>         at java/lang/String.defaultCharset ( String.java:688)
> >>>         at java/lang/String.<init> (String.java:252)
> >>>         at org/apache/harmony/luni/util/Util.toString (Util.java:74)
> >>>         at java/lang/System.getPropertyList (Native Method)
> >>>         at java/lang/System.ensureProperties (System.java:496)
> >>>         at java/lang/System.<clinit> (System.java:55)
> >>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
> >>>         at java/lang/J9VMInternals.initialize (J9VMInternals.java:185)
> >>>         at java/lang/ClassLoader.initializeClassLoaders (
> ClassLoader.java:6
> >>>
> >> 5)
> >>
> >>>         at java/lang/Thread.initialize (Thread.java:301)
> >>>         at java/lang/Thread.<init> (Thread.java:120)
> >>> JVMJ9VM015W Initialization error for library jclclear_23(14):
> JVMJ9VM009E J
> >>>
> >> 9VMDllMain failed
> >>
> >>> HMYEXEL062E Internal VM error: Failed to create Java VM
> >>> FAILED.
> >>>
> >>>
> >>> On 1 June 2006 at 22:26, "Mikhail Loenko" < mloenko@gmail.com> wrote:
> >>>
> >>>
> >>>> 2006/6/1, Geir Magnusson Jr <ge...@pobox.com>:
> >>>>
> >>>>
> >>>>> Mikhail Loenko wrote:
> >>>>>
> >>>>>
> >>>>>> 2006/6/1, Geir Magnusson Jr <geir@pobox.com >:
> >>>>>>
> >>>>>>
> >>>>>>> I don't quite grok what you mean - could you post what you are
> >>>>>>> suggesting in code here on -dev?
> >>>>>>>
> >>>>>>>
> >>>>>> Index:
> modules/luni/src/test/java/tests/api/java/net/Inet6AddressTest.ja
> >>>>>>
> >> v
> >>
> >>>>>>
> >>>>>>
> >>>> a
> >>>>
> >>>>
> >>>>>> ===================================================================
> >>>>>> ---
> >>>>>> modules/luni/src/test/java/tests/api/java/net/Inet6AddressTest.java
> >>>>>> (revision
> >>>>>> 410480)
> >>>>>> +++
> >>>>>> modules/luni/src/test/java/tests/api/java/net/Inet6AddressTest.java
>
> >>>>>> (working
> >>>>>> copy)
> >>>>>> @@ -785,7 +785,7 @@
> >>>>>>                 "FFFF:FFFF:FFFF:FFFF:FFFF:FFFF: 255.255.255.255",
> >>>>>>                 "0:0:0:0:0:0:0:0", "0:0:0:0:0:0:0.0.0.0" };
> >>>>>>
> >>>>>> -        String invalidIPAddresses[] = { ":", "FFFF:FFFF", "1:1",
> >>>>>> "::1.2.3.444" };
> >>>>>> +        String invalidIPAddresses[] = { "FFFF:FFFF" };
> >>>>>>
> >>>>>>         for (int i = 0; i < validIPAddresses.length; i++) {
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>> Thx - why do you want to remove testing the others?  Because you
> believe
> >>>>> that it is in effect testing the resolver in the OS since each
> returns
> >>>>> the same answer,and thus it's testing the same thing over and over?
> >>>>>
> >>>>>
> >>>> I believe so, but I'd like to hear confirmation from code authors
> >>>>
> >>>> Thanks,
> >>>> Mikhail
> >>>>
> >>>>
> >>>
> >>> ---------------------------------------------------------------------
> >>> Terms of use : http://incubator.apache.org/harmony/mailing.html
> >>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> >>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >>>
> >>>
> >>>
> >>>
> >> --
> >> Paulex Yang
> >> China Software Development Lab
> >> IBM
> >>
> >>
> >>
> >> ---------------------------------------------------------------------
> >> Terms of use : http://incubator.apache.org/harmony/mailing.html
> >> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> >> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >>
> >
> >
> >
> > ---------------------------------------------------------------------
> > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >
> >
> >
>
>
> --
> Paulex Yang
> China Software Development Lab
> IBM
>
>
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>

Re: [classlib] [testing] Coverage (was Re: 37% of total test execution time is spent in a single test)

Posted by Paulex Yang <pa...@gmail.com>.
I've attach the scripts and excluded class lists to JIRA,  please refer 
to https://issues.apache.org/jira/browse/HARMONY-564.  Enjoy it:).

Mark Hindess wrote:
> On 2 June 2006 at 10:37, Paulex Yang <pa...@gmail.com> wrote:
>   
>> Mark,
>>
>> I'm glad that there is someone else has interest on emma, I've tried it 
>> before. AFAIK, emma works by instrumentation, but sometimes for classes 
>> in bootclasspath, the instrumentation cannot work, there are two cases:
>> 1. Some instrumented classes cannot be loaded by VM.
>> 2. Some classes cannot be instrumented
>>
>> I have tried to look more inside to find some way to work around but I 
>> haven't got enough time yet.
>>
>> Specifically for nio-channel module, I had a list for these two cases (I 
>> believe the data is a little outdated and should be reevaluated)
>> case 1.
>> BaseByteBuffer.class
>> Buffer.class
>> BufferFactory.class
>> ByteBuffer.class
>> CharArrayBuffer.class
>> CharBuffer.class
>> HeapByteBuffer.class
>> ReadWriteCharArrayBuffer.class
>> ReadWriteHeapByteBuffer.class
>> FileChannel.class
>> AbstractInterruptibleChannel.class
>> FileChannelImpl.class
>> WriteOnlyFileChannel.class
>> LockManager.class
>> LockManager$1.class
>> ReadOnlyFileChannel.class
>>
>> case 2:
>> ByteChannel.class
>> Channel.class
>> GatheringByteChannel.class
>> InterruptibleChannel.class
>> WritableByteChannel.class
>>
>> And I have got some ant script and more excluded list for emma, if 
>> anyone has interests, I can upload it to JIRA.
>>     
>
> Yes!
>
> -Mark.
>
>   
>> Mark Hindess wrote:
>>     
>>> Anyone tried using emma (emma.sf.net) to look at test coverage for our
>>> tests?  That might help ensure we don't trim them down too much?
>>>
>>> I suspect we will soon reach a point where there are significantly fewer
>>> obvious opportunities for small contributions - e.g. 1.5 generification,
>>> etc from looking at JAPI reports.  Having test coverage reports might
>>> help by highlighting areas for improvement.
>>>
>>> I had a quick try with:
>>>
>>>   ant -f make/build.xml
>>>   ant -f modules/nio/make/build.xml test
>>>   java -cp emma.jar emma instr -m overwrite -cp deploy/jdk/jre/lib/boot/nio
>>>       
>> .jar
>>     
>>> and then:
>>>
>>>   java -Xbootclasspath/p:emma.jar \
>>>        -cp build/tests:modules/nio/bin/test:junit.jar:. \
>>>        junit.textui.TestRunner \
>>>        org.apache.harmony.tests.java.nio.channels.SocketChannelTest
>>>
>>> (Running all the tests is a little tricky because test depends on build, 
>>> which depends on clean which wipes out the instr'd jar. ;-( )
>>>
>>> Sadly it dies horribly with the stack trace below.  I'll take another
>>> look when I have a little more time, but I thought I'd mention it in
>>> case anyone else had any thoughts on the subject.
>>>
>>> Regards,
>>>  
>>> Mark.
>>>
>>> Exception in thread "main" java/lang/ExceptionInInitializerError
>>>         at java/lang/J9VMInternals.initialize (J9VMInternals.java:203)
>>>         at java/nio/Buffer.$VRi (Buffer.java:4294967295)
>>>         at java/nio/Buffer.<clinit> (Buffer.java:4294967295)
>>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
>>>         at java/lang/J9VMInternals.initialize (J9VMInternals.java:185)
>>>         at java/lang/J9VMInternals.initialize (J9VMInternals.java:154)
>>>         at java/lang/String.getBytes (String.java:783)
>>>         at org/apache/harmony/luni/util/Util.getBytes (Util.java:54)
>>>         at java/lang/ClassLoader.loadLibraryWithPath (ClassLoader.java:998)
>>>         at java/lang/ClassLoader.loadLibraryWithClassLoader (ClassLoader.ja
>>>       
>> va:970)
>>     
>>>         at java/lang/System.loadLibrary (System.java:706)
>>>         at com/ibm/icu4jni/common/ICU4JNILoader.loadLibrary (Unknown Source
>>>       
>> )
>>     
>>>         at com/ibm/icu4jni/converters/NativeConverter.<clinit> (Unknown Sou
>>>       
>> rce)
>>     
>>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
>>>         at java/lang/J9VMInternals.initialize (J9VMInternals.java:185)
>>>         at com/ibm/icu4jni/charset/CharsetProviderICU.charsetForName (Unkno
>>>       
>> wn Source)
>>     
>>>         at java/nio/charset/Charset.forNameInternal (Charset.java:450)
>>>         at java/nio/charset/Charset.forName (Charset.java:513)
>>>         at java/lang/String.defaultCharset (String.java:688)
>>>         at java/lang/String.<init> (String.java:252)
>>>         at org/apache/harmony/luni/util/Util.toString (Util.java:74)
>>>         at java/lang/System.getPropertyList (Native Method)
>>>         at java/lang/System.ensureProperties (System.java:496)
>>>         at java/lang/System.<clinit> (System.java:55)
>>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
>>>         at java/lang/J9VMInternals.initialize (J9VMInternals.java:185)
>>>         at java/lang/ClassLoader.initializeClassLoaders (ClassLoader.java:6
>>>       
>> 5)
>>     
>>>         at java/lang/Thread.initialize (Thread.java:301)
>>>         at java/lang/Thread.<init> (Thread.java:120)
>>> java/lang/NullPointerException
>>>         at java/lang/Throwable.printStackTrace (Throwable.java:231)
>>>         at com/vladium/emma/rt/RTExitHook.createClassLoaderClosure (RTExitH
>>>       
>> ook.java:63)
>>     
>>>         at com/vladium/emma/rt/RT.reset (RT.java:92)
>>>         at com/vladium/emma/rt/RT.<clinit> (RT.java:238)
>>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
>>>         at java/lang/J9VMInternals.initialize (J9VMInternals.java:185)
>>>         at java/nio/Buffer.$VRi (Buffer.java:4294967295)
>>>         at java/nio/Buffer.<clinit> (Buffer.java:4294967295)
>>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
>>>         at java/lang/J9VMInternals.initialize (J9VMInternals.java:185)
>>>         at java/lang/J9VMInternals.initialize (J9VMInternals.java:154)
>>>         at java/lang/String.getBytes (String.java:783)
>>>         at org/apache/harmony/luni/util/Util.getBytes (Util.java:54)
>>>         at java/lang/ClassLoader.loadLibraryWithPath (ClassLoader.java:998)
>>>         at java/lang/ClassLoader.loadLibraryWithClassLoader (ClassLoader.ja
>>>       
>> va:970)
>>     
>>>         at java/lang/System.loadLibrary (System.java:706)
>>>         at com/ibm/icu4jni/common/ICU4JNILoader.loadLibrary (Unknown Source
>>>       
>> )
>>     
>>>         at com/ibm/icu4jni/converters/NativeConverter.<clinit> (Unknown Sou
>>>       
>> rce)
>>     
>>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
>>>         at java/lang/J9VMInternals.initialize (J9VMInternals.java:185)
>>>         at com/ibm/icu4jni/charset/CharsetProviderICU.charsetForName (Unkno
>>>       
>> wn Source)
>>     
>>>         at java/nio/charset/Charset.forNameInternal (Charset.java:450)
>>>         at java/nio/charset/Charset.forName (Charset.java:513)
>>>         at java/lang/String.defaultCharset (String.java:688)
>>>         at java/lang/String.<init> (String.java:252)
>>>         at org/apache/harmony/luni/util/Util.toString (Util.java:74)
>>>         at java/lang/System.getPropertyList (Native Method)
>>>         at java/lang/System.ensureProperties (System.java:496)
>>>         at java/lang/System.<clinit> (System.java:55)
>>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
>>>         at java/lang/J9VMInternals.initialize (J9VMInternals.java:185)
>>>         at java/lang/ClassLoader.initializeClassLoaders (ClassLoader.java:6
>>>       
>> 5)
>>     
>>>         at java/lang/Thread.initialize (Thread.java:301)
>>>         at java/lang/Thread.<init> (Thread.java:120)
>>> JVMJ9VM015W Initialization error for library jclclear_23(14): JVMJ9VM009E J
>>>       
>> 9VMDllMain failed
>>     
>>> HMYEXEL062E Internal VM error: Failed to create Java VM
>>> FAILED.
>>>
>>>
>>> On 1 June 2006 at 22:26, "Mikhail Loenko" <ml...@gmail.com> wrote:
>>>   
>>>       
>>>> 2006/6/1, Geir Magnusson Jr <ge...@pobox.com>:
>>>>     
>>>>         
>>>>> Mikhail Loenko wrote:
>>>>>       
>>>>>           
>>>>>> 2006/6/1, Geir Magnusson Jr <ge...@pobox.com>:
>>>>>>         
>>>>>>             
>>>>>>> I don't quite grok what you mean - could you post what you are
>>>>>>> suggesting in code here on -dev?
>>>>>>>           
>>>>>>>               
>>>>>> Index: modules/luni/src/test/java/tests/api/java/net/Inet6AddressTest.ja
>>>>>>             
>> v
>>     
>>>>>>         
>>>>>>             
>>>> a
>>>>     
>>>>         
>>>>>> ===================================================================
>>>>>> ---
>>>>>> modules/luni/src/test/java/tests/api/java/net/Inet6AddressTest.java
>>>>>> (revision
>>>>>> 410480)
>>>>>> +++
>>>>>> modules/luni/src/test/java/tests/api/java/net/Inet6AddressTest.java
>>>>>> (working
>>>>>> copy)
>>>>>> @@ -785,7 +785,7 @@
>>>>>>                 "FFFF:FFFF:FFFF:FFFF:FFFF:FFFF:255.255.255.255",
>>>>>>                 "0:0:0:0:0:0:0:0", "0:0:0:0:0:0:0.0.0.0" };
>>>>>>
>>>>>> -        String invalidIPAddresses[] = { ":", "FFFF:FFFF", "1:1",
>>>>>> "::1.2.3.444" };
>>>>>> +        String invalidIPAddresses[] = { "FFFF:FFFF" };
>>>>>>
>>>>>>         for (int i = 0; i < validIPAddresses.length; i++) {
>>>>>>
>>>>>>
>>>>>>         
>>>>>>             
>>>>> Thx - why do you want to remove testing the others?  Because you believe
>>>>> that it is in effect testing the resolver in the OS since each returns
>>>>> the same answer,and thus it's testing the same thing over and over?
>>>>>       
>>>>>           
>>>> I believe so, but I'd like to hear confirmation from code authors
>>>>
>>>> Thanks,
>>>> Mikhail
>>>>     
>>>>         
>>>
>>> ---------------------------------------------------------------------
>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>>
>>>
>>>   
>>>       
>> -- 
>> Paulex Yang
>> China Software Development Lab
>> IBM
>>
>>
>>
>> ---------------------------------------------------------------------
>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>     
>
>
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>
>   


-- 
Paulex Yang
China Software Development Lab
IBM



---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org


Re: [classlib] [testing] Coverage (was Re: 37% of total test execution time is spent in a single test)

Posted by Vladimir Ivanov <iv...@gmail.com>.
Paulex,



> And I have got some ant script and more excluded list for emma, if
> anyone has interests, I can upload it to JIRA.

Yes please, upload the list into JIRA. I'm going to experiment with the list
and source coverage of HY API too.



When I tried to experiment with HY API by unit tests code coverage
measurement via Emma I saw such errors.

The reason why runtime dies is that Emma instruments core classes and
inserts calls to its classes, which, in their turn, depend on the being
initialized core classes (for example, initialization of java.lang.Object is
working, it calls instrumented code, i.e. Emma class to store profiling
information, but the class in its turn depends on Object).

 Thanks,
  Vladimir


On 6/2/06, Mark Hindess <mark.hindess@googlemail.com > wrote:
>
>
> On 2 June 2006 at 10:37, Paulex Yang < paulex.yang@gmail.com > wrote:
> >
> > Mark,
> >
> > I'm glad that there is someone else has interest on emma, I've tried it
> > before. AFAIK, emma works by instrumentation, but sometimes for classes
> > in bootclasspath, the instrumentation cannot work, there are two cases:
> > 1. Some instrumented classes cannot be loaded by VM.
> > 2. Some classes cannot be instrumented
> >
> > I have tried to look more inside to find some way to work around but I
> > haven't got enough time yet.
> >
> > Specifically for nio-channel module, I had a list for these two cases (I
> > believe the data is a little outdated and should be reevaluated)
> > case 1.
> > BaseByteBuffer.class
> > Buffer.class
> > BufferFactory.class
> > ByteBuffer.class
> > CharArrayBuffer.class
> > CharBuffer.class
> > HeapByteBuffer.class
> > ReadWriteCharArrayBuffer.class
> > ReadWriteHeapByteBuffer.class
> > FileChannel.class
> > AbstractInterruptibleChannel.class
> > FileChannelImpl.class
> > WriteOnlyFileChannel.class
> > LockManager.class
> > LockManager$1.class
> > ReadOnlyFileChannel.class
> >
> > case 2:
> > ByteChannel.class
> > Channel.class
> > GatheringByteChannel.class
> > InterruptibleChannel.class
> > WritableByteChannel.class
> >
> > And I have got some ant script and more excluded list for emma, if
> > anyone has interests, I can upload it to JIRA.
>
> Yes!
>
> -Mark.
>
> > Mark Hindess wrote:
> > > Anyone tried using emma ( emma.sf.net) to look at test coverage for
> our
> > > tests?  That might help ensure we don't trim them down too much?
> > >
> > > I suspect we will soon reach a point where there are significantly
> fewer
> > > obvious opportunities for small contributions - e.g. 1.5generification,
> > > etc from looking at JAPI reports.  Having test coverage reports might
> > > help by highlighting areas for improvement.
> > >
> > > I had a quick try with:
> > >
> > >   ant -f make/build.xml
> > >   ant -f modules/nio/make/build.xml test
> > >   java -cp emma.jar emma instr -m overwrite -cp
> deploy/jdk/jre/lib/boot/nio
> > .jar
> > >
> > > and then:
> > >
> > >   java -Xbootclasspath/p:emma.jar \
> > >        -cp build/tests:modules/nio/bin/test: junit.jar:. \
> > >        junit.textui.TestRunner \
> > >        org.apache.harmony.tests.java.nio.channels.SocketChannelTest
> > >
> > > (Running all the tests is a little tricky because test depends on
> build,
> > > which depends on clean which wipes out the instr'd jar. ;-( )
> > >
> > > Sadly it dies horribly with the stack trace below.  I'll take another
> > > look when I have a little more time, but I thought I'd mention it in
> > > case anyone else had any thoughts on the subject.
> > >
> > > Regards,
> > >
> > > Mark.
> > >
> > > Exception in thread "main" java/lang/ExceptionInInitializerError
> > >         at java/lang/J9VMInternals.initialize (J9VMInternals.java:203)
> > >         at java/nio/Buffer.$VRi (Buffer.java :4294967295)
> > >         at java/nio/Buffer.<clinit> (Buffer.java :4294967295)
> > >         at java/lang/J9VMInternals.initializeImpl (Native Method)
> > >         at java/lang/J9VMInternals.initialize ( J9VMInternals.java
> :185)
> > >         at java/lang/J9VMInternals.initialize ( J9VMInternals.java
> :154)
> > >         at java/lang/String.getBytes (String.java:783)
> > >         at org/apache/harmony/luni/util/Util.getBytes ( Util.java:54)
> > >         at java/lang/ClassLoader.loadLibraryWithPath (
> ClassLoader.java:998)
> > >         at java/lang/ClassLoader.loadLibraryWithClassLoader (
> ClassLoader.ja
> > va:970)
> > >         at java/lang/System.loadLibrary ( System.java:706)
> > >         at com/ibm/icu4jni/common/ICU4JNILoader.loadLibrary (Unknown
> Source
> > )
> > >         at com/ibm/icu4jni/converters/NativeConverter.<clinit>
> (Unknown Sou
> > rce)
> > >         at java/lang/J9VMInternals.initializeImpl (Native Method)
> > >         at java/lang/J9VMInternals.initialize ( J9VMInternals.java
> :185)
> > >         at com/ibm/icu4jni/charset/CharsetProviderICU.charsetForName
> (Unkno
> > wn Source)
> > >         at java/nio/charset/Charset.forNameInternal (Charset.java:450)
> > >         at java/nio/charset/Charset.forName ( Charset.java:513)
> > >         at java/lang/String.defaultCharset ( String.java:688)
> > >         at java/lang/String.<init> (String.java:252)
> > >         at org/apache/harmony/luni/util/Util.toString ( Util.java:74)
> > >         at java/lang/System.getPropertyList (Native Method)
> > >         at java/lang/System.ensureProperties (System.java:496)
> > >         at java/lang/System.<clinit> ( System.java:55)
> > >         at java/lang/J9VMInternals.initializeImpl (Native Method)
> > >         at java/lang/J9VMInternals.initialize (J9VMInternals.java:185)
> > >         at java/lang/ClassLoader.initializeClassLoaders (
> ClassLoader.java:6
> > 5)
> > >         at java/lang/Thread.initialize ( Thread.java:301)
> > >         at java/lang/Thread.<init> (Thread.java:120)
> > > java/lang/NullPointerException
> > >         at java/lang/Throwable.printStackTrace ( Throwable.java:231)
> > >         at com/vladium/emma/rt/RTExitHook.createClassLoaderClosure
> (RTExitH
> > ook.java:63)
> > >         at com/vladium/emma/rt/RT.reset (RT.java:92)
> > >         at com/vladium/emma/rt/RT.<clinit> ( RT.java:238)
> > >         at java/lang/J9VMInternals.initializeImpl (Native Method)
> > >         at java/lang/J9VMInternals.initialize (J9VMInternals.java:185)
> > >         at java/nio/Buffer.$VRi ( Buffer.java:4294967295)
> > >         at java/nio/Buffer.<clinit> (Buffer.java :4294967295)
> > >         at java/lang/J9VMInternals.initializeImpl (Native Method)
> > >         at java/lang/J9VMInternals.initialize ( J9VMInternals.java
> :185)
> > >         at java/lang/J9VMInternals.initialize ( J9VMInternals.java
> :154)
> > >         at java/lang/String.getBytes (String.java:783)
> > >         at org/apache/harmony/luni/util/Util.getBytes ( Util.java:54)
> > >         at java/lang/ClassLoader.loadLibraryWithPath (
> ClassLoader.java:998)
> > >         at java/lang/ClassLoader.loadLibraryWithClassLoader (
> ClassLoader.ja
> > va:970)
> > >         at java/lang/System.loadLibrary ( System.java:706)
> > >         at com/ibm/icu4jni/common/ICU4JNILoader.loadLibrary (Unknown
> Source
> > )
> > >         at com/ibm/icu4jni/converters/NativeConverter.<clinit>
> (Unknown Sou
> > rce)
> > >         at java/lang/J9VMInternals.initializeImpl (Native Method)
> > >         at java/lang/J9VMInternals.initialize ( J9VMInternals.java
> :185)
> > >         at com/ibm/icu4jni/charset/CharsetProviderICU.charsetForName
> (Unkno
> > wn Source)
> > >         at java/nio/charset/Charset.forNameInternal (Charset.java:450)
> > >         at java/nio/charset/Charset.forName ( Charset.java:513)
> > >         at java/lang/String.defaultCharset ( String.java:688)
> > >         at java/lang/String.<init> (String.java:252)
> > >         at org/apache/harmony/luni/util/Util.toString ( Util.java:74)
> > >         at java/lang/System.getPropertyList (Native Method)
> > >         at java/lang/System.ensureProperties (System.java:496)
> > >         at java/lang/System.<clinit> (System.java :55)
> > >         at java/lang/J9VMInternals.initializeImpl (Native Method)
> > >         at java/lang/J9VMInternals.initialize (J9VMInternals.java:185)
> > >         at java/lang/ClassLoader.initializeClassLoaders (
> ClassLoader.java:6
> > 5)
> > >         at java/lang/Thread.initialize ( Thread.java:301)
> > >         at java/lang/Thread.<init> (Thread.java:120)
> > > JVMJ9VM015W Initialization error for library jclclear_23(14):
> JVMJ9VM009E J
> > 9VMDllMain failed
> > > HMYEXEL062E Internal VM error: Failed to create Java VM
> > > FAILED.
> > >
> > >
> > > On 1 June 2006 at 22:26, "Mikhail Loenko" < mloenko@gmail.com> wrote:
> > >
> > >> 2006/6/1, Geir Magnusson Jr <ge...@pobox.com>:
> > >>
> > >>> Mikhail Loenko wrote:
> > >>>
> > >>>> 2006/6/1, Geir Magnusson Jr < geir@pobox.com>:
> > >>>>
> > >>>>> I don't quite grok what you mean - could you post what you are
> > >>>>> suggesting in code here on -dev?
> > >>>>>
> > >>>> Index:
> modules/luni/src/test/java/tests/api/java/net/Inet6AddressTest.ja
> > v
> > >>>>
> > >> a
> > >>
> > >>>> ===================================================================
>
> > >>>> ---
> > >>>> modules/luni/src/test/java/tests/api/java/net/Inet6AddressTest.java
>
> > >>>> (revision
> > >>>> 410480)
> > >>>> +++
> > >>>> modules/luni/src/test/java/tests/api/java/net/Inet6AddressTest.java
> > >>>> (working
> > >>>> copy)
> > >>>> @@ -785,7 +785,7 @@
> > >>>>                 "FFFF:FFFF:FFFF:FFFF:FFFF:FFFF: 255.255.255.255",
> > >>>>                 "0:0:0:0:0:0:0:0", "0:0:0:0:0:0: 0.0.0.0" };
> > >>>>
> > >>>> -        String invalidIPAddresses[] = { ":", "FFFF:FFFF", "1:1",
> > >>>> ":: 1.2.3.444" };
> > >>>> +        String invalidIPAddresses[] = { "FFFF:FFFF" };
> > >>>>
> > >>>>         for (int i = 0; i < validIPAddresses.length; i++) {
> > >>>>
> > >>>>
> > >>>>
> > >>> Thx - why do you want to remove testing the others?  Because you
> believe
> > >>> that it is in effect testing the resolver in the OS since each
> returns
> > >>> the same answer,and thus it's testing the same thing over and over?
> > >>>
> > >> I believe so, but I'd like to hear confirmation from code authors
> > >>
> > >> Thanks,
> > >> Mikhail
> > >>
> > >
> > >
> > >
> > > ---------------------------------------------------------------------
> > > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> > >
> > >
> > >
> >
> >
> > --
> > Paulex Yang
> > China Software Development Lab
> > IBM
> >
> >
> >
> > ---------------------------------------------------------------------
> > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>