API coverage results for Harmony?

[Stuart Ballard <st...@gmail.com>]

(please keep me cc'd as I'm not subscribed to this list)

First off, I'd like to admit up front that I haven't been following the Harmony
project very closely and so I'm pretty ignorant about the state of your
progress, so please forgive any stupid misconceptions I may have.

I maintain a set of programs called japitools
(http://www.kaffe.org/~stuart/japi/ - not part of the kaffe project as such,
they just generously offered to host the homepage) which can be used to produce
(among other things) reports of how well a particular J2SE implementation covers
the API defined by Sun's reference implementation. In collaboration with the
Classpath developers I've been producing nightly results of Classpath's coverage
of each JDK release, as seen for example here (and others linked from the
homepage):

http://www.kaffe.org/~stuart/japi/htmlout/h-jdk14-classpath
http://www.kaffe.org/~stuart/japi/htmlout/h-jdk15-classpath-generics

I also send out emails to the classpath-testresults mailing list with reports on
the differences, if any, between each night's results, which is a handy way to
catch any API coverage regressions that might be introduced, as well as a nice
concrete way to see progress being made.

My understanding is that the Harmony project has made pretty significant
progress on its own class library implementation and I thought you might be
interested in producing similar nightly reports for Harmony. If you are
interested, there are a couple of ways we could go about it.

My machine doesn't have the resources to build a J2SE implementation itself so
for each project I report on I rely on their own nightly build infrastructure.
(I don't know if Harmony has such a thing set up already; alternatively a
developer could set up nightly build scripts on their own machine). Ideally (and
to match what other projects do) the script would also check out the latest
japitools source code, build it, and run the "Japize" tool to produce a
"harmony.japi.gz" file, and then make that file available for download
somewhere. I'd then configure my scripts to download that file and produce the
comparisons you see (I can also provide those scripts and the JDK japi files so
if you want to run them yourselves you can - there's nothing secret about them
;) ).

If running Japize yourselves is a problem for any reason, if you instead made
your equivalent(s) of rt.jar (and jce.jar and jsse.jar if you have them)
available for download, I could do the Japize as part of my script too.

Is there any interest in something like this?

If you want nightly difference reports sent to some mailing list, please also
let me know which list :)

Thanks,
Stuart.

Re: API coverage results for Harmony?

[Leo Simons <ma...@leosimons.com>]

On Sat, Feb 11, 2006 at 04:01:25PM +0000, Stuart Ballard wrote:
> Is there any interest in something like this?

Awesome!

The ASF has hardware for this kind of stuff, we just need to get
it properly set up to do this.

/me thinks about what it would take to fire up japitools from
within Apache Gump...

- LSD

Re: API coverage results for Harmony?

[Geir Magnusson Jr <ge...@pobox.com>]

Stuart Ballard wrote:
> (please keep me cc'd as I'm not subscribed to this list)

Wanna be subscribed?

> 
> First off, I'd like to admit up front that I haven't been following the Harmony
> project very closely and so I'm pretty ignorant about the state of your
> progress, so please forgive any stupid misconceptions I may have.
> 

No problem.

> I maintain a set of programs called japitools
> (http://www.kaffe.org/~stuart/japi/ - not part of the kaffe project as such,
> they just generously offered to host the homepage) which can be used to produce
> (among other things) reports of how well a particular J2SE implementation covers
> the API defined by Sun's reference implementation. In collaboration with the
> Classpath developers I've been producing nightly results of Classpath's coverage
> of each JDK release, as seen for example here (and others linked from the
> homepage):
> 
> http://www.kaffe.org/~stuart/japi/htmlout/h-jdk14-classpath
> http://www.kaffe.org/~stuart/japi/htmlout/h-jdk15-classpath-generics
> 
> I also send out emails to the classpath-testresults mailing list with reports on
> the differences, if any, between each night's results, which is a handy way to
> catch any API coverage regressions that might be introduced, as well as a nice
> concrete way to see progress being made.
> 
> My understanding is that the Harmony project has made pretty significant
> progress on its own class library implementation and I thought you might be
> interested in producing similar nightly reports for Harmony. If you are
> interested, there are a couple of ways we could go about it.

Yes, that would be interesting


> My machine doesn't have the resources to build a J2SE implementation itself so
> for each project I report on I rely on their own nightly build infrastructure.
> (I don't know if Harmony has such a thing set up already; alternatively a
> developer could set up nightly build scripts on their own machine). Ideally (and
> to match what other projects do) the script would also check out the latest
> japitools source code, build it, and run the "Japize" tool to produce a
> "harmony.japi.gz" file, and then make that file available for download
> somewhere. I'd then configure my scripts to download that file and produce the
> comparisons you see (I can also provide those scripts and the JDK japi files so
> if you want to run them yourselves you can - there's nothing secret about them
> ;) ).

We'll get our act together on the nightlies soon (next week?) so maybe 
we can work off of that?

> 
> If running Japize yourselves is a problem for any reason, if you instead made
> your equivalent(s) of rt.jar (and jce.jar and jsse.jar if you have them)
> available for download, I could do the Japize as part of my script too.
> 
> Is there any interest in something like this?

Definitely.

> 
> If you want nightly difference reports sent to some mailing list, please also
> let me know which list :)

harmony-dev@incubator.apache.org :)

> 
> Thanks,
> Stuart.
> 
> 

Re: API coverage results for Harmony?

[Tim Ellison <t....@gmail.com>]

Hi Stuart,

I've used japitools before and it is very cool.  It would be great to
set up and run the tools on the Harmony codebase.

How close are you to getting support for new 5.0 language features?

Regards,
Tim

Stuart Ballard wrote:
> (please keep me cc'd as I'm not subscribed to this list)
> 
> First off, I'd like to admit up front that I haven't been following the Harmony
> project very closely and so I'm pretty ignorant about the state of your
> progress, so please forgive any stupid misconceptions I may have.
> 
> I maintain a set of programs called japitools
> (http://www.kaffe.org/~stuart/japi/ - not part of the kaffe project as such,
> they just generously offered to host the homepage) which can be used to produce
> (among other things) reports of how well a particular J2SE implementation covers
> the API defined by Sun's reference implementation. In collaboration with the
> Classpath developers I've been producing nightly results of Classpath's coverage
> of each JDK release, as seen for example here (and others linked from the
> homepage):
> 
> http://www.kaffe.org/~stuart/japi/htmlout/h-jdk14-classpath
> http://www.kaffe.org/~stuart/japi/htmlout/h-jdk15-classpath-generics
> 
> I also send out emails to the classpath-testresults mailing list with reports on
> the differences, if any, between each night's results, which is a handy way to
> catch any API coverage regressions that might be introduced, as well as a nice
> concrete way to see progress being made.
> 
> My understanding is that the Harmony project has made pretty significant
> progress on its own class library implementation and I thought you might be
> interested in producing similar nightly reports for Harmony. If you are
> interested, there are a couple of ways we could go about it.
> 
> My machine doesn't have the resources to build a J2SE implementation itself so
> for each project I report on I rely on their own nightly build infrastructure.
> (I don't know if Harmony has such a thing set up already; alternatively a
> developer could set up nightly build scripts on their own machine). Ideally (and
> to match what other projects do) the script would also check out the latest
> japitools source code, build it, and run the "Japize" tool to produce a
> "harmony.japi.gz" file, and then make that file available for download
> somewhere. I'd then configure my scripts to download that file and produce the
> comparisons you see (I can also provide those scripts and the JDK japi files so
> if you want to run them yourselves you can - there's nothing secret about them
> ;) ).
> 
> If running Japize yourselves is a problem for any reason, if you instead made
> your equivalent(s) of rt.jar (and jce.jar and jsse.jar if you have them)
> available for download, I could do the Japize as part of my script too.
> 
> Is there any interest in something like this?
> 
> If you want nightly difference reports sent to some mailing list, please also
> let me know which list :)
> 
> Thanks,
> Stuart.
> 
> 

-- 

Tim Ellison (t.p.ellison@gmail.com)
IBM Java technology centre, UK.