Weekly code coverage reports?

[Kathey Marsden <km...@sbcglobal.net>]

  The weekly code coverage reports seem to have stopped in April and the 
detail from that time is missing.

http://dbtg.foundry.sun.com/derby/test/coverage/

Is there someone that can take a look?  It would be great to get this 
transitioned to Hudson if possible.

Thanks

Kathey

Re: Weekly code coverage reports?

[Kristian Waagan <kr...@oracle.com>]

  On 10.09.10 18:10, Kathey Marsden wrote:
>  The weekly code coverage reports seem to have stopped in April and 
> the detail from that time is missing.
>
> http://dbtg.foundry.sun.com/derby/test/coverage/
>
> Is there someone that can take a look?  It would be great to get this 
> transitioned to Hudson if possible.

Hi Kathey,

I had a go at moving to Clover in Hudson a while back. It wasn't as easy 
as I had hoped for...
After having eliminated a number of initial obstacles (changes to 
classpath in ant and maybe even in a few tests, JVM parameters, don't 
remember the rest), here is the executive summary:
  o it takes a long time
  o it takes a lot of resources - some of the Derby tests are resource 
intensive, the cost of the coverage tool comes on top
  o the coverage database is/will be large (was it around 5 GB?)
  o the report generation (separate step) failed

It seemed to me that Clover wasn't capable of handling the amount of 
information generated by the Derby test suites. In the end I think I 
tried running it with a 64-bit JVM allocating a heap more than 10 GB 
large. We may be able to tweak Clover to reduce the amount of 
information gathered and produced. We could turn off history and 
individual test traces, but the main problem is still that almost every 
test sequence will touch a lot of core code and generate large amounts 
of trace data.

Besides from the technical problems, I'm not sure if hogging shared ASF 
resources for extended periods of time is okay.
Further, I see it as unrealistic to generate frequent reports with the 
current resources (mostly due to the fact that they are shared). We 
could create several jobs, running only a suite or two each day. The 
results would all be written to the same db, and the report generation 
job could run once a week, every other week or once a month. This 
approach will add to the admin overhead in Hudson though...

Note that Emma is also supported in Hudson, and there is also some 
support for Cobertura and Sonar (not sure if this works with ant).
I think the work I did on Clover can be revived and tested with the new 
version of the tool, and we already know Emma used to work.


-- 
Kristian

>
> Thanks
>
> Kathey
>
>

Re: Weekly code coverage reports?

[Knut Anders Hatlen <kn...@oracle.com>]

Kathey Marsden <km...@sbcglobal.net> writes:

>  The weekly code coverage reports seem to have stopped in April and
> the detail from that time is missing.
>
> http://dbtg.foundry.sun.com/derby/test/coverage/
>
> Is there someone that can take a look?  It would be great to get this
> transitioned to Hudson if possible.

Around that time the emma test runs started to fail on almost each
run. I think the problem was that emma's report generator often didn't
understand the coverage data that had been written by the tests. I
haven't looked into it yet, but my suspicion would be that this is a
concurrency issue where multiple threads/processes write to the same
file and make it corrupt.

Has anyone else been able to run the tests with emma lately?

-- 
Knut Anders