Maven debug and source artifacts for Derby

[Kristian Waagan <kr...@oracle.com>]

Hi,

A few JIRAs have been logged where users are requesting that the Derby 
community provide debug artifacts for Maven (links at the end):
   o DERBY-5668: Derby does not publish source artifacts to Maven Central
   o DERBY-5543: include debug info in derby builds uploaded to maven
   o DERBY-3910: debug artifacts should be available in maven repositories

Regarding the source artifacts, I think we should just do it! Does 
anyone see a reason why we don't want to do it?

The debug artifacts requires a little more consideration. There are two 
requests at play here:
   a) include line numbers
   b) include the extra checks guarded by the SanityManager blocks
      (what's called a sane build)

I think many of the users would want to use artifacts (a). In fact I'd 
say they should probably be the default. I imagine artifacts (b) to be 
used during development and when you debug a problem. What we are 
currently proving is more along the lines of (c):
   c) reduced disk-footprint JARs for small devices

(a) and (c) are suitable for production, (b) isn't because of the 
performance degradation caused by the extra sanity checks.


A recent build of trunk shows the following increases in size when 
including line numbers [1]:
   o derby.jar: 2.6 MB -> 3.5 MB
   o derbyclient.jar: 523 KB -> 686 KB
   o derbynet.jar: 237 KB -> 302 KB

Not compiling away the extra checks only adds a little more. The growth 
is most significant for derby.jar, where it ends up at 3.7 MB.

They way I see it, adding line numbers has the following potential benefits:
   o make it easier for users to debug/investigate a problem where they 
get a stack trace
   o make it easier for the Derby community to start debugging a problem 
when they have a stack trace

The only downside I see is the increase in size. This probably isn't a 
big deal for most users, but I expect there will be some users that want 
to keep things as small as possible. As a note to this, there has been 
some speculation (?) that the increased size may slow down class 
loading. Again I assume this is a concern to only a subset of our users.

Together with source artifacts, I believe adding debug information will 
make it easier for people using Maven as their build tool to get 
involved with Derby. I think providing these artifacts can benefit both 
these users and the Derby community.
I can't see how providing these artifacts would hurt.

What do others think?


Links to the issues mentioned:
https://issues.apache.org/jira/browse/DERBY-5668
https://issues.apache.org/jira/browse/DERBY-5543
https://issues.apache.org/jira/browse/DERBY-3910


Regards,
-- 
Kristian

[1] We're only specifying debug=true, which I assume has the same effect 
as debug=on. This again I'm assuming causes -g to be passed to the 
compiler, since we're not specifying debuglevel, and -g means all debug 
information (-g:lines,vars,source).

Re: Maven debug and source artifacts for Derby

[Kristian Waagan <kr...@oracle.com>]

On 03.05.12 21:52, Rick Hillegas wrote:
> Thanks for starting this conversation, Kristian. Here's my $0.02:
>
> I see a number of issues here:
>
> 1) An impedance mismatch between the Derby and Maven concepts of release
> artifacts.
>
> 2) The bewildering number of Derby release artifacts today.
>
> 3) A request for different Derby release artifacts.
>
> 4) A request for a source Maven artifact.
>
> Just to be clear, I think we should try to drive our release process
> toward producing fewer artifacts, not more.

Hi Rick,

That last statement is certainly valid, but I prefer that we produce the 
artifacts our users need rather than focus too much on keeping the 
number down to a minimum. If the resulting process is too hard on the 
community, and more specifically the release manager, we have to 
simplify it with better scripting/tooling to make it bearable.
Now, actually finding out which artifacts are needed isn't necessarily easy.

Let me explain why I brought up the topic of Maven artifacts:
There are a lot of Maven users out there, and the central Maven 
repository contains a huge amount of software. Further, there are 
several other systems/tools that can integrate with Maven, for instance 
Apache Buildr, Apache Ivy, Groovy Grape, Grails, and Scala SBT.
By having good Maven artifacts for Derby, we make it easier for people 
to use Derby. If it's too hard, and I don't think it takes much to reach 
this state, people may very well simply switch to another database 
system - for new projects with little current investment in Derby that's 
a matter of changing a few dependency definitions.

Apache is all about open-source, communities and involvement.
In my opinion, not distributing a source bundle in Maven makes it harder 
for Maven users to get involved. Not including line numbers in the class 
files makes it harder for someone new to understand what's going on and 
to debug the code ([1]). If their application fails because of Derby, 
seeing the Derby source code on line X, where the NPE happened, may 
convince them that this is indeed a Derby bug and not their own "silly 
mistake". The result may be that we get a JIRA report and can fix a bug.
This isn't about what these users are able to do or not, it's about how 
much effort it takes for them to get involved. Too much effort means no 
involvement ([2]). Derby is only one project out of a large number of 
open-source projects that are fighting for involvement from contributers.
I'm not saying we'll see a flurry of new contributions if we fix these 
issues, but at least we remove some obstacles for those who have a 
little bit of curiosity in them.


Based on your reply I think it would be best to discuss items (2) and 
(3) first. Given that we are getting close to a new release, I'm 
suggesting that we let this discussion live for a while and that we 
don't change anything before 10.9 goes out.

Even if the above means postponing any actions on these topics, people 
are very welcome to present their opinions on these topics.


Thanks,
-- 
Kristian, with his community hat on :)


[1] I know we have a source zip and debug bundles on the web site, and I 
know people can access the Subversion repository if they want to. But 
it's more tedious than clicking "Download Sources" in my IDE and 
double-clicking on the class I'm interested in.

[2] I've seen this myself, where I have given up on reporting a 
[potential] bug in a piece of software because I couldn't find the 
information I needed, or because filling out the bug report would take 
me hours...

< snip - lots of points for discussion >

Re: Maven debug and source artifacts for Derby

[Rick Hillegas <ri...@oracle.com>]

Thanks for starting this conversation, Kristian. Here's my $0.02:

I see a number of issues here:

1) An impedance mismatch between the Derby and Maven concepts of release 
artifacts.

2) The bewildering number of Derby release artifacts today.

3) A request for different Derby release artifacts.

4) A request for a source Maven artifact.

Just to be clear, I think we should try to drive our release process 
toward producing fewer artifacts, not more.


------- Impedance Mismatch Between Derby and Maven ----------

For Derby, a release artifact is a zip (or tar) ball containing a 
complete system of jar files which cohere with one another. For Maven, a 
release artifact is a single jar file which can be mixed and matched 
with other jar files at different rev levels. The two world views 
diverge at a very high level:

a) For Derby, jar files are children of version numbers

b) For Maven, version numbers are children of jar files

I don't know how we go about bridging this gap. The Maven model allows 
for mixing Derby jars from different versions, something which we don't 
generally allow. Maybe there is a way in Maven to declare that the 
version numbers of derbynet.jar and derby.jar must agree.


------- Bewildering Number of Derby Release Artifacts ----------

IMHO we already produce too many Derby artifacts. Do we really need bin 
and lib artifacts? Do we really need zips and tarballs? I'd be happy 
with zip-only artifacts representing the following:

i) Source (I believe this is the only artifact required by Apache)

ii) Docs, templates, and demos

iii) Bin (executable code, including jars and shell scripts)

iv) Debug-bin

That's 4 artifacts rather than the 8 we produce today (remember, we're 
deprecating the Eclipse bits).


-------  Different Derby Artifacts ----------

There's a request to produce an artifact which is half-way between our 
bin and debug-bin artifact, viz., executable code which includes line 
numbers but no assertions. How much extra value does this add?

I think this would provide extra information for heisenbugs which don't 
have repros, but in my experience we don't make much headway on those 
bugs, with or without line numbers.

For bugs which have repros, we (the Derby developers who do almost all 
of the bug fixing around here) see the line numbers as soon as we run 
the repro, because we're all using sane builds in our sandboxes.

So who is this new artifact for? Are there other developers out there 
who would be contributing fixes back to the community if we provided an 
artifact with line numbers? These would be people who would take the 
time to script, debug, and fix an issue. To do this, they would need to 
download the source distro or set up a subversion client on one of our 
branches. But they wouldn't be willing to download a debug distro.


-------  Source Maven Artifacts ----------

I don't have any objection to providing a source Maven artifact for the 
purposes described by DERBY-5668. Does the source artifact need to be in 
some special format to be usable by IDEs?

Thanks,
-Rick

Re: Maven debug and source artifacts for Derby

["Dag H. Wanvik" <da...@oracle.com>]

Kristian Waagan <kr...@oracle.com> writes:

> Together with source artifacts, I believe adding debug information
> will make it easier for people using Maven as their build tool to get
> involved with Derby. I think providing these artifacts can benefit
> both these users and the Derby community.
> I can't see how providing these artifacts would hurt.

I think this question is larger than the Maven artifacts: do we want to
provide bundles that have line number debug information, i.e. your a),
in addition to b) and b) or possibly as a replacement for b) or c)?

I agree that line numbers would often by the preferred default, and the
fully stripped jars would mostly needed for constrained environments.

But we have many artifacts already, so it could increase confusion. What
do others think?

Dag