[all] What's in a distribution?

[Dennis Lundberg <de...@apache.org>]

Hi

I've been looking at creating distributions for commons logging using 
Maven 2. So I did some reading on ASF policy regarding distributions and 
poked around in different commons components, to see if I could find a 
least common denominator.

Unfortunately I haven't. So I've got a couple of questions for you to 
think about:

1. What should a source distribution include?

2. What should a binary distribution include?


Reading the documentation for the assembly-plugin [1] for Maven 2 I 
found an interesting feature that will come in the next version. It will 
create an assembly (distribution) of "the entire source project, 
including the Maven POM and other files outside of your source directory 
structure, but excluding all SCM files and the target directory" and 
archives for it (tar.gz, tar.bz2 and zip). Now this sound to me like the 
ideal source distro. A complete checkout. Then it's up to the user to 
build, read or do whatever he or she feels like with it.

Who downloads a binary distribution, why and what for? Someone who 
doesn't use Maven to build their products. Because the don't want to, or 
don't know how to, build the commons component their selves. Because 
someone did all the building and assembling for them, i.e. they are lazy 
as in good lazy. So that's a broad audience we've got there.

What do they need? Well in the case of commons they want the jar of 
course. What else? Some documentation on how to use it, list of 
dependencies and such.

The ASF also has needs when it comes to licensing. We need to put a 
LICENSE and NOTICE file in there.

What are your thoughts on this subject?


[1] 
http://maven.apache.org/plugins/maven-assembly-plugin/descriptor-refs.html

-- 
Dennis Lundberg

---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org

Re: [all] What's in a distribution?

[Rahul Akolkar <ra...@gmail.com>]

On 1/3/07, Henri Yandell <fl...@gmail.com> wrote:
<snip/>
>
> Part of all that would be to take the common information out of the
> component sites and into a Jakarta page for that component. Much like
> the download section currently. The tricky part is one of l&f, it's
> very odd to jump out of a site and into another site, however to have
> a Jakarta with many components we have to step away from the common
> l&f attempt we currently have (which I think just causes us pain) and
> be more lenient.
>
<snap/>

I've snipped most bits that I more or less agree with. On the LnF
front, I don't think we're overly concerned about the CSS a component
uses per se, if at all a component wants to use a different one
(though I think its nice that most Commons components have sites that
look similar). Its more about standardization with the objective of
making lives of users and developers easier (for users, fact that
Javadocs, licenses are in a predictable place as you mentioned -- and
for developers, the fact that the site is generated using similar
means etc. which allows, for example, me to jump in and feel at home
with the [digester] site if I want to volunteer RM'ing it). Generally
boiling down to my opinion that a Commons/Jakarta component needs to
have communal responsibility.


>
> Time to do some work - I'll stop rambling on :)
>
<snip/>

Thanks for taking time to elaborate :)

-Rahul


> Hen
>

---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org

Re: [all] What's in a distribution?

[Henri Yandell <fl...@gmail.com>]

On 1/3/07, Rahul Akolkar <ra...@gmail.com> wrote:
> On 1/2/07, Henri Yandell <fl...@gmail.com> wrote:
> <snip/>
> >
> > I dislike the website being put in the distributions. It's a cheap way
> > to think you're documenting your project; but having the documentation
> > in there is good. I think the solution to this part is to make our
> > websites leaner (by moving things into Jakarta's site and nightly
> > builds) so that what is left is a better fit for going in the
> > distribution.
> >
> <snap/>
>
> IIUC, the cheap bit you are refering to is treating ${reports} as the
> entire documentation

that and the overlap between the website and the documentation. A
website is chiefly (I think) concerned with getting someone to a) use
the product and b) join in the community. The documentation should be
much about explaining how to use the product. So the front page for a
website and the frontpage for documentation are completely different
pages.

Reports are also useless on both the website and the documentation -
they're only really of value to the developer. The few that are of
value to the user shouldn't be under reports (imo) - license + javadoc
- neither are reports. A license report would have value to the user
(something that showed the license of all dependencies and linked to
information about them), but Maven doesn't have that afaik.

Reports should be a part of the CI system.

>, though am not sure how much value there is in
> breaking them off as you propose. In any case, components like [SCXML]
> have a multi-page cross-referenced user guide that is best viewed via
> a web browser, IMO, and that is unlikely to be distributed in any
> other form in the near future.

+1 for HTML as a format for documentation, definitely not arguing for
PDF or OpenOffice etc.

I've been grumbling about the above for a while - generally to myself
but every now and then to the list.

On the other side of things, I get grumbly about the Jakarta site and
thoughts on how to organize a Jakarta with lots of components in it.
One of my thoughts is that it should largely be database driven
(meaning an xml file like downloads.xml but for much more
information). Things like svn url, javadoc page, download page, web
page, nightly builds location. Even better is if most of these things
are name convention based: ie) 'scxml' gets used in each url.

The doap files for proejcts.apache.org should then be generatable
(it's a layer above/below the doap files so we can't use those
directly).

Part of all that would be to take the common information out of the
component sites and into a Jakarta page for that component. Much like
the download section currently. The tricky part is one of l&f, it's
very odd to jump out of a site and into another site, however to have
a Jakarta with many components we have to step away from the common
l&f attempt we currently have (which I think just causes us pain) and
be more lenient.

One way around l&f pain is to generate an xml file for each page and
let the l&f be governed by the component. While I think that'll work
for pan-ASF things like people.apache.org and projects.apache.org, I
don't think it'll work well within Jakarta.

Time to do some work - I'll stop rambling on :)

Hen

---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org

Re: [all] What's in a distribution?

[Rahul Akolkar <ra...@gmail.com>]

On 1/2/07, Henri Yandell <fl...@gmail.com> wrote:
<snip/>
>
> I dislike the website being put in the distributions. It's a cheap way
> to think you're documenting your project; but having the documentation
> in there is good. I think the solution to this part is to make our
> websites leaner (by moving things into Jakarta's site and nightly
> builds) so that what is left is a better fit for going in the
> distribution.
>
<snap/>

IIUC, the cheap bit you are refering to is treating ${reports} as the
entire documentation, though am not sure how much value there is in
breaking them off as you propose. In any case, components like [SCXML]
have a multi-page cross-referenced user guide that is best viewed via
a web browser, IMO, and that is unlikely to be distributed in any
other form in the near future.

-Rahul


>
> Hen
>

---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org

Re: [all] What's in a distribution?

[Craig McClanahan <cr...@apache.org>]

On 1/2/07, Henri Yandell <fl...@gmail.com> wrote:
>
> I dislike the website being put in the distributions. It's a cheap way
> to think you're documenting your project; but having the documentation
> in there is good. I think the solution to this part is to make our
> websites leaner (by moving things into Jakarta's site and nightly
> builds) so that what is left is a better fit for going in the
> distribution.


Well, you at least need the *sources* for the website, in your source
distro, in order to reproduce those bits :-).

For the Commons Projects that I originated, I tried to put all the geeky
how-to technical documentation into the overview.html or package.html  of
the package (see BeanUtils and Digester), thus making them visible as part
of the Javadocs -- in theory the audience interested in this, and the
details of the individual methods, are likely to be the same person.  This
doesn't scale perfectly to large scale frameworks, where the developer needs
to understand how lots of classes (in lots of packages) work together -- but
it's still a good practice IMHO.

Craig

Re: [all] What's in a distribution?

[Henri Yandell <fl...@gmail.com>]

On 1/2/07, Dennis Lundberg <de...@apache.org> wrote:
> Hi
>
> I've been looking at creating distributions for commons logging using
> Maven 2. So I did some reading on ASF policy regarding distributions and
> poked around in different commons components, to see if I could find a
> least common denominator.
>
> Unfortunately I haven't. So I've got a couple of questions for you to
> think about:
>
> 1. What should a source distribution include?
>
> 2. What should a binary distribution include?

...

> What are your thoughts on this subject?

I rarely download a zip file - my use cases are:

* Want to use the project  -> download jar from ibiblio. Unless I'm
using Maven as the build tool, in which case I'd figure out its
details and put them in my pom.

* Want to build/investigate a historic project. My first port of call
is the svn tags, but some are just plain confusing so I use the source
files.

* Want to look at javadoc - I look at the online javadoc.

Other use case I've had in the past:

* Want to publish javadoc internally. Download the binary dist just to
get the javadoc.

And one I know exists at some big companies:

* Grab the source zip so a trusted version can be built from it.
Though I don't know if they use an svn tag when they can. I would :)

+1 to Craig's opine that the source distro should be able to create
the binary distro. I'm quite happy to merge the source and binary
distributions into one, having two seems quite pointless. I thought
Maven repository 'distributions' had gone away, so I need to go learn
about them again.

I dislike the website being put in the distributions. It's a cheap way
to think you're documenting your project; but having the documentation
in there is good. I think the solution to this part is to make our
websites leaner (by moving things into Jakarta's site and nightly
builds) so that what is left is a better fit for going in the
distribution.

Defining the distribution/assembly in the super pom would be great. I
think it would be valuable to put the non-optional dependencies in
there too, but I can see that that might hit problems (VFS's LGPL
dependency for example).

Hen

---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org

Re: [all] What's in a distribution?

[lu...@free.fr]

Stephen Colebourne wrote:


> I include the binary jar file in the source distribution as I want to
> ensure that the maximum number of people possible get the genuine binary
> jar as created by us and no-one else (eg. minimises JDK1.3 vs JDK1.6 issues)

[snip]

> I include the source zip in the binary distribution as I think most
> people want to attach the source to their jar files, at least in commons.

I don't agree with this point of view. If people need these two parts (pristine
sources and trusted binary), then they could grab both archives. But many other
people only want either one or the other, depending on the use cases Hen
depicted.

We should take care about people having low bandwidth. There are many places in
the world were 56K modems are still the only means to connect to the internet
and were communication costs are high. Putting everything (source, binary,
website ...) in an all-purpose distribution is not user friendly for these
people.

Luc

---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org

Re: [all] What's in a distribution?

[Stephen Colebourne <sc...@btopenworld.com>]

Dennis Lundberg wrote:
> 1. What should a source distribution include?

Two things:
- commons-foo-1.0.jar - the binary jar
- The entire svn tree for the project

I include the binary jar file in the source distribution as I want to 
ensure that the maximum number of people possible get the genuine binary 
jar as created by us and no-one else (eg. minimises JDK1.3 vs JDK1.6 issues)


> 2. What should a binary distribution include?

Three things:
- commons-foo-1.0.jar - the binary jar
- commons-foo-1.0-src-ide.zip - the source code
- The javadoc


I include the source zip in the binary distribution as I think most 
people want to attach the source to their jar files, at least in commons.

Stephen

---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org

Re: [all] What's in a distribution?

[Craig McClanahan <cr...@apache.org>]

I will opine a bit based on what I have personally done historically on
Commons projects, plus some thoughts on what we're currently doing in
Shale[1] that might prove to be of interest (and/or a source of ideas on how
to do it).

On 1/2/07, Dennis Lundberg <de...@apache.org> wrote:
>
> Hi
>
> I've been looking at creating distributions for commons logging using
> Maven 2. So I did some reading on ASF policy regarding distributions and
> poked around in different commons components, to see if I could find a
> least common denominator.
>
> Unfortunately I haven't. So I've got a couple of questions for you to
> think about:
>
> 1. What should a source distribution include?


IMHO, a "source distribution" should always include all the files necessary
to build the corresponding binary distribution with the project's favorite
build tool.  For an Ant based project, that meant having a build.xml file in
the top level, plus a way to define and/or get the dependencies, plus all
the sources in a hierarchy that the build.xml expects.  For a Maven2 based
project, it's a similar deal ... a pom.xml at the top, and all the sources
in the right structure, so that the user could unpack the distro, run "mvn
install" at the top level, and get the same results as what they'd get with
the binary distro.

2. What should a binary distribution include?


>From a standalone "distro" perspective, I would expect to see the binary
jars of the project itself, binary jars of dependencies (assuming license
compatibility), and the javadocs.  However, my current preferred practice
(and what we are now doing with Shale) is to combine the source and binary
distributions into a single "distro" that includes all of the above.

In a Maven2 based world, you also need to consider the act of publishing
into the Maven2 repository to be a "distribution" as well.  In that case,
you're generally including just POMs and JARs, but you have to deal with
checksums and signatures on these artifacts as well.

Reading the documentation for the assembly-plugin [1] for Maven 2 I
> found an interesting feature that will come in the next version. It will
> create an assembly (distribution) of "the entire source project,
> including the Maven POM and other files outside of your source directory
> structure, but excluding all SCM files and the target directory" and
> archives for it (tar.gz, tar.bz2 and zip). Now this sound to me like the
> ideal source distro. A complete checkout. Then it's up to the user to
> build, read or do whatever he or she feels like with it.


Shale uses a custom assembly for the "framework" (which includes a master
module and a bunch of submodules), and a custom assembly for each sample
webapp, that conform to the "combined" philosophy described above.  For a
class library like most from Commons, it's pretty simple to set up something
similar.

Who downloads a binary distribution, why and what for? Someone who
> doesn't use Maven to build their products. Because the don't want to, or
> don't know how to, build the commons component their selves. Because
> someone did all the building and assembling for them, i.e. they are lazy
> as in good lazy. So that's a broad audience we've got there.


Even on my Maven projects, I also download binary distros when I need the
Javadocs and they weren't published into the repository.  Same for the
source distro if I need to debug.

What do they need? Well in the case of commons they want the jar of
> course. What else? Some documentation on how to use it, list of
> dependencies and such.
>
> The ASF also has needs when it comes to licensing. We need to put a
> LICENSE and NOTICE file in there.


"In there" actually means a couple of things for a Commons project -- in the
distribution artifact itself (.zip or .tar.gz or whatever), *and* inside the
library JAR's META-INF directory.

What are your thoughts on this subject?
>
>
> [1]
> http://maven.apache.org/plugins/maven-assembly-plugin/descriptor-refs.html
>
> --
> Dennis Lundberg


Craig

[1] http://shale.apache.org/