M4-alpha

[Rafael Schloming <ra...@redhat.com>]

I've created an M4-alpha release from svn revision 714267. You can find 
it here:

http://people.apache.org/~rhs/qpid-incubating-M4-alpha/

Please have at it.

I'm going to be a bit scarce next week, but I should be able to make a 
new release if there are any egregious problems. Otherwise, I'd like to 
make a beta on the 25th, and hopefully have a final RC the first week of 
December.

--Rafael

Re: [CONF] Apache Qpid: Project Status (comment added)

[lahiru gunathilake <la...@apache.org>]

Perfect !!!

Thank you
Lahiru

On Sun, Nov 16, 2008 at 11:54 AM, Carl Trieloff <cc...@redhat.com>wrote:

>
> "Carl, my name is not here, mind if I ask you to put my name in to the
> list. I think this list is not about PPMC's of the
> project."
>
> Lahiru,
>
> The resolution only has the PPMC, you are listed on the people page
>
>  You are on the people page and on the project status:
> http://cwiki.apache.org/confluence/display/qpid/People
> http://incubator.apache.org/projects/qpid.html
>
> I will be scarce next week, but will do project admin again the last week
> of Nov, now that our graduation vote has
> passed we need to vote to add people to the PMC.
> regards,
> Carl.
>

Re: [CONF] Apache Qpid: Project Status (comment added)

[Carl Trieloff <cc...@redhat.com>]

"Carl, my name is not here, mind if I ask you to put my name in to the 
list. I think this list is not about PPMC's of the
project."

Lahiru,

The resolution only has the PPMC, you are listed on the people page

  You are on the people page and on the project status:
http://cwiki.apache.org/confluence/display/qpid/People
http://incubator.apache.org/projects/qpid.html

I will be scarce next week, but will do project admin again the last 
week of Nov, now that our graduation vote has
passed we need to vote to add people to the PMC.
regards,
Carl.

Re: M4-alpha

[Marnie McCormack <ma...@googlemail.com>]

Hi,

 <snip>
>


>
> As an end user I can see the advantage of downloading a single binary
>> package however I don't then want to have to pick through over 60 jar
>> files to find the require dependencies for my client code. I would
>> much prefer to have individual packages so that we can then patch and
>> release changes as required. If there is a critical bug in the client
>> code we shouldn't have to do a full Qpid release.
>>
>
> I don't think as an Apache project we would ever want to do separate
> releases of particular sub-component, but I can see how it would be useful
> for those commercial entities providing support for Qpid to be able to do
> that sort of thing. So for me, this becomes a nice to have sort of thing,
> but not really a priority for an Apache release.
>>
>>
>>>
>>
>> Most of my work is about user interaction .... I think having distinct
distros, regardless of how we do releases and what we include, is really
useful.

For example, most (almost all) users do not want to use their client on the
same env as their broker. Thus one big distro forces them to install a heap
of stuff they don't want to use. It also makes it far harder (in most
controlled environments) to manage the versions of Qpid in use i.e. the
broker & client package is all in one. Much easier to work here where the
client & broker are separate. I know users who have been bitten by this,
and it applies equally to Apache release distros.

We may want to do releases of individual components (we have done in the
past). There's no reason why we could not at any rate.

I use scripts to generate what I'd like to use, so am happy to go with the
majority on Apache releases. Just my tuppence from the coalface so to speak.

I am pretty ambivalent about how we do it, both approaches have real merit
but i think the output is the key item for me.

TTfn,
Marnie

Re: M4-alpha

[Rafael Schloming <ra...@redhat.com>]

Martin Ritchie wrote:
> 2008/11/20 Rafael Schloming <ra...@redhat.com>:
>> Martin Ritchie wrote:
>>> I'd agree here. a zip of the build directory is not a binary release.
>> One of the original design principles of the build system was that the build
>> directory should be as close as is possible to the binary release. The idea
>> being that the closer our development environment is to a released
>> environment, the fewer surprises there will be when we release, and the less
>> disparity there will be between developer docs and user docs.
>>
>> This principle may have been lost a bit in some of the subsequent
>> modifications since the inception of the build system, but I still think
>> it's a good one. So while currently a zip of the build directory might make
>> a sub-optimal binary release, IMHO it *should* be the case that it makes a
>> perfectly reasonable binary release. Certainly there should be a very good
>> reason for each piece of extra scripty magic that you need to run to convert
>> the build dir to a release dir, since each one of these is a potential
>> pitfall for our users. (I am yet to be convinced that we currently need
>> *any* scripty magic.)
> 
> Having the build system generate a directory structure that is our aim
> to release is fine however that would need to be documented as a goal
> so that new people to the project know that. The QMan war generation
> is a prime example as it uses the build directory as a scratch space
> to generate the release war. Something that the review process could
> have picked up if it was more widely known that the build directory
> was our release artefact.

I think had I been more prescient I would have documented this carefully 
from the beginning, but to be honest I didn't realize at first exactly 
why I did it that way. It just felt *simple* and *right*. It was only 
later that I realized there was some merit beyond my own prejudices.

> I don't think we ever quite got to the goal of having the build
> directory being the ideal binary distribution. Looking at the contents
> of build after an 'ant clean build' results in a large number of files
> that we do not want to ship. The build target sets up the build
> directory for running the tests as it generates the results hierarchy
> and and the data directory. These are not recent changes as far as I
> am aware.

IIRC there were initially a small number of scratch directories under 
build that were excluded when generating the tarball, e.g. the classes 
directories. It might be better to gather these under one scratch dir to 
simplify the exception.

> If our plan is to have 'ant clean build' produce a build directory
> that we can zip up for a release then we are going to have to work on
> that and document it so we are don't put files in the the build
> directory that we would not want to be distributed.

Agreed.

> I'm still firmly of the belief that it would be normal for each module
> to produce a release artefact and it would only be common as the
> exception that doesn't, unless we standardise a low level api.
> Management may also be an exception as it generates multiple artefacts
> one for each of its sub modules.

I think what constitutes a release artifact should actually be defined 
external to the build system proper, and the build scripts shouldn't be 
filled with special case logic to cope with the idiosyncrasies of each 
sub-module. This may put some constraints on what the release tarball 
looks like, e.g. any distinct components should be easy to pick out of 
the full release structure, but this is IMHO a fairly mild and 
reasonable constraint.

> As an end user I can see the advantage of downloading a single binary
> package however I don't then want to have to pick through over 60 jar
> files to find the require dependencies for my client code. I would
> much prefer to have individual packages so that we can then patch and
> release changes as required. If there is a critical bug in the client
> code we shouldn't have to do a full Qpid release.

I don't think as an Apache project we would ever want to do separate 
releases of particular sub-component, but I can see how it would be 
useful for those commercial entities providing support for Qpid to be 
able to do that sort of thing. So for me, this becomes a nice to have 
sort of thing, but not really a priority for an Apache release.

>>> If we are interested in doing separate broker and client packages then
>>> the 'ant release-bin' currently generates a useful package.
>> Along the same lines as above, IMHO generating separate sub-packages for
>> distinct pieces of the project should be as simple as identifying the
>> appropriate subset of files underneath build.
> 
> The build system already knows the majority of the subset why should
> we have to go and identify the files again to be included in a module
> build?

I'm not suggesting we should duplicate anything. I'm suggesting the 
structure of a full release should make it easy to pick out how the 
different artifacts relate to each other.

> Perhaps I'm just not seeing what you are. If you have a proposal of
> how this would look then perhaps you can sway my thoughts.

I wouldn't worry about how it works so much. The first step is to figure 
out what a release should look like, i.e. what downloads there are, how 
each one is structured, and how they relate to each other. Once that is 
settled we can figure out how to make it work. I think going straight 
into hacking the build system will only result in a release structure 
that is too intimately tied into the details of the build system.

--Rafael

Re: M4-alpha

[Martin Ritchie <ri...@apache.org>]

2008/11/20 Rafael Schloming <ra...@redhat.com>:
> Martin Ritchie wrote:
>>
>> I'd agree here. a zip of the build directory is not a binary release.
>
> One of the original design principles of the build system was that the build
> directory should be as close as is possible to the binary release. The idea
> being that the closer our development environment is to a released
> environment, the fewer surprises there will be when we release, and the less
> disparity there will be between developer docs and user docs.
>
> This principle may have been lost a bit in some of the subsequent
> modifications since the inception of the build system, but I still think
> it's a good one. So while currently a zip of the build directory might make
> a sub-optimal binary release, IMHO it *should* be the case that it makes a
> perfectly reasonable binary release. Certainly there should be a very good
> reason for each piece of extra scripty magic that you need to run to convert
> the build dir to a release dir, since each one of these is a potential
> pitfall for our users. (I am yet to be convinced that we currently need
> *any* scripty magic.)

Having the build system generate a directory structure that is our aim
to release is fine however that would need to be documented as a goal
so that new people to the project know that. The QMan war generation
is a prime example as it uses the build directory as a scratch space
to generate the release war. Something that the review process could
have picked up if it was more widely known that the build directory
was our release artefact.

I don't think we ever quite got to the goal of having the build
directory being the ideal binary distribution. Looking at the contents
of build after an 'ant clean build' results in a large number of files
that we do not want to ship. The build target sets up the build
directory for running the tests as it generates the results hierarchy
and and the data directory. These are not recent changes as far as I
am aware.

If our plan is to have 'ant clean build' produce a build directory
that we can zip up for a release then we are going to have to work on
that and document it so we are don't put files in the the build
directory that we would not want to be distributed.

I'm still firmly of the belief that it would be normal for each module
to produce a release artefact and it would only be common as the
exception that doesn't, unless we standardise a low level api.
Management may also be an exception as it generates multiple artefacts
one for each of its sub modules.

As an end user I can see the advantage of downloading a single binary
package however I don't then want to have to pick through over 60 jar
files to find the require dependencies for my client code. I would
much prefer to have individual packages so that we can then patch and
release changes as required. If there is a critical bug in the client
code we shouldn't have to do a full Qpid release.


>> If we are interested in doing separate broker and client packages then
>> the 'ant release-bin' currently generates a useful package.
>
> Along the same lines as above, IMHO generating separate sub-packages for
> distinct pieces of the project should be as simple as identifying the
> appropriate subset of files underneath build.

The build system already knows the majority of the subset why should
we have to go and identify the files again to be included in a module
build?

Perhaps I'm just not seeing what you are. If you have a proposal of
how this would look then perhaps you can sway my thoughts.

Regards

Martin

> --Rafael
>



-- 
Martin Ritchie

Re: M4-alpha

[Rafael Schloming <ra...@redhat.com>]

Marnie McCormack wrote:
> Hi,
> 
> 
> On Thu, Nov 20, 2008 at 4:32 PM, Rafael Schloming <ra...@redhat.com>wrote:
> 
>> Martin Ritchie wrote:
>>
>>> I'd agree here. a zip of the build directory is not a binary release.
>>>
>> One of the original design principles of the build system was that the
>> build directory should be as close as is possible to the binary release. The
>> idea being that the closer our development environment is to a released
>> environment, the fewer surprises there will be when we release, and the less
>> disparity there will be between developer docs and user docs.
>>
> 
> I agree with this, though I don't think it's factual as things stand now
> i.e. there are suprises. Developer docs vs user docs not such an issue here,
> as we only only have minimal docs about the build content (which don't hold
> true now everything is in together).

I'm also thinking about developer/user interactions, e.g. I don't want 
to accidentally tell a user to go run such-and-such script or look in 
so-and-so file that doesn't actually exist because it's somewhere 
different in the build environment.

>> This principle may have been lost a bit in some of the subsequent
>> modifications since the inception of the build system, but I still think
>> it's a good one. So while currently a zip of the build directory might make
>> a sub-optimal binary release, IMHO it *should* be the case that it makes a
>> perfectly reasonable binary release. Certainly there should be a very good
>> reason for each piece of extra scripty magic that you need to run to convert
>> the build dir to a release dir, since each one of these is a potential
>> pitfall for our users. (I am yet to be convinced that we currently need
>> *any* scripty magic.)
> 
> 
> I'm all for the idea, but it's not where we are just now imho. Perhaps a
> does of pragmatism here ....

I agree we have strayed a bit from it. My point is simply that we should 
*not* "fix" things by adding more scripty magic to mangle the build 
directory into a shapely release tarball, but rather fix things by 
making the build directory more suitable for direct release.

>> If we are interested in doing separate broker and client packages then
>>> the 'ant release-bin' currently generates a useful package.
>>>
>> Along the same lines as above, IMHO generating separate sub-packages for
>> distinct pieces of the project should be as simple as identifying the
>> appropriate subset of files underneath build.
>>
> 
> So, what do we need to do to have a binary which is componentised ? Maybe we
> can capture/share the work out here ?

I think we need to define, document, and agree on a release structure 
that makes sense both as one directory structure underneath build, (i.e. 
a full release with all components included) and also makes sense as 
separately downloadable tarballs, e.g. there needs to be a clear and 
simple relationship between the separate component tarballs and the full 
release.

Once we've done this the changes to the build system and release scripts 
should be relatively straightforward. I'll volunteer for that part. We 
just need to be careful that future build system changes and new 
subprojects don't mess things up.

--Rafael

Re: M4-alpha

[Marnie McCormack <ma...@googlemail.com>]

Hi,


On Thu, Nov 20, 2008 at 4:32 PM, Rafael Schloming <ra...@redhat.com>wrote:

> Martin Ritchie wrote:
>
>> I'd agree here. a zip of the build directory is not a binary release.
>>
>
> One of the original design principles of the build system was that the
> build directory should be as close as is possible to the binary release. The
> idea being that the closer our development environment is to a released
> environment, the fewer surprises there will be when we release, and the less
> disparity there will be between developer docs and user docs.
>

I agree with this, though I don't think it's factual as things stand now
i.e. there are suprises. Developer docs vs user docs not such an issue here,
as we only only have minimal docs about the build content (which don't hold
true now everything is in together).

>
> This principle may have been lost a bit in some of the subsequent
> modifications since the inception of the build system, but I still think
> it's a good one. So while currently a zip of the build directory might make
> a sub-optimal binary release, IMHO it *should* be the case that it makes a
> perfectly reasonable binary release. Certainly there should be a very good
> reason for each piece of extra scripty magic that you need to run to convert
> the build dir to a release dir, since each one of these is a potential
> pitfall for our users. (I am yet to be convinced that we currently need
> *any* scripty magic.)


I'm all for the idea, but it's not where we are just now imho. Perhaps a
does of pragmatism here ....

>
>
> If we are interested in doing separate broker and client packages then
>> the 'ant release-bin' currently generates a useful package.
>>
>
> Along the same lines as above, IMHO generating separate sub-packages for
> distinct pieces of the project should be as simple as identifying the
> appropriate subset of files underneath build.
>

So, what do we need to do to have a binary which is componentised ? Maybe we
can capture/share the work out here ?

Regards,
Marnie

>
> --Rafael
>

Re: M4-alpha

[Rafael Schloming <ra...@redhat.com>]

Martin Ritchie wrote:
> I'd agree here. a zip of the build directory is not a binary release.

One of the original design principles of the build system was that the 
build directory should be as close as is possible to the binary release. 
The idea being that the closer our development environment is to a 
released environment, the fewer surprises there will be when we release, 
and the less disparity there will be between developer docs and user docs.

This principle may have been lost a bit in some of the subsequent 
modifications since the inception of the build system, but I still think 
it's a good one. So while currently a zip of the build directory might 
make a sub-optimal binary release, IMHO it *should* be the case that it 
makes a perfectly reasonable binary release. Certainly there should be a 
very good reason for each piece of extra scripty magic that you need to 
run to convert the build dir to a release dir, since each one of these 
is a potential pitfall for our users. (I am yet to be convinced that we 
currently need *any* scripty magic.)

> If we are interested in doing separate broker and client packages then
> the 'ant release-bin' currently generates a useful package.

Along the same lines as above, IMHO generating separate sub-packages for 
distinct pieces of the project should be as simple as identifying the 
appropriate subset of files underneath build.

--Rafael

Re: M4-alpha

[Martin Ritchie <ri...@apache.org>]

I'd agree here. a zip of the build directory is not a binary release.

If we are interested in doing separate broker and client packages then
the 'ant release-bin' currently generates a useful package.

I think that it would be good if the other packages management cli,
qman etc all had a release-bin target that would correctly package up
their files for a binary release.

Regards

Martin

2008/11/20 Marnie McCormack <ma...@googlemail.com>:
> Hi,
>
> Thanks for this Rafi. Took a look at the Java tar.
>
> There are a couple of things that catch the eye:
>
> - My it's huge :-(
> - I don't think we need to/should ship the management/servlet/lib dir with
> content (all 16 jars included in the war)
> - The bin dir seems to be a little too random now i.e. the readme talks
> about examples and the files in here are obv the merge of all bin dirs which
> means it has lost context. Is there something we could do to clean this up a
> little ? Could we sensibly ship the cli & management tools separately
> perhaps ?
>
>
> My starting precept has been that the binary should be useable in an 'out of
> the box' way but it's become a little monstrous. Happy to document for the
> release, but ideas about how we can thin it out a little would be great ?
>
> Bfn,
> Thanks,
> Marnie
> On Sat, Nov 15, 2008 at 3:11 PM, Rafael Schloming <ra...@redhat.com>wrote:
>
>> I've created an M4-alpha release from svn revision 714267. You can find it
>> here:
>>
>> http://people.apache.org/~rhs/qpid-incubating-M4-alpha/
>>
>> Please have at it.
>>
>> I'm going to be a bit scarce next week, but I should be able to make a new
>> release if there are any egregious problems. Otherwise, I'd like to make a
>> beta on the 25th, and hopefully have a final RC the first week of December.
>>
>> --Rafael
>>
>



-- 
Martin Ritchie

Re: M4-alpha

[Marnie McCormack <ma...@googlemail.com>]

Hi,

Thanks for this Rafi. Took a look at the Java tar.

There are a couple of things that catch the eye:

- My it's huge :-(
- I don't think we need to/should ship the management/servlet/lib dir with
content (all 16 jars included in the war)
- The bin dir seems to be a little too random now i.e. the readme talks
about examples and the files in here are obv the merge of all bin dirs which
means it has lost context. Is there something we could do to clean this up a
little ? Could we sensibly ship the cli & management tools separately
perhaps ?


My starting precept has been that the binary should be useable in an 'out of
the box' way but it's become a little monstrous. Happy to document for the
release, but ideas about how we can thin it out a little would be great ?

Bfn,
Thanks,
Marnie
On Sat, Nov 15, 2008 at 3:11 PM, Rafael Schloming <ra...@redhat.com>wrote:

> I've created an M4-alpha release from svn revision 714267. You can find it
> here:
>
> http://people.apache.org/~rhs/qpid-incubating-M4-alpha/
>
> Please have at it.
>
> I'm going to be a bit scarce next week, but I should be able to make a new
> release if there are any egregious problems. Otherwise, I'd like to make a
> beta on the 25th, and hopefully have a final RC the first week of December.
>
> --Rafael
>