You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@qpid.apache.org by Robbie Gemmell <ro...@gmail.com> on 2009/12/09 17:55:19 UTC
Release artifacts
Hi all,
Now that we have entered another release phase I think it would be a good time to discuss the release artifacts. I would like to see 0.6 ship with some updates to the artifacts we produce to make things more useful and obvious for our users.
The "Java broker, client, and tools" multi-component package appears to be just a copy of the etc/ lib/ and bin/ directories from the qpid/java/build/ dir after performing a build and this makes it quite messy. A lot of test configuration files are left in the etc/ dir and there are loads of jars in the lib/ dir, which make it very hard to know which is being used where or allow splitting them up easily. You also end up with jars that just aren't intended to be used from there (eg the eclipse-plugin used in the JMX Management Console releases). We have single component packages for the Java broker, client, and management tools so if we want a multi-component package including any of these I think we should simply bundle those up instead of just tarring the build folder and mashing everything together.
Another result of the above is that QMan is shipped in the Java multi-component bundle despite the fact it is only useful if you have a C++ broker. I realise it is written in Java, but those packages don't seem to me like they should necessarily be language specific. If anything I would say that QMan should surely ship in the multi-component bundle along with the items it can actually be used with (ie the C++ broker and client bundle), or just be made available standalone (which it already is). Users aren't necessarily going to expect they need to download both multi-component packages to make use of everything contained in one of them, and as mentioned above, splitting QMan alone out of the java multi-component bundle would be a bit hideous and non obvious anyway.
I would also like to see us tagging the source-only release artifacts with 'src', ie the C++ and the "full" release artifact that is just a copy of the entire repo. It isn't exactly helpful for a user to download the 50MB file then find out what they are looking for isn't actually in there.
Robbie
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
Re: Release artifacts
Posted by Rajith Attapattu <ra...@gmail.com>.
On Wed, Dec 9, 2009 at 12:30 PM, Robert Godfrey <ro...@gmail.com> wrote:
> 2009/12/9 Rafael Schloming <ra...@redhat.com>
>
>> Robbie Gemmell wrote:
>>
>>> Hi all,
>>>
>>> Now that we have entered another release phase I think it would be a good
>>> time to discuss the release artifacts. I would like to see 0.6 ship with
>>> some updates to the artifacts we produce to make things more useful and
>>> obvious for our users.
>>>
>>
>> I think it's a bit late for 0.6, if I understand the timeline correctly.
>>
>>
>> The "Java broker, client, and tools" multi-component package appears to be
>>> just a copy of the etc/ lib/ and bin/ directories from the qpid/java/build/
>>> dir after performing a build and this makes it quite messy. A lot of test
>>> configuration files are left in the etc/ dir and there are loads of jars in
>>> the lib/ dir, which make it very hard to know which is being used where or
>>> allow splitting them up easily. You also end up with jars that just aren't
>>> intended to be used from there (eg the eclipse-plugin used in the JMX
>>> Management Console releases). We have single component packages for the Java
>>> broker, client, and management tools so if we want a multi-component package
>>> including any of these I think we should simply bundle those up instead of
>>> just tarring the build folder and mashing everything together.
>>>
>>
>> I sort of disagree with this. There is a "philosophy" behind using the
>> "just a copy" approach. The benefit is that we can eliminate a whole class
>> of bugs resulting from complex transformations necessary to turn the build
>> artifacts into release artifacts. Not to mention it is easier for us to
>> reproducer user issues if dev and user environments are as similar as
>> possible.
>>
>> That said I'm fine cleaning up the release artifacts and what not, I'm just
>> -1 on doing it by introducing complex transformations between build and
>> release artifacts. If we want the release artifacts to look different we
>> should make the build artifacts look different.
>>
>> As I've said before I'm happy to help with this, but we sort of need to
>> figure out what we want things to look like first, and IMHO such an effort
>> should also include making release artifacts more consistent across the
>> whole project.
>>
>>
> I think this is the key point - we need to work out how our build and
> release artefacts look; since what we have today is clearly unsatisfactory.
>
> If there are quick, non-risky, changes that we could put in place for 0.6
> that are universally acknowledged as a step in the right direction, I
> wouldn't necessarily be against them - however I don't think we want to
> embark on anything that will negatively impact a 0.6 release date at this
> stage. I am hugely +1 in doing this work before 0.7 (it's clearly a
> necessity for us if we ever want to get to a 1.0 release)
Totally agree here.
I think for the 0.7 release we definitely need to consider sorting out
our documentation and release artifacts a top priority.
I also agree that we need to have a consistent story across all our
components than a per language approach.
>>
>> Another result of the above is that QMan is shipped in the Java
>>> multi-component bundle despite the fact it is only useful if you have a C++
>>> broker. I realise it is written in Java, but those packages don't seem to me
>>> like they should necessarily be language specific. If anything I would say
>>> that QMan should surely ship in the multi-component bundle along with the
>>> items it can actually be used with (ie the C++ broker and client bundle), or
>>> just be made available standalone (which it already is). Users aren't
>>> necessarily going to expect they need to download both multi-component
>>> packages to make use of everything contained in one of them, and as
>>> mentioned above, splitting QMan alone out of the java multi-component bundle
>>> would be a bit hideous and non obvious anyway.
>>>
>>> I would also like to see us tagging the source-only release artifacts with
>>> 'src', ie the C++ and the "full" release artifact that is just a copy of the
>>> entire repo. It isn't exactly helpful for a user to download the 50MB file
>>> then find out what they are looking for isn't actually in there.
>>>
>>
>> I also think it's a shame that our artifacts look like they come from
>> entirely different projects. Even little things like readme capitalization,
>> general directory layout, etc is completely different from artifact to
>> artifact, and this is even after the release script hacks to rearchive stuff
>> to make things a bit more consistent.
>>
>>
> Agree again completely. We have a lot of work to do - which we should do as
> a matter of priority - in sorting out our artefacts to present a more
> consistent appearance across the project.
>
> -- Rob
>
>
>> --Rafael
>>
>>
>>
>> ---------------------------------------------------------------------
>> Apache Qpid - AMQP Messaging Implementation
>> Project: http://qpid.apache.org
>> Use/Interact: mailto:dev-subscribe@qpid.apache.org
>>
>>
>
--
Regards,
Rajith Attapattu
Red Hat
http://rajith.2rlabs.com/
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
Re: Release artifacts
Posted by Robert Godfrey <ro...@gmail.com>.
2009/12/9 Rafael Schloming <ra...@redhat.com>
> Robbie Gemmell wrote:
>
>> Hi all,
>>
>> Now that we have entered another release phase I think it would be a good
>> time to discuss the release artifacts. I would like to see 0.6 ship with
>> some updates to the artifacts we produce to make things more useful and
>> obvious for our users.
>>
>
> I think it's a bit late for 0.6, if I understand the timeline correctly.
>
>
> The "Java broker, client, and tools" multi-component package appears to be
>> just a copy of the etc/ lib/ and bin/ directories from the qpid/java/build/
>> dir after performing a build and this makes it quite messy. A lot of test
>> configuration files are left in the etc/ dir and there are loads of jars in
>> the lib/ dir, which make it very hard to know which is being used where or
>> allow splitting them up easily. You also end up with jars that just aren't
>> intended to be used from there (eg the eclipse-plugin used in the JMX
>> Management Console releases). We have single component packages for the Java
>> broker, client, and management tools so if we want a multi-component package
>> including any of these I think we should simply bundle those up instead of
>> just tarring the build folder and mashing everything together.
>>
>
> I sort of disagree with this. There is a "philosophy" behind using the
> "just a copy" approach. The benefit is that we can eliminate a whole class
> of bugs resulting from complex transformations necessary to turn the build
> artifacts into release artifacts. Not to mention it is easier for us to
> reproducer user issues if dev and user environments are as similar as
> possible.
>
> That said I'm fine cleaning up the release artifacts and what not, I'm just
> -1 on doing it by introducing complex transformations between build and
> release artifacts. If we want the release artifacts to look different we
> should make the build artifacts look different.
>
> As I've said before I'm happy to help with this, but we sort of need to
> figure out what we want things to look like first, and IMHO such an effort
> should also include making release artifacts more consistent across the
> whole project.
>
>
I think this is the key point - we need to work out how our build and
release artefacts look; since what we have today is clearly unsatisfactory.
If there are quick, non-risky, changes that we could put in place for 0.6
that are universally acknowledged as a step in the right direction, I
wouldn't necessarily be against them - however I don't think we want to
embark on anything that will negatively impact a 0.6 release date at this
stage. I am hugely +1 in doing this work before 0.7 (it's clearly a
necessity for us if we ever want to get to a 1.0 release)
>
> Another result of the above is that QMan is shipped in the Java
>> multi-component bundle despite the fact it is only useful if you have a C++
>> broker. I realise it is written in Java, but those packages don't seem to me
>> like they should necessarily be language specific. If anything I would say
>> that QMan should surely ship in the multi-component bundle along with the
>> items it can actually be used with (ie the C++ broker and client bundle), or
>> just be made available standalone (which it already is). Users aren't
>> necessarily going to expect they need to download both multi-component
>> packages to make use of everything contained in one of them, and as
>> mentioned above, splitting QMan alone out of the java multi-component bundle
>> would be a bit hideous and non obvious anyway.
>>
>> I would also like to see us tagging the source-only release artifacts with
>> 'src', ie the C++ and the "full" release artifact that is just a copy of the
>> entire repo. It isn't exactly helpful for a user to download the 50MB file
>> then find out what they are looking for isn't actually in there.
>>
>
> I also think it's a shame that our artifacts look like they come from
> entirely different projects. Even little things like readme capitalization,
> general directory layout, etc is completely different from artifact to
> artifact, and this is even after the release script hacks to rearchive stuff
> to make things a bit more consistent.
>
>
Agree again completely. We have a lot of work to do - which we should do as
a matter of priority - in sorting out our artefacts to present a more
consistent appearance across the project.
-- Rob
> --Rafael
>
>
>
> ---------------------------------------------------------------------
> Apache Qpid - AMQP Messaging Implementation
> Project: http://qpid.apache.org
> Use/Interact: mailto:dev-subscribe@qpid.apache.org
>
>
RE: Release artifacts
Posted by Robbie Gemmell <ro...@gmail.com>.
We all seems to agree that it should change, so why dont we start work towards agreeing on what it should change to so everyone ends up happy :)
I like Martin's idea of having the build system output artifacts into build/broker, build/client etc, each composed of the jars needed to run that component rather than everything ending up in build/ as present. That way we can test exactly what we release, avoid class path hell as much as possible, reduce the number of potentially useless jars going into releases, and when it comes to release time all we do is tar up whichever module sub-directories we want in a particular component bundle...eg broker, client, broker + client, management/x, management/y, all of the above.
Robbie
> -----Original Message-----
> From: Rafael Schloming [mailto:rafaels@redhat.com]
> Sent: 10 December 2009 21:21
> To: dev@qpid.apache.org
> Subject: Re: Release artifacts
>
> Martin Ritchie wrote:
> > 2009/12/9 Rafael Schloming <ra...@redhat.com>:
> >> Robbie Gemmell wrote:
> >>> I think you have misunderstood what I meant by that entirely, as by
> "the
> >>> build system in general to always generate the release style
> artifacts" I
> >>> meant exactly what you are suggesting, that the build system should
> be
> >>> modified to generate something we can simply take a (partial)
> tarball of and
> >>> end up with what we release...ie release style. The reason there is
> anything
> >>> else in place right now is that they arent, IMO.
> >> Sorry, I see how you meant that now. My mistake. Clearly I've been
> too eager
> >> to rant recently. ;)
> >>
> >> --Rafael
> >
> > As the one who modified our current Java build system to produce
> > 'release' artefacts I can say it is very easy to generate a packaged
> > that contains the dependecies for a module. Add 1 line to the
> module's
> > build.xml:
> >
> > <target name="release-bin" depends="release-bin-tasks"/>
>
> I wasn't suggesting that this is difficult to do, merely that it uses a
> completely different mechanism than the one for generating build
> artifacts.
>
> > That said I undestand the desire to have a directory that we then
> just
> > tar up for release. However, the thought that tar-ing a directory is
> > in some way better than using the properties of the build system to
> > make a tar is some what erroneous. There is no reason that the two
> > cannot be the same.
>
> I think I may be missing something here. If they're the same, why not
> do
> it the simple way (i.e. tar up the directory).
>
> In any case, I'm not really saying that we shouldn't use build system
> properties. What I'm saying is that I don't want the process for
> generating release artifacts to be independent from the one used for
> generating build artifacts.
>
> I have to say I have NEVER used the files in the
> > build directory and the fact that all the jars end up in the lib dir
> > and so get put in to qpid-all.jar makes for a very dangerous
> situation
> > IMO. Testing with this setup can result in runtime dependencies not
> > being reflected by the build system.
>
> You may not use the files in the build directory directly, but you do
> use them indirectly because that's the environment that the tests are
> run in, e.g. when the external java broker is launched, it's done by
> executing build/bin/qpid-server.
>
> > I would be all for having the release artefacts being built in their
> > own build directories build/broker, build/client, build/qmf, etc. I'm
> > totally open to suggestions on how we can do this I simply used the
> > build system for the task last time as it has knowledge of all the
> > library dependencies and shipping jars that we don't need as part of
> > our packages is quite unhelpful, forcing these on to the class path
> > via qpid-all.jar as we do in the current build directory is just a
> > receipe for disaster.
> >
> > Now I feel I may have rant a bit, that wasn't really my intention. I
> > mearly wanted to point out that if we want to do component releases
> > form the Java directory for 0.6 it would take less that 2minutes to
> > sort out. So I think it is a totally doable.
>
> I completely agree qpid-all.jar is a bad idea. I believe it exists as a
> substitute for the giant client.jar that we used to generate by munging
> all the class files together into a single jar, and it replicates some
> of the same problems that system had. I'm not sure why this still
> exists, or indeed why it ever existed. I would be +1 on killing
> qpid-all.jar and using properly defined classpaths instead. (After 0.6
> goes out.)
>
> I'm also fine on adding a component release for qman for 0.6. What I
> don't think is a good idea at this stage (i.e. for 0.6) is to remove
> the
> full release or recompose it from the component releases since the full
> release is the environment that we actually run our tests in.
>
> --Rafael
>
>
> ---------------------------------------------------------------------
> Apache Qpid - AMQP Messaging Implementation
> Project: http://qpid.apache.org
> Use/Interact: mailto:dev-subscribe@qpid.apache.org
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
Re: Release artifacts
Posted by Rafael Schloming <ra...@redhat.com>.
Martin Ritchie wrote:
> 2009/12/9 Rafael Schloming <ra...@redhat.com>:
>> Robbie Gemmell wrote:
>>> I think you have misunderstood what I meant by that entirely, as by "the
>>> build system in general to always generate the release style artifacts" I
>>> meant exactly what you are suggesting, that the build system should be
>>> modified to generate something we can simply take a (partial) tarball of and
>>> end up with what we release...ie release style. The reason there is anything
>>> else in place right now is that they arent, IMO.
>> Sorry, I see how you meant that now. My mistake. Clearly I've been too eager
>> to rant recently. ;)
>>
>> --Rafael
>
> As the one who modified our current Java build system to produce
> 'release' artefacts I can say it is very easy to generate a packaged
> that contains the dependecies for a module. Add 1 line to the module's
> build.xml:
>
> <target name="release-bin" depends="release-bin-tasks"/>
I wasn't suggesting that this is difficult to do, merely that it uses a
completely different mechanism than the one for generating build artifacts.
> That said I undestand the desire to have a directory that we then just
> tar up for release. However, the thought that tar-ing a directory is
> in some way better than using the properties of the build system to
> make a tar is some what erroneous. There is no reason that the two
> cannot be the same.
I think I may be missing something here. If they're the same, why not do
it the simple way (i.e. tar up the directory).
In any case, I'm not really saying that we shouldn't use build system
properties. What I'm saying is that I don't want the process for
generating release artifacts to be independent from the one used for
generating build artifacts.
I have to say I have NEVER used the files in the
> build directory and the fact that all the jars end up in the lib dir
> and so get put in to qpid-all.jar makes for a very dangerous situation
> IMO. Testing with this setup can result in runtime dependencies not
> being reflected by the build system.
You may not use the files in the build directory directly, but you do
use them indirectly because that's the environment that the tests are
run in, e.g. when the external java broker is launched, it's done by
executing build/bin/qpid-server.
> I would be all for having the release artefacts being built in their
> own build directories build/broker, build/client, build/qmf, etc. I'm
> totally open to suggestions on how we can do this I simply used the
> build system for the task last time as it has knowledge of all the
> library dependencies and shipping jars that we don't need as part of
> our packages is quite unhelpful, forcing these on to the class path
> via qpid-all.jar as we do in the current build directory is just a
> receipe for disaster.
>
> Now I feel I may have rant a bit, that wasn't really my intention. I
> mearly wanted to point out that if we want to do component releases
> form the Java directory for 0.6 it would take less that 2minutes to
> sort out. So I think it is a totally doable.
I completely agree qpid-all.jar is a bad idea. I believe it exists as a
substitute for the giant client.jar that we used to generate by munging
all the class files together into a single jar, and it replicates some
of the same problems that system had. I'm not sure why this still
exists, or indeed why it ever existed. I would be +1 on killing
qpid-all.jar and using properly defined classpaths instead. (After 0.6
goes out.)
I'm also fine on adding a component release for qman for 0.6. What I
don't think is a good idea at this stage (i.e. for 0.6) is to remove the
full release or recompose it from the component releases since the full
release is the environment that we actually run our tests in.
--Rafael
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
Re: Release artifacts
Posted by Martin Ritchie <ri...@apache.org>.
2009/12/9 Rafael Schloming <ra...@redhat.com>:
> Robbie Gemmell wrote:
>>
>> I think you have misunderstood what I meant by that entirely, as by "the
>> build system in general to always generate the release style artifacts" I
>> meant exactly what you are suggesting, that the build system should be
>> modified to generate something we can simply take a (partial) tarball of and
>> end up with what we release...ie release style. The reason there is anything
>> else in place right now is that they arent, IMO.
>
> Sorry, I see how you meant that now. My mistake. Clearly I've been too eager
> to rant recently. ;)
>
> --Rafael
As the one who modified our current Java build system to produce
'release' artefacts I can say it is very easy to generate a packaged
that contains the dependecies for a module. Add 1 line to the module's
build.xml:
<target name="release-bin" depends="release-bin-tasks"/>
That said I undestand the desire to have a directory that we then just
tar up for release. However, the thought that tar-ing a directory is
in some way better than using the properties of the build system to
make a tar is some what erroneous. There is no reason that the two
cannot be the same. I have to say I have NEVER used the files in the
build directory and the fact that all the jars end up in the lib dir
and so get put in to qpid-all.jar makes for a very dangerous situation
IMO. Testing with this setup can result in runtime dependencies not
being reflected by the build system.
I would be all for having the release artefacts being built in their
own build directories build/broker, build/client, build/qmf, etc. I'm
totally open to suggestions on how we can do this I simply used the
build system for the task last time as it has knowledge of all the
library dependencies and shipping jars that we don't need as part of
our packages is quite unhelpful, forcing these on to the class path
via qpid-all.jar as we do in the current build directory is just a
receipe for disaster.
Now I feel I may have rant a bit, that wasn't really my intention. I
mearly wanted to point out that if we want to do component releases
form the Java directory for 0.6 it would take less that 2minutes to
sort out. So I think it is a totally doable.
Regards
Martin
> ---------------------------------------------------------------------
> Apache Qpid - AMQP Messaging Implementation
> Project: http://qpid.apache.org
> Use/Interact: mailto:dev-subscribe@qpid.apache.org
>
>
--
Martin Ritchie
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
Re: Release artifacts
Posted by Rafael Schloming <ra...@redhat.com>.
Robbie Gemmell wrote:
> I think you have misunderstood what I meant by that entirely, as by "the build system in general to always generate the release style artifacts" I meant exactly what you are suggesting, that the build system should be modified to generate something we can simply take a (partial) tarball of and end up with what we release...ie release style. The reason there is anything else in place right now is that they arent, IMO.
Sorry, I see how you meant that now. My mistake. Clearly I've been too
eager to rant recently. ;)
--Rafael
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
RE: Release artifacts
Posted by Robbie Gemmell <ro...@gmail.com>.
I think you have misunderstood what I meant by that entirely, as by "the build system in general to always generate the release style artifacts" I meant exactly what you are suggesting, that the build system should be modified to generate something we can simply take a (partial) tarball of and end up with what we release...ie release style. The reason there is anything else in place right now is that they arent, IMO.
Robbie
> -----Original Message-----
> From: Rafael Schloming [mailto:rafaels@redhat.com]
> Sent: 09 December 2009 19:38
> To: dev@qpid.apache.org
> Subject: Re: Release artifacts
>
> Robbie Gemmell wrote:
> > I can see what you are saying and thus agree that it would be
> beneficial for the build system in general to always generate the
> release style artifacts. However I don't see that this means we
> shouldn't use what we have at our disposal now in order to give our
> users a better experience than we currently are.
>
> I'm saying more than just always generate the release style artifacts,
> they need to be generated in such a way that starting up a devel server
> is using the same scripting and artifacts that the end user encounters
> when they attempt the same sort of exercise from the release tarball.
> Moreover I'm saying that there should be no such thing as "release
> style
> artifacts", only build artifacts, and the various release tarballs
> should simply be archives (either whole or in part) of the build
> artifacts.
>
> And on top of all this it needs to be simple, fast, and properly do
> incremental builds so that it's actually usable for those of us who
> have
> chosen not to participate in this passing IDE fad. ;)
>
> I don't want to seem discouraging, I agree with your goals, I'm just
> particular about how we get there and I'm not eager to pile another
> quick hack on top of things. Doing all this without hacks shouldn't be
> terribly difficult as long as we figure out what we want first and
> choose something sensible.
>
> --Rafael
>
> ---------------------------------------------------------------------
> Apache Qpid - AMQP Messaging Implementation
> Project: http://qpid.apache.org
> Use/Interact: mailto:dev-subscribe@qpid.apache.org
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
Re: Release artifacts
Posted by Rafael Schloming <ra...@redhat.com>.
Robbie Gemmell wrote:
> I can see what you are saying and thus agree that it would be beneficial for the build system in general to always generate the release style artifacts. However I don't see that this means we shouldn't use what we have at our disposal now in order to give our users a better experience than we currently are.
I'm saying more than just always generate the release style artifacts,
they need to be generated in such a way that starting up a devel server
is using the same scripting and artifacts that the end user encounters
when they attempt the same sort of exercise from the release tarball.
Moreover I'm saying that there should be no such thing as "release style
artifacts", only build artifacts, and the various release tarballs
should simply be archives (either whole or in part) of the build artifacts.
And on top of all this it needs to be simple, fast, and properly do
incremental builds so that it's actually usable for those of us who have
chosen not to participate in this passing IDE fad. ;)
I don't want to seem discouraging, I agree with your goals, I'm just
particular about how we get there and I'm not eager to pile another
quick hack on top of things. Doing all this without hacks shouldn't be
terribly difficult as long as we figure out what we want first and
choose something sensible.
--Rafael
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
RE: Release artifacts
Posted by Robbie Gemmell <ro...@gmail.com>.
I can see what you are saying and thus agree that it would be beneficial for the build system in general to always generate the release style artifacts. However I don't see that this means we shouldn't use what we have at our disposal now in order to give our users a better experience than we currently are.
Robbie
> -----Original Message-----
> From: Rafael Schloming [mailto:rafaels@redhat.com]
> Sent: 09 December 2009 17:45
> To: dev@qpid.apache.org
> Subject: Re: Release artifacts
>
> Robbie Gemmell wrote:
> > I'm not meaning to suggest doing any complex transformations, rather
> using what is already there. We have single-component packages that we
> already release separately for all these items, I'm just talking about
> whacking the relevant ones in a new tar and calling job done, instead
> of having a mashup that contains things people don't want or that can't
> even be used together.
>
> My issue with this is that the single-component packages weren't done
> according to the "philosophy" of the build system, and as such the
> process for producing component release artifacts diverges somewhat
> significantly from the process for producing build artifacts. So while
> what you suggest might not involve additional complex transformations,
> it does result in something that I would prefer to avoid, which is two
> separate systems, one for producing build artifacts, and the other for
> producing release artifacts.
>
> --Rafael
>
>
> ---------------------------------------------------------------------
> Apache Qpid - AMQP Messaging Implementation
> Project: http://qpid.apache.org
> Use/Interact: mailto:dev-subscribe@qpid.apache.org
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
Re: Release artifacts
Posted by Rafael Schloming <ra...@redhat.com>.
Robbie Gemmell wrote:
> I'm not meaning to suggest doing any complex transformations, rather using what is already there. We have single-component packages that we already release separately for all these items, I'm just talking about whacking the relevant ones in a new tar and calling job done, instead of having a mashup that contains things people don't want or that can't even be used together.
My issue with this is that the single-component packages weren't done
according to the "philosophy" of the build system, and as such the
process for producing component release artifacts diverges somewhat
significantly from the process for producing build artifacts. So while
what you suggest might not involve additional complex transformations,
it does result in something that I would prefer to avoid, which is two
separate systems, one for producing build artifacts, and the other for
producing release artifacts.
--Rafael
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
RE: Release artifacts
Posted by Robbie Gemmell <ro...@gmail.com>.
I'm not meaning to suggest doing any complex transformations, rather using what is already there. We have single-component packages that we already release separately for all these items, I'm just talking about whacking the relevant ones in a new tar and calling job done, instead of having a mashup that contains things people don't want or that can't even be used together.
Robbie
> -----Original Message-----
> From: Rafael Schloming [mailto:rafaels@redhat.com]
> Sent: 09 December 2009 17:17
> To: dev@qpid.apache.org
> Subject: Re: Release artifacts
>
> Robbie Gemmell wrote:
> > Hi all,
> >
> > Now that we have entered another release phase I think it would be a
> good time to discuss the release artifacts. I would like to see 0.6
> ship with some updates to the artifacts we produce to make things more
> useful and obvious for our users.
>
> I think it's a bit late for 0.6, if I understand the timeline
> correctly.
>
> > The "Java broker, client, and tools" multi-component package appears
> to be just a copy of the etc/ lib/ and bin/ directories from the
> qpid/java/build/ dir after performing a build and this makes it quite
> messy. A lot of test configuration files are left in the etc/ dir and
> there are loads of jars in the lib/ dir, which make it very hard to
> know which is being used where or allow splitting them up easily. You
> also end up with jars that just aren't intended to be used from there
> (eg the eclipse-plugin used in the JMX Management Console releases). We
> have single component packages for the Java broker, client, and
> management tools so if we want a multi-component package including any
> of these I think we should simply bundle those up instead of just
> tarring the build folder and mashing everything together.
>
> I sort of disagree with this. There is a "philosophy" behind using the
> "just a copy" approach. The benefit is that we can eliminate a whole
> class of bugs resulting from complex transformations necessary to turn
> the build artifacts into release artifacts. Not to mention it is easier
> for us to reproducer user issues if dev and user environments are as
> similar as possible.
>
> That said I'm fine cleaning up the release artifacts and what not, I'm
> just -1 on doing it by introducing complex transformations between
> build
> and release artifacts. If we want the release artifacts to look
> different we should make the build artifacts look different.
>
> As I've said before I'm happy to help with this, but we sort of need to
> figure out what we want things to look like first, and IMHO such an
> effort should also include making release artifacts more consistent
> across the whole project.
>
> > Another result of the above is that QMan is shipped in the Java
> multi-component bundle despite the fact it is only useful if you have a
> C++ broker. I realise it is written in Java, but those packages don't
> seem to me like they should necessarily be language specific. If
> anything I would say that QMan should surely ship in the multi-
> component bundle along with the items it can actually be used with (ie
> the C++ broker and client bundle), or just be made available standalone
> (which it already is). Users aren't necessarily going to expect they
> need to download both multi-component packages to make use of
> everything contained in one of them, and as mentioned above, splitting
> QMan alone out of the java multi-component bundle would be a bit
> hideous and non obvious anyway.
> >
> > I would also like to see us tagging the source-only release artifacts
> with 'src', ie the C++ and the "full" release artifact that is just a
> copy of the entire repo. It isn't exactly helpful for a user to
> download the 50MB file then find out what they are looking for isn't
> actually in there.
>
> I also think it's a shame that our artifacts look like they come from
> entirely different projects. Even little things like readme
> capitalization, general directory layout, etc is completely different
> from artifact to artifact, and this is even after the release script
> hacks to rearchive stuff to make things a bit more consistent.
>
> --Rafael
>
>
> ---------------------------------------------------------------------
> Apache Qpid - AMQP Messaging Implementation
> Project: http://qpid.apache.org
> Use/Interact: mailto:dev-subscribe@qpid.apache.org
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
Re: Release artifacts
Posted by Rafael Schloming <ra...@redhat.com>.
Robbie Gemmell wrote:
> Hi all,
>
> Now that we have entered another release phase I think it would be a good time to discuss the release artifacts. I would like to see 0.6 ship with some updates to the artifacts we produce to make things more useful and obvious for our users.
I think it's a bit late for 0.6, if I understand the timeline correctly.
> The "Java broker, client, and tools" multi-component package appears to be just a copy of the etc/ lib/ and bin/ directories from the qpid/java/build/ dir after performing a build and this makes it quite messy. A lot of test configuration files are left in the etc/ dir and there are loads of jars in the lib/ dir, which make it very hard to know which is being used where or allow splitting them up easily. You also end up with jars that just aren't intended to be used from there (eg the eclipse-plugin used in the JMX Management Console releases). We have single component packages for the Java broker, client, and management tools so if we want a multi-component package including any of these I think we should simply bundle those up instead of just tarring the build folder and mashing everything together.
I sort of disagree with this. There is a "philosophy" behind using the
"just a copy" approach. The benefit is that we can eliminate a whole
class of bugs resulting from complex transformations necessary to turn
the build artifacts into release artifacts. Not to mention it is easier
for us to reproducer user issues if dev and user environments are as
similar as possible.
That said I'm fine cleaning up the release artifacts and what not, I'm
just -1 on doing it by introducing complex transformations between build
and release artifacts. If we want the release artifacts to look
different we should make the build artifacts look different.
As I've said before I'm happy to help with this, but we sort of need to
figure out what we want things to look like first, and IMHO such an
effort should also include making release artifacts more consistent
across the whole project.
> Another result of the above is that QMan is shipped in the Java multi-component bundle despite the fact it is only useful if you have a C++ broker. I realise it is written in Java, but those packages don't seem to me like they should necessarily be language specific. If anything I would say that QMan should surely ship in the multi-component bundle along with the items it can actually be used with (ie the C++ broker and client bundle), or just be made available standalone (which it already is). Users aren't necessarily going to expect they need to download both multi-component packages to make use of everything contained in one of them, and as mentioned above, splitting QMan alone out of the java multi-component bundle would be a bit hideous and non obvious anyway.
>
> I would also like to see us tagging the source-only release artifacts with 'src', ie the C++ and the "full" release artifact that is just a copy of the entire repo. It isn't exactly helpful for a user to download the 50MB file then find out what they are looking for isn't actually in there.
I also think it's a shame that our artifacts look like they come from
entirely different projects. Even little things like readme
capitalization, general directory layout, etc is completely different
from artifact to artifact, and this is even after the release script
hacks to rearchive stuff to make things a bit more consistent.
--Rafael
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org