You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@jackrabbit.apache.org by Felix Meschberger <fm...@gmail.com> on 2008/04/20 15:03:57 UTC
Concerns about release vote behaviour
Hi all,
I have a growing concern about our latest releases. Most of the time we
barely get the required minimum of 3 +1 votes to release the stuff. Take
as an example some recent release vote results:
* Jackrabbit 1.3.4 - 3 votes
* jackrabbit-core 1.4.2 - 4 votes (of which 1 non-binding)
* Jackrabbit 1.4 - 4 votes (of which one non-binding -1)
* jackrabbit-jcr-commons 1.4.2 - 4 votest
* jackrabbit-core 1.4.1 - 4 votes
* Jackrabbit 1.3.3 - 5 votes (of which 1 non-binding)
* Jackrabbit 1.3.1 - 5 votes (of which 2 non-binding)
Now, compared to the number of committers/PMC members we have - 20
according to [1] - this is IMHO not enough backing for releases.
How come ? Could it be that we just don't feel comfortable enough with
the code base we are working on day-on and day-off ? Is it, that we
cannot bear the responsibility of releasing some code, we could not test
thorougly ourselves ? I cannot tell. And the reasons for these
abstentions are probably none of my business.
What I really am looking for is more votes on our releases to show our
user community that the Jackrabbit PMC is in fact proud and confident of
its product.
It is true, that the PMC is responsible for the published releases:
The main role of the PMC is not code and not coding - but to
ensure that all legal issues are addressed, that procedure is
followed, and that each and every release is the product of the
community as a whole (see [2]).
So a release vote, as I understand it, is not primarily about whether
the product code is 100% correct. Rather the question is whether the
code was developped correctly with respect to the community and that
legal issues have been addressed, e.g. required files such as LICENSE
and NOTICE files are in place.
And this is actually, what I do when considering my vote:
* I get the complete release from the release candidate location
* I check all checksums and signatures
* I run rat to check for the license headers ([3] and [4])
* Check NOTICE files
If all goes well, which it normally does, I vote +1. There is nothing
more I do. In particular I do not normally test drive the releases. And
I do not think, that this is really needed because of the excellent test
cases we have and the community constantly working on the code and
trying its best to keep it going. In short, I trust in the community
(aka developpers) in producing good quality code.
I hope these words help raise the release vote activity again in the
future. I really think Jackrabbit deserves more than just 3 +1 votes on
releases.
Thanks for your patience and time. Have fun !
Regards
Felix
PS: I uploaded two helper scripts which I use to get (getrelease) and
check releases (ckrelease; checksums and signatures only) to [5]. Use as
you see fit.
[1] http://jackrabbit.apache.org/jackrabbit-team.html
[2] http://www.apache.org/foundation/how-it-works.html#pmc
[3] http://incubator.apache.org/rat/
[4] http://code.google.com/p/arat/
[5] http://people.apache.org/~fmeschbe/release_helpers/
Re: Concerns about release vote behaviour
Posted by "Roy T. Fielding" <fi...@gbiv.com>.
On Apr 21, 2008, at 11:47 AM, Thomas Mueller wrote:
> Hi,
>
>> The important release artifact to check is the source archive, the
>> binary artifacts are mostly a convenience to users.
>
>> The binaries are irrelevant.
>
> OK, I understand, but I don't agree. Most users download the binaries;
> very few download the source code and even less build the binaries
Apache's users download the source code and build from source.
Jukka's users may just run the binaries.
> themselves. I think the binaries are important. If the release scripts
> are correct the binaries should be correct. But then, if the release
> scripts are correct then 'rat' is already run and I don't need to do
> that again... The binaries could contain a virus (there are some Java
> viruses). I know that some developers disabled the virus scanner (well
> I do that sometimes). Probably it's not that urgent, but maybe when we
> have time to improve the release process we find a solution for that
> as well.
Thomas, there is no way to verify that a binary is redistributable
without building the entire computer from trusted sources each time.
That's why we don't vote on binaries. Don't waste your abilities on
testing binaries when we need them to test the source code.
Allow me to repeat: WE DON'T VOTE ON BINARIES. We CAN'T vote on
binaries.
To vote would imply that we have the magical ability to evaluate them on
behalf of the ASF. None of us has that ability. That's why the ASF
does
not release binaries!
If it really becomes too hard for folks to understand that the binaries
do not matter, then I will ask the RM to stop building binaries. They
don't belong in the release vote, period. Is that clear? The HTTP
Server
project has never, in its entire history, voted on the release of
binaries.
Apache Jackrabbit has no reason to do so now. We let Jukka upload
binaries
that he has built from the released source code bits because we trust
Jukka, not because we trust the binaries.
....Roy
Re: Concerns about release vote behaviour
Posted by Thomas Mueller <th...@gmail.com>.
Hi,
> The important release artifact to check is the source archive, the
> binary artifacts are mostly a convenience to users.
> The binaries are irrelevant.
OK, I understand, but I don't agree. Most users download the binaries;
very few download the source code and even less build the binaries
themselves. I think the binaries are important. If the release scripts
are correct the binaries should be correct. But then, if the release
scripts are correct then 'rat' is already run and I don't need to do
that again... The binaries could contain a virus (there are some Java
viruses). I know that some developers disabled the virus scanner (well
I do that sometimes). Probably it's not that urgent, but maybe when we
have time to improve the release process we find a solution for that
as well.
Regards,
Thomas
Re: Concerns about release vote behaviour
Posted by "Roy T. Fielding" <fi...@gbiv.com>.
On Apr 20, 2008, at 6:03 AM, Felix Meschberger wrote:
> I have a growing concern about our latest releases. Most of the
> time we
> barely get the required minimum of 3 +1 votes to release the stuff.
> Take
> as an example some recent release vote results:
>
> * Jackrabbit 1.3.4 - 3 votes
> * jackrabbit-core 1.4.2 - 4 votes (of which 1 non-binding)
> * Jackrabbit 1.4 - 4 votes (of which one non-binding -1)
> * jackrabbit-jcr-commons 1.4.2 - 4 votest
> * jackrabbit-core 1.4.1 - 4 votes
> * Jackrabbit 1.3.3 - 5 votes (of which 1 non-binding)
> * Jackrabbit 1.3.1 - 5 votes (of which 2 non-binding)
>
>
> Now, compared to the number of committers/PMC members we have - 20
> according to [1] - this is IMHO not enough backing for releases.
Three people giving a true +1 to a release is adequate. Not ideal, but
still adequate, providing that none of those +1s are bogus.
> How come ? Could it be that we just don't feel comfortable enough with
> the code base we are working on day-on and day-off ? Is it, that we
> cannot bear the responsibility of releasing some code, we could not
> test
> thorougly ourselves ? I cannot tell. And the reasons for these
> abstentions are probably none of my business.
The Apache release process is designed to work with part-time
volunteers.
We only need enough attention, not absolute attention.
> What I really am looking for is more votes on our releases to show our
> user community that the Jackrabbit PMC is in fact proud and
> confident of
> its product.
>
> It is true, that the PMC is responsible for the published releases:
> The main role of the PMC is not code and not coding - but to
> ensure that all legal issues are addressed, that procedure is
> followed, and that each and every release is the product of
> the
> community as a whole (see [2]).
>
> So a release vote, as I understand it, is not primarily about whether
> the product code is 100% correct. Rather the question is whether the
> code was developped correctly with respect to the community and that
> legal issues have been addressed, e.g. required files such as LICENSE
> and NOTICE files are in place.
>
> And this is actually, what I do when considering my vote:
>
> * I get the complete release from the release candidate location
> * I check all checksums and signatures
> * I run rat to check for the license headers ([3] and [4])
> * Check NOTICE files
>
> If all goes well, which it normally does, I vote +1. There is nothing
> more I do.
Sorry, that is a bogus +1. All of your release votes are now invalid.
Each time you do that you decrease the ability of the team as a whole
to verify that at least three people have built the source code on their
own system and run whatever normal tests that they consider
sufficient to
verify that the SOURCE CODE provided does indeed build into a release
that is, in their opinion, at least as good if not better than the
prior release of that product. That's what +1 means for a release.
The binaries are irrelevant. Aside from checking the signatures, just
trust that the RM built them from the same source. What matters for the
release vote is that the source code matches what we have in subversion,
is buildable, and runs on your own platform. I stopped voting
because my
platform stopped working, but I should be able to rely on the others to
verify that it is just my personal install of java that sucks and not
our released code.
I don't care how often a release gets delayed because not enough people
have built and tested it -- they will show up when progress is delayed.
I do care a great deal when folks say that they will submit bogus +1s
just to make up the appearance of progress. That simply is not good
enough.
If it persists, you will be removed from the PMC. Apache cannot afford
to allow releases to become an exercise in marketing.
....Roy
Re: Concerns about release vote behaviour
Posted by Jukka Zitting <ju...@gmail.com>.
Hi,
On Mon, Apr 21, 2008 at 5:46 PM, Thomas Mueller
<th...@gmail.com> wrote:
> The reason that I don't vote very often is because I am not very
> comfortable with it.
>
> I would like to make sure the jar files reflect the source code in the
> branch. I would need to compile the source code myself using the same
> compiler (JVM) and compare the jar files in binary mode. It would be
> good to know what compiler was used.
The important release artifact to check is the source archive, the
binary artifacts are mostly a convenience to users. Personally I trust
the release manager to have compiled things correctly, so I generally
just check the checksums of the binaries and try to spot any obvious
surprises in binary artifact sizes. Sometimes I also deploy the
binaries to a test installation and try them out, but only if I feel
like it and have the extra time. In general there's no good way to
review binaries so you in any case need to trust the release manager
to some extent.
> Checksums: If the checksums are on the same server as the compiled
> files, an attacker would only have to replace both files. It probably
> makes sense to always distribute the checksums in some other way (for
> example in the mail). This was done sometimes, but not always.
+1 I've recently started including the checksums in the VOTE messages
for exactly this reason.
> If multiple components are going to be released, I would prefer to only
> check one file (for example a .zip file that contains all other files).
As mentioned above, the only really important file is the source archive.
At some point I've even considered whether it would be OK to separate
at least the Maven repository update from the release vote. We would
first vote on a pure source release (packaged svn export of a tag),
and if the vote passes the release manager could then run "mvn deploy"
on the approved sources without a separate review. I'm not sure how
well this would play out with Apache policies, but it would make
releasing much simpler.
BR,
Jukka Zitting
Re: Concerns about release vote behaviour
Posted by Thomas Mueller <th...@gmail.com>.
Hi,
The reason that I don't vote very often is because I am not very
comfortable with it.
I would like to make sure the jar files reflect the source code in the
branch. I would need to compile the source code myself using the same
compiler (JVM) and compare the jar files in binary mode. It would be
good to know what compiler was used.
Checksums: If the checksums are on the same server as the compiled
files, an attacker would only have to replace both files. It probably
makes sense to always distribute the checksums in some other way (for
example in the mail). This was done sometimes, but not always. If
multiple components are going to be released, I would prefer to only
check one file (for example a .zip file that contains all other
files).
Regards,
Thomas
On Sun, Apr 20, 2008 at 3:03 PM, Felix Meschberger <fm...@gmail.com> wrote:
> Hi all,
>
> I have a growing concern about our latest releases. Most of the time we
> barely get the required minimum of 3 +1 votes to release the stuff. Take
> as an example some recent release vote results:
>
> * Jackrabbit 1.3.4 - 3 votes
> * jackrabbit-core 1.4.2 - 4 votes (of which 1 non-binding)
> * Jackrabbit 1.4 - 4 votes (of which one non-binding -1)
> * jackrabbit-jcr-commons 1.4.2 - 4 votest
> * jackrabbit-core 1.4.1 - 4 votes
> * Jackrabbit 1.3.3 - 5 votes (of which 1 non-binding)
> * Jackrabbit 1.3.1 - 5 votes (of which 2 non-binding)
>
>
> Now, compared to the number of committers/PMC members we have - 20
> according to [1] - this is IMHO not enough backing for releases.
>
> How come ? Could it be that we just don't feel comfortable enough with
> the code base we are working on day-on and day-off ? Is it, that we
> cannot bear the responsibility of releasing some code, we could not test
> thorougly ourselves ? I cannot tell. And the reasons for these
> abstentions are probably none of my business.
>
> What I really am looking for is more votes on our releases to show our
> user community that the Jackrabbit PMC is in fact proud and confident of
> its product.
>
> It is true, that the PMC is responsible for the published releases:
> The main role of the PMC is not code and not coding - but to
> ensure that all legal issues are addressed, that procedure is
> followed, and that each and every release is the product of the
> community as a whole (see [2]).
>
> So a release vote, as I understand it, is not primarily about whether
> the product code is 100% correct. Rather the question is whether the
> code was developped correctly with respect to the community and that
> legal issues have been addressed, e.g. required files such as LICENSE
> and NOTICE files are in place.
>
> And this is actually, what I do when considering my vote:
>
> * I get the complete release from the release candidate location
> * I check all checksums and signatures
> * I run rat to check for the license headers ([3] and [4])
> * Check NOTICE files
>
> If all goes well, which it normally does, I vote +1. There is nothing
> more I do. In particular I do not normally test drive the releases. And
> I do not think, that this is really needed because of the excellent test
> cases we have and the community constantly working on the code and
> trying its best to keep it going. In short, I trust in the community
> (aka developpers) in producing good quality code.
>
> I hope these words help raise the release vote activity again in the
> future. I really think Jackrabbit deserves more than just 3 +1 votes on
> releases.
>
> Thanks for your patience and time. Have fun !
>
> Regards
> Felix
>
> PS: I uploaded two helper scripts which I use to get (getrelease) and
> check releases (ckrelease; checksums and signatures only) to [5]. Use as
> you see fit.
>
> [1] http://jackrabbit.apache.org/jackrabbit-team.html
> [2] http://www.apache.org/foundation/how-it-works.html#pmc
> [3] http://incubator.apache.org/rat/
> [4] http://code.google.com/p/arat/
> [5] http://people.apache.org/~fmeschbe/release_helpers/
>
>
Re: Concerns about release vote behaviour
Posted by Jukka Zitting <ju...@gmail.com>.
Hi,
On Sun, Apr 20, 2008 at 4:03 PM, Felix Meschberger <fm...@gmail.com> wrote:
> I have a growing concern about our latest releases. Most of the time we
> barely get the required minimum of 3 +1 votes to release the stuff.
Agreed. Luckily we haven't seen any release votes that could not pass
due to missing votes (though a few have needed extended vote periods
to get all the required votes), so I don't think the issue is too bad.
But it's certainly worthwhile to look at the release and review
processes to figure out how we could make it easier to review things.
> So a release vote, as I understand it, is not primarily about whether
> the product code is 100% correct. Rather the question is whether the
> code was developped correctly with respect to the community and that
> legal issues have been addressed, e.g. required files such as LICENSE
> and NOTICE files are in place.
> [...]
> If all goes well, which it normally does, I vote +1. There is nothing
> more I do. In particular I do not normally test drive the releases. And
> I do not think, that this is really needed because of the excellent test
> cases we have and the community constantly working on the code and
> trying its best to keep it going. In short, I trust in the community
> (aka developpers) in producing good quality code.
I personally do put concern also on the code quality in a release,
i.e. the release should be reasonably stable, have no major
undocumented issues, etc. But that certainly doesn't mean that the
code would need to be perfect. In fact I think the known issues
section in the release notes is one of the most important things we
produce, and I'm quite proud of how well we do keep track of all known
defects.
BR,
Jukka Zitting