You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by Daniel Shahaf <d....@daniel.shahaf.name> on 2017/08/12 17:35:07 UTC
[RELEASES] Relaxing the release signature requirements
There has been some random discussion about relaxing the release
signature requirements.
Our hard requirements are:
- One signature [ASF policy]
- Three +1 votes [ASF policy]
Our current policy is:
- One RM signature
- Three unix +1s with signatures
- Three windows +1s with signatures
+ Traditionally these would be seven separate persons, but nowadays we
are less strict about this.
[The above is facts. The below is opinion.]
Properties we want the new policy to have:
- Fewer than seven people involved
- Testing on all platforms
- Complies with ASF policy
So, how about:
- Allowing the RM to cast a +1.
I think this shouldn't be automatic; the RM does not automatically
cast a +1 on account of having rolled the tarballs, rather, the RM
only casts a +1 if he specifically sends a "Summary: +1 to release"
vote indicating he'd tested the tarballs [which he'd produced] the
usual way.
The rationale is that _rolling_ a tarball is entirely different to
_testing_ it.
- Requiring fewer than three +1s per platform.
E.g., we could require just two windows +1s and two unix +1s.
If we do this, I would prefer to see the two unix +1s from different
unix variants. (We have at least four devs on debian/ubuntu, and I
don't think four +1s one these two are equal to two +1s from two more
different unixoid platforms.) Likewise on windows, I suppose, but I
don't know that platform's variations well enough to have an opinion.
These two changes together would mean that only four people would be
needed to make a release: two devs per platform, one of whom doubles as
an RM. We could even theoretically manage a release with only three
developers, if one of them tested on two platforms, two tested on one
platform each, and one of the three acted as RM — but having fewer
people involved increases the risk of overlooking some showstopper.
To be concrete, here's the suggestion again without annotations:
- A tarball is rolled by the RM.
- The RM signs the tarball before uploading to /dist/dev.
- The tarball is tested by at least two windows developers, who
SHOULD use different variants of Windows, and receives their +1
votes and signatures. The RM MAY be one of these developers.
- The tarball is tested by at least two unix developers, who
SHOULD use different Unixoid platforms, and receives their +1
votes and signatures. The RM MAY be one of these developers.
- There MAY be more than two testers per platform. The testers need
not be committers.
- There SHOULD be at least four different testers.
- Developers MAY sign the tarball without testing it, only if they
have verified that it matches the tag [with the expected differences].
- The release timelines (at least 72 hours and preferably more for
testing/votes, then 24 hours for the mirrors) are unchanged.
WDYT?
Re: [RELEASES] Relaxing the release signature requirements
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Branko Čibej wrote on Sun, 13 Aug 2017 16:23 +0200:
> On 13.08.2017 16:11, Mark Phippard wrote:
> > Our policy should reflect our minimum requirements. They do not replace judgement. The problem with our current policy and this proposal is it still requires us to adjust the policy on the fly when our better judgement tells us we need to push out a release.
>
> Good point. And the best example of that are the recent two releases.
>
> +1 for assuming the RM is rational and responsible.
That's exactly what I was trying to get at with the use of SHOULD's: to
simultaneously express best practice on the one hand, and give the
community and the RM the leeway to release without meeting that bar if
circumstances merit doing so.
For example, the proposal does not _require_ having both a Linux and a
non-Linux unix signature; it merely says that it is preferable to have
signatures from varied platforms than from homogenous platforms.
I do think there's value in explicitly documenting best practice when we
have fewer developers, since that means there's less opportunity for
"informal" inter-generation knowledge transfer.
Re: [RELEASES] Relaxing the release signature requirements
Posted by Johan Corveleyn <jc...@gmail.com>.
On Wed, Aug 16, 2017 at 10:42 PM, Evgeny Kotkov
<ev...@visualsvn.com> wrote:
> Daniel Shahaf <d....@daniel.shahaf.name> writes:
>
>> Ping? We seem to have consensus on lowering the bar, but not yet consensus
>> on what to. It'd be nice to have finished this by the next release.
>
> Dropping my 2 cents, I think that lowering the requirement to 2+2 votes
> (where the RM may also cast a vote, if needed) would be good and enough
> to prevent unwanted stalls during the release process.
>
> Based on some of the previous releases (for example, 1.9.7, 1.8.19 or
> 1.8.17), I think that usually we're just one vote short for one of the
> platforms. Lowering the bar to 2+2 and allowing the RM to vote should
> probably be enough to cover such situations while still maintaining a
> decent test coverage and guarding against various mistakes in the
> process.
+1. Let's lower the bar to 2+2 (with RM allowed to vote) like in
Daniel's proposal. We can always lower it further like Mark suggested,
if 2+2 still proves to be problematic or too close for comfort.
--
Johan
Re: [RELEASES] Relaxing the release signature requirements
Posted by Evgeny Kotkov <ev...@visualsvn.com>.
Daniel Shahaf <d....@daniel.shahaf.name> writes:
> Ping? We seem to have consensus on lowering the bar, but not yet consensus
> on what to. It'd be nice to have finished this by the next release.
Dropping my 2 cents, I think that lowering the requirement to 2+2 votes
(where the RM may also cast a vote, if needed) would be good and enough
to prevent unwanted stalls during the release process.
Based on some of the previous releases (for example, 1.9.7, 1.8.19 or
1.8.17), I think that usually we're just one vote short for one of the
platforms. Lowering the bar to 2+2 and allowing the RM to vote should
probably be enough to cover such situations while still maintaining a
decent test coverage and guarding against various mistakes in the
process.
Thanks,
Evgeny Kotkov
Re: [RELEASES] Relaxing the release signature requirements
Posted by Stefan Sperling <st...@elego.de>.
On Tue, Aug 15, 2017 at 04:33:31PM +0000, Daniel Shahaf wrote:
> Ping? We seem to have consensus on lowering the bar, but not yet consensus on what to. It'd be nice to have finished this by the next release.
>
> (editor hat on)
I agree with Mark.
Let's require a lower count of +1 votes (3 in total, including RM), drop
any OS-specific testing requirements, and we're done.
The real problem we have is that we're barely able to reach our self-imposed
signature counts. We haven't yet had a problem where any particular platform
was underrepresented during release testing. So I agree that we don't need to
formalize platform-specific tests and can rely on the RM's judgement for this.
I expect that, in practice, we'll be doing releases with one RM signature,
at least one unix-like signature, and at least one windows signature.
Re: [RELEASES] Relaxing the release signature requirements
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Ping? We seem to have consensus on lowering the bar, but not yet consensus on what to. It'd be nice to have finished this by the next release.
(editor hat on)
Branko Čibej wrote on Sun, 13 Aug 2017 16:23 +0200:
> On 13.08.2017 16:11, Mark Phippard wrote:
> > I maintain that we should adopt a policy similar to httpd, which would essentially be to require at least three +1 votes total with no specific OS requirements.
> >
> > Nothing forces the RM to publish a release. If all votes are from the same OS and the RM thinks more votes are needed, then use your judgement. Announce you want more signatures and hold the release.
> >
> > Our policy should reflect our minimum requirements. They do not replace judgement. The problem with our current policy and this proposal is it still requires us to adjust the policy on the fly when our better judgement tells us we need to push out a release.
>
> Good point. And the best example of that are the recent two releases.
>
> +1 for assuming the RM is rational and responsible.
>
> -- Brane
Re: [RELEASES] Relaxing the release signature requirements
Posted by Branko Čibej <br...@apache.org>.
On 13.08.2017 16:11, Mark Phippard wrote:
> I maintain that we should adopt a policy similar to httpd, which would essentially be to require at least three +1 votes total with no specific OS requirements.
>
> Nothing forces the RM to publish a release. If all votes are from the same OS and the RM thinks more votes are needed, then use your judgement. Announce you want more signatures and hold the release.
>
> Our policy should reflect our minimum requirements. They do not replace judgement. The problem with our current policy and this proposal is it still requires us to adjust the policy on the fly when our better judgement tells us we need to push out a release.
Good point. And the best example of that are the recent two releases.
+1 for assuming the RM is rational and responsible.
-- Brane
Re: [RELEASES] Relaxing the release signature requirements
Posted by Mark Phippard <ma...@gmail.com>.
I maintain that we should adopt a policy similar to httpd, which would essentially be to require at least three +1 votes total with no specific OS requirements.
Nothing forces the RM to publish a release. If all votes are from the same OS and the RM thinks more votes are needed, then use your judgement. Announce you want more signatures and hold the release.
Our policy should reflect our minimum requirements. They do not replace judgement. The problem with our current policy and this proposal is it still requires us to adjust the policy on the fly when our better judgement tells us we need to push out a release.
Mark
Re: [RELEASES] Relaxing the release signature requirements
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Johan Corveleyn wrote on Sun, 13 Aug 2017 14:37 +0200:
> What do you mean with "The testers need not be committers"? Do you
> mean the *extra* testers from the sentence before ("There MAY be more
> than two testers per platform")? I.e. anyone who tests and sends their
> results to the list provides extra value, but the two binding votes we
> need (per platform) have to come from committers / pmc members, right?
Yes. Lieven, for example, used to test releases even before he'd been made
a full committer; I meant to document and encourage that pattern.
Re: [RELEASES] Relaxing the release signature requirements
Posted by Johan Corveleyn <jc...@gmail.com>.
On Sat, Aug 12, 2017 at 7:35 PM, Daniel Shahaf <d....@daniel.shahaf.name> wrote:
> There has been some random discussion about relaxing the release
> signature requirements.
>
> Our hard requirements are:
>
> - One signature [ASF policy]
> - Three +1 votes [ASF policy]
>
> Our current policy is:
>
> - One RM signature
> - Three unix +1s with signatures
> - Three windows +1s with signatures
> + Traditionally these would be seven separate persons, but nowadays we
> are less strict about this.
>
> [The above is facts. The below is opinion.]
>
> Properties we want the new policy to have:
>
> - Fewer than seven people involved
> - Testing on all platforms
> - Complies with ASF policy
>
> So, how about:
>
> - Allowing the RM to cast a +1.
>
> I think this shouldn't be automatic; the RM does not automatically
> cast a +1 on account of having rolled the tarballs, rather, the RM
> only casts a +1 if he specifically sends a "Summary: +1 to release"
> vote indicating he'd tested the tarballs [which he'd produced] the
> usual way.
>
> The rationale is that _rolling_ a tarball is entirely different to
> _testing_ it.
>
> - Requiring fewer than three +1s per platform.
>
> E.g., we could require just two windows +1s and two unix +1s.
>
> If we do this, I would prefer to see the two unix +1s from different
> unix variants. (We have at least four devs on debian/ubuntu, and I
> don't think four +1s one these two are equal to two +1s from two more
> different unixoid platforms.) Likewise on windows, I suppose, but I
> don't know that platform's variations well enough to have an opinion.
>
> These two changes together would mean that only four people would be
> needed to make a release: two devs per platform, one of whom doubles as
> an RM. We could even theoretically manage a release with only three
> developers, if one of them tested on two platforms, two tested on one
> platform each, and one of the three acted as RM — but having fewer
> people involved increases the risk of overlooking some showstopper.
>
> To be concrete, here's the suggestion again without annotations:
>
> - A tarball is rolled by the RM.
>
> - The RM signs the tarball before uploading to /dist/dev.
>
> - The tarball is tested by at least two windows developers, who
> SHOULD use different variants of Windows, and receives their +1
> votes and signatures. The RM MAY be one of these developers.
>
> - The tarball is tested by at least two unix developers, who
> SHOULD use different Unixoid platforms, and receives their +1
> votes and signatures. The RM MAY be one of these developers.
>
> - There MAY be more than two testers per platform. The testers need
> not be committers.
>
> - There SHOULD be at least four different testers.
>
> - Developers MAY sign the tarball without testing it, only if they
> have verified that it matches the tag [with the expected differences].
>
> - The release timelines (at least 72 hours and preferably more for
> testing/votes, then 24 hours for the mirrors) are unchanged.
>
> WDYT?
Overall, I like your proposal. Reducing it to 2+2 sounds quite reasonable to me.
What do you mean with "The testers need not be committers"? Do you
mean the *extra* testers from the sentence before ("There MAY be more
than two testers per platform")? I.e. anyone who tests and sends their
results to the list provides extra value, but the two binding votes we
need (per platform) have to come from committers / pmc members, right?
I'm a bit worried about the extra requirement of "SHOULD use different
unixoid/windows platforms" (and I'm not sure what it would mean for
Windows: different versions (Seven vs. 10)? 32-bit vs. 64-bit?
Different VS versions?). We don't have that requirement right now, so
this would raise the bar from where we are now. At present, in theory
the 3 unix sigs can all come from Ubuntu, and the 3 windows sigs can
all come from the exact same Windows version.
Of course you said "SHOULD", so it's more a preference than a
requirement ... maybe it's not a problem in practice. If the two
binding unix votes come from the same platform, will we accept the
vote as valid?
What about the bindings? I've never built / tested them myself, so my
votes are less "complete" than some of the other devs. Yet it counts
for the current 3 sigs requirement (fortunately other devs do test the
bindings). In the new scheme: what if both Windows sigs would not
include the bindings? Should we require at least one vote per platform
to include the bindings?
--
Johan