Experiment: Voting and revoting

[David Blevins <da...@gmail.com>]

We typically reroll and revote several times.  This ok and rerolling is expected (you need it for quality).  But there is a cost to rerolling:

  - if the binaries are too much of moving target people wait to give their feedback
  - it isn't realistic or a good use of time to expect people to honestly evaluate multiple sets of binaries
  - we loose many votes on each reroll and final votes usually have a fraction of the +1s
  - people feel bad about voting -1 and they shouldn't -- it's ok if done in a timely manner

As an experiment for this vote (we'll see how it goes), I'd like to enforce a strict 72 hour period.  If you plan on voting, get your vote in by Friday 10pm PDT.


Here's the thinking....


FIRST VOTE

You should feel welcome and encouraged to vote -1 in this time.  You aren't raining on anyone's parade :)  In fact, you are contributing to quality.  It looks negative "-1", but it is actually a positive contribution.  Please use the "-1" for clarity.


SECOND VOTE

At the end of this 72 hour period we will roll a second set of binaries that incorporate all the feedback of the first.

 - Everyone who voted in the first vote will be expected to vote again.
 - The vote will close when either:
     - everyone from the first vote has vote again
     - 72 hours

The vote can close early if everyone reaffirms their +1s or that their issue (-1) was addressed, so please reaffirm your vote.

This can be a very short vote if everyone cooperates.  So 5 minutes or 72 hours, it's up to us :)

Subsequent votes, if needed, will follow these rules.


MAKING UP FOR BEING LATE

If you did not vote at all in the first vote and feel the need to vote -1 in a subsequent vote, your feedback is still welcome but understand that your feedback is also very late and you have cost everyone who did take the time to vote at least an hour of their time (more if there have already been a couple votes).  Do not do so lightly.

Here are some great ways you can offset the cost:

 - Immediately fix the issue you point out
 - Give each individual who voted an hour of service, per vote, on a task of their choosing.

That would probably cover it nicely :)


POSSIBLE VARIATIONS

We'll see how it goes, but if the basic concept works out, I can see maybe extending the first vote period to say 4 days (96 hours) instead of 3 (72 hours).


Anyway, we'll try it out and see if it has a positive impact on the flow of releases.


-David

Re: Experiment: Voting and revoting

[Jean-Louis MONTEIRO <je...@gmail.com>]

+1 Jon.
If we want to check the release, there is a lot of things to do.

I usually also rebuild everything from an empty repo. Without my new
computer, ... you know the history ;-) even with my new MAC it took more
than an hour just to rebuild everything.

Just a note.
Thx anyway guys
Jean-Louis
Le 20 avr. 2012 01:54, "Jonathan Gallimore" <jo...@gmail.com>
a écrit :

> I think this is a great idea and I'm looking forward to seeing the outcome.
> Just one thought I had (and you have already noted it) is that for me, it
> would be slightly more ideal if had some weekend time to look over things -
> a 72 hour vote that ends on a Friday is a little tricky for me. I'll do
> what I can tomorrow to try and get a vote in.
>
> I might be over-doing it, but I usually spend a reasonably big chunk of
> time on voting: I usually run a build from the tag with all the tests,
> check that I can download the artifacts, and that the checksums are
> correct. I take a good look at the legal report and try and spot anything
> that isn't right. Finally I try a clean unzip of the binaries and try some
> other applications and give it a good test - usually using something I'm
> working on in the day job at the time. I'd say it takes quite a bit longer
> than an hour - probably more like 3 or 4.
>
> Jon
>
> On Thu, Apr 19, 2012 at 12:06 AM, David Blevins <david.blevins@gmail.com
> >wrote:
>
> > We typically reroll and revote several times.  This ok and rerolling is
> > expected (you need it for quality).  But there is a cost to rerolling:
> >
> >  - if the binaries are too much of moving target people wait to give
> their
> > feedback
> >  - it isn't realistic or a good use of time to expect people to honestly
> > evaluate multiple sets of binaries
> >  - we loose many votes on each reroll and final votes usually have a
> > fraction of the +1s
> >  - people feel bad about voting -1 and they shouldn't -- it's ok if done
> > in a timely manner
> >
> > As an experiment for this vote (we'll see how it goes), I'd like to
> > enforce a strict 72 hour period.  If you plan on voting, get your vote in
> > by Friday 10pm PDT.
> >
> >
> > Here's the thinking....
> >
> >
> > FIRST VOTE
> >
> > You should feel welcome and encouraged to vote -1 in this time.  You
> > aren't raining on anyone's parade :)  In fact, you are contributing to
> > quality.  It looks negative "-1", but it is actually a positive
> > contribution.  Please use the "-1" for clarity.
> >
> >
> > SECOND VOTE
> >
> > At the end of this 72 hour period we will roll a second set of binaries
> > that incorporate all the feedback of the first.
> >
> >  - Everyone who voted in the first vote will be expected to vote again.
> >  - The vote will close when either:
> >     - everyone from the first vote has vote again
> >     - 72 hours
> >
> > The vote can close early if everyone reaffirms their +1s or that their
> > issue (-1) was addressed, so please reaffirm your vote.
> >
> > This can be a very short vote if everyone cooperates.  So 5 minutes or 72
> > hours, it's up to us :)
> >
> > Subsequent votes, if needed, will follow these rules.
> >
> >
> > MAKING UP FOR BEING LATE
> >
> > If you did not vote at all in the first vote and feel the need to vote -1
> > in a subsequent vote, your feedback is still welcome but understand that
> > your feedback is also very late and you have cost everyone who did take
> the
> > time to vote at least an hour of their time (more if there have already
> > been a couple votes).  Do not do so lightly.
> >
> > Here are some great ways you can offset the cost:
> >
> >  - Immediately fix the issue you point out
> >  - Give each individual who voted an hour of service, per vote, on a task
> > of their choosing.
> >
> > That would probably cover it nicely :)
> >
> >
> > POSSIBLE VARIATIONS
> >
> > We'll see how it goes, but if the basic concept works out, I can see
> maybe
> > extending the first vote period to say 4 days (96 hours) instead of 3 (72
> > hours).
> >
> >
> > Anyway, we'll try it out and see if it has a positive impact on the flow
> > of releases.
> >
> >
> > -David
> >
> >
>

Re: Experiment: Voting and revoting

[Jonathan Gallimore <jo...@gmail.com>]

I think this is a great idea and I'm looking forward to seeing the outcome.
Just one thought I had (and you have already noted it) is that for me, it
would be slightly more ideal if had some weekend time to look over things -
a 72 hour vote that ends on a Friday is a little tricky for me. I'll do
what I can tomorrow to try and get a vote in.

I might be over-doing it, but I usually spend a reasonably big chunk of
time on voting: I usually run a build from the tag with all the tests,
check that I can download the artifacts, and that the checksums are
correct. I take a good look at the legal report and try and spot anything
that isn't right. Finally I try a clean unzip of the binaries and try some
other applications and give it a good test - usually using something I'm
working on in the day job at the time. I'd say it takes quite a bit longer
than an hour - probably more like 3 or 4.

Jon

On Thu, Apr 19, 2012 at 12:06 AM, David Blevins <da...@gmail.com>wrote:

> We typically reroll and revote several times.  This ok and rerolling is
> expected (you need it for quality).  But there is a cost to rerolling:
>
>  - if the binaries are too much of moving target people wait to give their
> feedback
>  - it isn't realistic or a good use of time to expect people to honestly
> evaluate multiple sets of binaries
>  - we loose many votes on each reroll and final votes usually have a
> fraction of the +1s
>  - people feel bad about voting -1 and they shouldn't -- it's ok if done
> in a timely manner
>
> As an experiment for this vote (we'll see how it goes), I'd like to
> enforce a strict 72 hour period.  If you plan on voting, get your vote in
> by Friday 10pm PDT.
>
>
> Here's the thinking....
>
>
> FIRST VOTE
>
> You should feel welcome and encouraged to vote -1 in this time.  You
> aren't raining on anyone's parade :)  In fact, you are contributing to
> quality.  It looks negative "-1", but it is actually a positive
> contribution.  Please use the "-1" for clarity.
>
>
> SECOND VOTE
>
> At the end of this 72 hour period we will roll a second set of binaries
> that incorporate all the feedback of the first.
>
>  - Everyone who voted in the first vote will be expected to vote again.
>  - The vote will close when either:
>     - everyone from the first vote has vote again
>     - 72 hours
>
> The vote can close early if everyone reaffirms their +1s or that their
> issue (-1) was addressed, so please reaffirm your vote.
>
> This can be a very short vote if everyone cooperates.  So 5 minutes or 72
> hours, it's up to us :)
>
> Subsequent votes, if needed, will follow these rules.
>
>
> MAKING UP FOR BEING LATE
>
> If you did not vote at all in the first vote and feel the need to vote -1
> in a subsequent vote, your feedback is still welcome but understand that
> your feedback is also very late and you have cost everyone who did take the
> time to vote at least an hour of their time (more if there have already
> been a couple votes).  Do not do so lightly.
>
> Here are some great ways you can offset the cost:
>
>  - Immediately fix the issue you point out
>  - Give each individual who voted an hour of service, per vote, on a task
> of their choosing.
>
> That would probably cover it nicely :)
>
>
> POSSIBLE VARIATIONS
>
> We'll see how it goes, but if the basic concept works out, I can see maybe
> extending the first vote period to say 4 days (96 hours) instead of 3 (72
> hours).
>
>
> Anyway, we'll try it out and see if it has a positive impact on the flow
> of releases.
>
>
> -David
>
>