You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@whimsical.apache.org by Daniel Shahaf <d....@daniel.shahaf.name> on 2019/03/13 15:13:17 UTC
Re: Link to KEYS file on our download page
[ removing dev@svn, adding dev@whimsy — sorry if that's not the right list. ]
Sebb,
KEYS formats have multiple problems. They are a custom format that
needs to be updated by hand, and if someone is a committer on N projects
then he needs to do O(N) work to update their keys. That's a very poor design
from a human factors point of view.
p.a.o/keys/ was invented to solve these problems. If it has downsides,
fine, but don't throw the baby out with the bathwater.
It would be better to change the p.a.o/keys/ cron job (1) to work, not
off of LDAP but off of an append-only dataset mapping PMCs to committers
and committers to keys; and (2) instead of publishing the results on
minotaur, to auto-commit them to /repos/dist/$PMC/release/KEYS.
To be clear, what I mean is something along the lines of:
[[[
# runnable prototype
HEAD=`svn info --show-item=revision $URL/to/pmcs.txt`
{ svn cat $URL/to/pmcs.txt@${HEAD} && ldapsearch -xLLL -b ou=pmc,ou=committees,ou=groups,dc=apache,dc=org cn | sed -ne 's/^cn: //p'; } | sort | uniq | svnmucc -m "Automatically regenerated by $0" -r "${HEAD}" put /dev/stdin $URL/to/pmcs.txt
]]]
Do that, do the same for the set of committers of each PMC, make the
site generation scripts use this as the input instead of LDAP, make them
commit /keys/group/foo.pmc to /dist/release/foo/KEYS, profit.
Daniel
P.S. The output of that ldapsearch is sorted by length first and
alphabetically second. Odd…
Julian Foad wrote on Tue, 12 Mar 2019 16:34 +00:00:
> Hello, devs. Apparently our handling of the KEYS file needs to change:
> see below.
>
> Is anybody willing to handle this?
>
> ----- Original message -----
> From: announce-owner@apache.org
> Subject: Returned post for announce@apache.org
>
> Sorry, but the download page is missing a required link to the KEYS file.
> The URL must be of the form: https://www.apache.org/dist/<project>/KEYS
>
> Links to http://people.apache.org/ are not acceptable; see the note here:
> https://people.apache.org/keys/
>
> Please resubmit the announce email when this has been corrected.
> Thanks,
>
> Sebb
>
> Attachments:
> * Email.eml
Re: Link to KEYS file on our download page
Posted by sebb <se...@gmail.com>.
On Wed, 13 Mar 2019 at 15:42, sebb <se...@gmail.com> wrote:
>
> On Wed, 13 Mar 2019 at 15:15, Daniel Shahaf <d....@daniel.shahaf.name> wrote:
> >
> > [ removing dev@svn, adding dev@whimsy — sorry if that's not the right list. ]
> >
> > Sebb,
> >
> > KEYS formats have multiple problems. They are a custom format that
> > needs to be updated by hand, and if someone is a committer on N projects
> > then he needs to do O(N) work to update their keys. That's a very poor design
> > from a human factors point of view.
>
> They only need to add their key on projects where they have signed a release.
>
> That is a much smaller number.
>
> > p.a.o/keys/ was invented to solve these problems. If it has downsides,
> > fine, but don't throw the baby out with the bathwater.
> >
> > It would be better to change the p.a.o/keys/ cron job (1) to work, not
> > off of LDAP but off of an append-only dataset mapping PMCs to committers
> > and committers to keys; and (2) instead of publishing the results on
> > minotaur, to auto-commit them to /repos/dist/$PMC/release/KEYS.
>
> I think that will cause problems.
>
> KEYS are used for archives validation as well.
> Entries should not be deleted if they have ever been used to sign a release.
>
> KEYS will grow and grow.
> That is not very friendly to downloaders who want to use the KEYS file.
>
> Also I am wary about allowing a bot to update the KEYS file.
> The provenance of the data is not nearly so easy to establish.
Also, not all projects have a KEYS file in the canonical location (
/repos/dist/$PMC/release/KEYS)
And some projects maintain the KEYS file elsewhere and copy it to the
canonical location.
However, if there really is an issue with the work needed to update
the KEYS files, then maybe there could be a tool to make it easier for
a release signer to add their key to whatever KEYS file is maintained
by their project.
> Sebb.
Re: Link to KEYS file on our download page
Posted by Julian Foad <ju...@apache.org>.
Please let's encourage anyone who wants to help, to automate this.
Managing keys distribution manually is grunt work, not something I want to be doing myself, not something I want other ASF committers to be doing. As a contributing factor to the development of our open source software, it's a negative. It lends itself well to being managed consistently across all projects [1]. Speaking for myself, I want this automated. Even a crude automation would be better than none.
Can we move forward by accepting some automation as better than none?
- Julian
[1] It doesn't have to be mandatory.
Re: Link to KEYS file on our download page
Posted by sebb <se...@gmail.com>.
On Wed, 13 Mar 2019 at 17:26, Daniel Shahaf <da...@apache.org> wrote:
>
> On Wed, Mar 13, 2019 at 03:42:13PM +0000, sebb wrote:
> > On Wed, 13 Mar 2019 at 15:15, Daniel Shahaf <d....@daniel.shahaf.name> wrote:
> > >
> > > [ removing dev@svn, adding dev@whimsy — sorry if that's not the right list. ]
> > >
> > > Sebb,
> > >
> > > KEYS formats have multiple problems. They are a custom format that
> > > needs to be updated by hand, and if someone is a committer on N projects
> > > then he needs to do O(N) work to update their keys. That's a very poor design
> > > from a human factors point of view.
> >
> > They only need to add their key on projects where they have signed a release.
> >
> > That is a much smaller number.
> >
>
> It's still O(N⋅M) work that you are inflicting on everyone, year on year,
where M is nearly the same as N, I think.
> whereas the solution I proposed is O(1) work, one time. Also, you require
> everyone to keep state:
>
> "Oh, new serf release. Let me sign it. Hmm, have I signed a serf release
> since I created my new annual signing subkey?…"
Not really.
It will be obvious if they need to add their key when the release vote
is performed.
AIUI there is a script to check that the KEYS file contains the keys
necessary to validate releases.
> > > p.a.o/keys/ was invented to solve these problems. If it has downsides,
> > > fine, but don't throw the baby out with the bathwater.
> > >
> > > It would be better to change the p.a.o/keys/ cron job (1) to work, not
> > > off of LDAP but off of an append-only dataset mapping PMCs to committers
> > > and committers to keys; and (2) instead of publishing the results on
> > > minotaur, to auto-commit them to /repos/dist/$PMC/release/KEYS.
> >
> > I think that will cause problems.
> >
> > KEYS are used for archives validation as well.
> > Entries should not be deleted if they have ever been used to sign a release.
>
> You haven't actually explained what problem you see here in context of my
> proposal. Did you see the words "append-only" in my proposal? Did you observe
> how my prototype code creates an append-only set of unix names? The same would
> be done with the lists of committers per PMC and keyids per committer. Thus
> the files would be append-only (not in the line-based, 'echo >>' sense but in
> the set-of-OpenPGP-packets sense).
> Also, it is not a law of nature that each project shall have exactly one KEYS
> file for all their releases, current and historical. It's simply a system that
> somebody wrote up fifteen years ago, and it has its bugs. Just to illustrate
> how arbitrary the current system is, why shouldn't there be a convention that
> foo.tar.gz be accompanied by foo.tar.gz.keys? That file would be created as
> a symlink to some file that contains all the right keys. Each PMC could have,
> say, one KEYS file per year, and each release's .keys symlink would target the
> appropriate KEYS file.
>
> > KEYS will grow and grow.
> No, they won't.
It will certainly grow at least once when all project ids are added.
> /keys/group/foo.asc and /dist/release/foo/KEYS contain the same data.
No, KEYS will be bigger because it needs to contain historical keys.
The corresponding committers might not even exist in LDAP.
Note also that RMs are not necessarily members of the PMC.
This increases the potential size of the KEYS file by a large amount
for some PMCs.
> > Also I am wary about allowing a bot to update the KEYS file.
> > The provenance of the data is not nearly so easy to establish.
>
> The PGP security model does not require public keys to be transported over an
> authenticated channel.
Yes, but that only works for keys which have been countersigned.
AFAIK the vast majority have not been.
> Daniel
Re: Link to KEYS file on our download page
Posted by Daniel Shahaf <da...@apache.org>.
On Wed, Mar 13, 2019 at 03:42:13PM +0000, sebb wrote:
> On Wed, 13 Mar 2019 at 15:15, Daniel Shahaf <d....@daniel.shahaf.name> wrote:
> >
> > [ removing dev@svn, adding dev@whimsy — sorry if that's not the right list. ]
> >
> > Sebb,
> >
> > KEYS formats have multiple problems. They are a custom format that
> > needs to be updated by hand, and if someone is a committer on N projects
> > then he needs to do O(N) work to update their keys. That's a very poor design
> > from a human factors point of view.
>
> They only need to add their key on projects where they have signed a release.
>
> That is a much smaller number.
>
It's still O(N⋅M) work that you are inflicting on everyone, year on year,
whereas the solution I proposed is O(1) work, one time. Also, you require
everyone to keep state:
"Oh, new serf release. Let me sign it. Hmm, have I signed a serf release
since I created my new annual signing subkey?…"
> > p.a.o/keys/ was invented to solve these problems. If it has downsides,
> > fine, but don't throw the baby out with the bathwater.
> >
> > It would be better to change the p.a.o/keys/ cron job (1) to work, not
> > off of LDAP but off of an append-only dataset mapping PMCs to committers
> > and committers to keys; and (2) instead of publishing the results on
> > minotaur, to auto-commit them to /repos/dist/$PMC/release/KEYS.
>
> I think that will cause problems.
>
> KEYS are used for archives validation as well.
> Entries should not be deleted if they have ever been used to sign a release.
You haven't actually explained what problem you see here in context of my
proposal. Did you see the words "append-only" in my proposal? Did you observe
how my prototype code creates an append-only set of unix names? The same would
be done with the lists of committers per PMC and keyids per committer. Thus
the files would be append-only (not in the line-based, 'echo >>' sense but in
the set-of-OpenPGP-packets sense).
Also, it is not a law of nature that each project shall have exactly one KEYS
file for all their releases, current and historical. It's simply a system that
somebody wrote up fifteen years ago, and it has its bugs. Just to illustrate
how arbitrary the current system is, why shouldn't there be a convention that
foo.tar.gz be accompanied by foo.tar.gz.keys? That file would be created as
a symlink to some file that contains all the right keys. Each PMC could have,
say, one KEYS file per year, and each release's .keys symlink would target the
appropriate KEYS file.
> KEYS will grow and grow.
No, they won't. /keys/group/foo.asc and /dist/release/foo/KEYS contain the same data.
> Also I am wary about allowing a bot to update the KEYS file.
> The provenance of the data is not nearly so easy to establish.
The PGP security model does not require public keys to be transported over an
authenticated channel.
Daniel
Re: Link to KEYS file on our download page
Posted by sebb <se...@gmail.com>.
On Wed, 13 Mar 2019 at 15:15, Daniel Shahaf <d....@daniel.shahaf.name> wrote:
>
> [ removing dev@svn, adding dev@whimsy — sorry if that's not the right list. ]
>
> Sebb,
>
> KEYS formats have multiple problems. They are a custom format that
> needs to be updated by hand, and if someone is a committer on N projects
> then he needs to do O(N) work to update their keys. That's a very poor design
> from a human factors point of view.
They only need to add their key on projects where they have signed a release.
That is a much smaller number.
> p.a.o/keys/ was invented to solve these problems. If it has downsides,
> fine, but don't throw the baby out with the bathwater.
>
> It would be better to change the p.a.o/keys/ cron job (1) to work, not
> off of LDAP but off of an append-only dataset mapping PMCs to committers
> and committers to keys; and (2) instead of publishing the results on
> minotaur, to auto-commit them to /repos/dist/$PMC/release/KEYS.
I think that will cause problems.
KEYS are used for archives validation as well.
Entries should not be deleted if they have ever been used to sign a release.
KEYS will grow and grow.
That is not very friendly to downloaders who want to use the KEYS file.
Also I am wary about allowing a bot to update the KEYS file.
The provenance of the data is not nearly so easy to establish.
Sebb.