You are viewing a plain text version of this content. The canonical link for it is here.
Posted to general@incubator.apache.org by Christopher <ct...@apache.org> on 2017/09/01 01:29:58 UTC

Re: Digests in releases

On Wed, Aug 30, 2017 at 5:08 PM Julian Hyde <jh...@apache.org> wrote:

> What is the correct forum for discussing release distribution policy?
>
>
Good question. I hope it's this one, since this is where the discussion is
happening.



> Current policy [1] states:
>
>   Every artifact distributed to the public through Apache channels MUST
>   be accompanied by one file containing an OpenPGP compatible ASCII
>   armored detached signature and another file containing an MD5 checksum.
>
>   ...
>
>   An SHA checksum SHOULD also be created.
>
>
> MD5 is no longer deemed secure[2]. I think we should remove it from
> our releases and mandate SHA256 or SHA512.
>
>

A good policy is simple and flexible, in my opinion. Rather than require
any specific checksum with others optional, I would personally like to see
the policy changed to simply require "a detached ASCII-armored GPG
signature named <file>.asc for each <file> release artifact, and one or
more standard checksums with a minimum strength of MD5 in a standard file
format with a convenient file naming convention"

Such a policy could easily be updated by simply changing the "minimum
strength", if necessary, in the future. But changing the policy to allow
PMCs to drop support for legacy hashes, replaced by newer standards, is
infinitely better, in my opinion.

If my wording needs clarification:

  By "standard checksums", I mean MD5, SHA1, or any of the SHA2 family
currently, but maybe SHA3 family in future.
  By "standard file format", I mean a plain text file containing only the
ASCII encoded hex representation of the hash or in a format such as those
output by the 'sha*sum' suite of tools (example:
https://www.systutorials.com/docs/linux/man/1-sha512sum/).
  By "convenient file naming convention", I mean the artifact file name
with an appended ".md5 or .sha\d*" or aggregated into a file such as
CHECKSUMS, SHA1SUM, MD5SUM, etc. for the convenience of verifying multiple
artifact files from a release.

Modifying the policy in this way, we can eliminate requirements for legacy
hashes and inconvenient (as determined by PMCs for their users, not by
INFRA) file naming conventions. Of course, the file naming conventions
would still have to fit into constraints imposed by INFRA for the mirroring
system, or Maven deployments, or whatever, but these would simply be
implementation details, rather than enshrined in policy (which I think is
better, because policy should be simple and shouldn't change as much; INFRA
should be able to update implementation details, upon request, to allow
more conventions as new projects come on board with their own conventions,
and as verification tools and de facto standards around the internet
evolve).



> Julian
>
> [1] http://www.apache.org/dev/release-distribution.html#sigs-and-sums
>
> [2] https://en.wikipedia.org/wiki/Md5sum
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> For additional commands, e-mail: general-help@incubator.apache.org
>
>

Re: Digests in releases

Posted by Christopher <ct...@apache.org>.
My proposal was an attempt to achieve two goals:

1. Drop the absolute MD5 requirement in favor of an "at least MD5"
requirement, and
2. Make [2] consistent with [3] by dropping file naming conventions in [2],
and deferring to the constraints in [3].

If my existing wording is too complex, I'm sure it could be rewritten to
achieve both of these goals with simpler wording. If considering both goals
at the same time is too much to address at once, we could just focus on #1,
since that's more relevant to the topic this thread started with.

On Sat, Sep 2, 2017 at 6:00 AM Henk P. Penning <pe...@uu.nl> wrote:

> On Fri, 1 Sep 2017, Christopher wrote:
>
> > Date: Fri, 1 Sep 2017 03:29:58 +0200
> > From: Christopher <ct...@apache.org>
> > To: general@incubator.apache.org
> > Subject: Re: Digests in releases
> >
> > On Wed, Aug 30, 2017 at 5:08 PM Julian Hyde <jh...@apache.org> wrote:
> >
> >> What is the correct forum for discussing release distribution policy?
> >>
> >>
> > Good question. I hope it's this one, since this is where the discussion
> is
> > happening.
> >
> >
> >
> >> Current policy [1] states:
> >>
> >>   Every artifact distributed to the public through Apache channels MUST
> >>   be accompanied by one file containing an OpenPGP compatible ASCII
> >>   armored detached signature and another file containing an MD5
> checksum.
> >>
> >>   ...
> >>
> >>   An SHA checksum SHOULD also be created.
> >>
> >>
> >> MD5 is no longer deemed secure[2]. I think we should remove it from
> >> our releases and mandate SHA256 or SHA512.
>
> >> [1] http://www.apache.org/dev/release-distribution.html#sigs-and-sums
> >> [2] https://en.wikipedia.org/wiki/Md5sum
>
> >
> > A good policy is simple and flexible, in my opinion. Rather than require
> > any specific checksum with others optional, I would personally like to
> see
> > the policy changed to simply require "a detached ASCII-armored GPG
> > signature named <file>.asc for each <file> release artifact, and one or
> > more standard checksums with a minimum strength of MD5 in a standard file
> > format with a convenient file naming convention"
> >
> > Such a policy could easily be updated by simply changing the "minimum
> > strength", if necessary, in the future. But changing the policy to allow
> > PMCs to drop support for legacy hashes, replaced by newer standards, is
> > infinitely better, in my opinion.
>
>    The '(legal) release policy' [1] says /what/ we want, and /why/.
>    -- note that [1] doesn't even mention checksums.
>    The 'Release Distribution Policy' [2] says /how/ we do it.
>    -- because [2] is about 'implementation' it should preferably be :
>       -- simple, strict, unambiguous and short
>       -- easy to explain and easy to understand
>       -- ASF-wide
>
>    I think the current scheme has the prefered properties.
>
>    [1] http://www.apache.org/legal/release-policy.html
>    [2] http://www.apache.org/dev/release-distribution
>    [3] http://www.apache.org/dev/release-publishing
>
> > If my wording needs clarification:
> >
> >  By "standard checksums", I mean MD5, SHA1, or any of the SHA2 family
> > currently, but maybe SHA3 family in future.
> >  By "standard file format", I mean a plain text file containing only the
> > ASCII encoded hex representation of the hash or in a format such as those
> > output by the 'sha*sum' suite of tools (example:
> > https://www.systutorials.com/docs/linux/man/1-sha512sum/).
> >  By "convenient file naming convention", I mean the artifact file name
> > with an appended ".md5 or .sha\d*" or aggregated into a file such as
> > CHECKSUMS, SHA1SUM, MD5SUM, etc. for the convenience of verifying
> multiple
> > artifact files from a release.
>
>    If I understand you correctly, the operational word here is "or"
>    in "or aggregated into a file [...]".
>
>    -- The scheme you propose is more complex than the current scheme ;
>       therefor, a priori, it isn't better.
>    -- In your scheme, the checksum of a given file can be in multiple
>       places ; this error-prone, harder to use, check and report.
>    -- Note that [3] /allows/ you to supply 'MD5SUM' and 'SHA*SUM' files,
>       but [3] also says that in general checksum files shouldn't leak
>       to the mirrors, so a list of excluded files is provided ;
>       a short list is better than a long list.
>       [the list appears to be incomplete, which proves my point ;-]
>    -- You talk about a "standard file format" [for checksum files] ;
>       note that the current scheme does /not/ specify anything
>       about the contents of a checksum file ; yet it works (*).
>       Look at the checksum files in /dist/ ; they vary wildly.
>       Prescribing the format of checksum files is a bad idea.
>
>    (*) The checker finds some bad checksum files in /dist/ :
>        https://checker.apache.org/dist/sums.html
>        AFAIK it doesn't report any false negatives or positives ;
>        please send comments etc to me (henkp@apache.org).
>
>    Regards,
>
>    Henk Penning
>
> ------------------------------------------------------------   _
> Henk P. Penning, ICT-beta                 R Uithof HFG-406   _/ \_
> Faculty of Science, Utrecht University    T +31 30 253 4106
> <+31%2030%20253%204106> / \_/ \
> Budapestlaan 6, 3584CD Utrecht, NL        F +31 30 253 4553
> <+31%2030%20253%204553> \_/ \_/
> http://www.staff.science.uu.nl/~penni101/ M penning@uu.nl     \_/
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> For additional commands, e-mail: general-help@incubator.apache.org
>
>

Re: Digests in releases

Posted by "Henk P. Penning" <pe...@uu.nl>.
On Fri, 1 Sep 2017, Christopher wrote:

> Date: Fri, 1 Sep 2017 03:29:58 +0200
> From: Christopher <ct...@apache.org>
> To: general@incubator.apache.org
> Subject: Re: Digests in releases
> 
> On Wed, Aug 30, 2017 at 5:08 PM Julian Hyde <jh...@apache.org> wrote:
>
>> What is the correct forum for discussing release distribution policy?
>>
>>
> Good question. I hope it's this one, since this is where the discussion is
> happening.
>
>
>
>> Current policy [1] states:
>>
>>   Every artifact distributed to the public through Apache channels MUST
>>   be accompanied by one file containing an OpenPGP compatible ASCII
>>   armored detached signature and another file containing an MD5 checksum.
>>
>>   ...
>>
>>   An SHA checksum SHOULD also be created.
>>
>>
>> MD5 is no longer deemed secure[2]. I think we should remove it from
>> our releases and mandate SHA256 or SHA512.

>> [1] http://www.apache.org/dev/release-distribution.html#sigs-and-sums
>> [2] https://en.wikipedia.org/wiki/Md5sum

>
> A good policy is simple and flexible, in my opinion. Rather than require
> any specific checksum with others optional, I would personally like to see
> the policy changed to simply require "a detached ASCII-armored GPG
> signature named <file>.asc for each <file> release artifact, and one or
> more standard checksums with a minimum strength of MD5 in a standard file
> format with a convenient file naming convention"
>
> Such a policy could easily be updated by simply changing the "minimum
> strength", if necessary, in the future. But changing the policy to allow
> PMCs to drop support for legacy hashes, replaced by newer standards, is
> infinitely better, in my opinion.

   The '(legal) release policy' [1] says /what/ we want, and /why/.
   -- note that [1] doesn't even mention checksums.
   The 'Release Distribution Policy' [2] says /how/ we do it.
   -- because [2] is about 'implementation' it should preferably be :
      -- simple, strict, unambiguous and short
      -- easy to explain and easy to understand
      -- ASF-wide

   I think the current scheme has the prefered properties.

   [1] http://www.apache.org/legal/release-policy.html
   [2] http://www.apache.org/dev/release-distribution
   [3] http://www.apache.org/dev/release-publishing

> If my wording needs clarification:
>
>  By "standard checksums", I mean MD5, SHA1, or any of the SHA2 family
> currently, but maybe SHA3 family in future.
>  By "standard file format", I mean a plain text file containing only the
> ASCII encoded hex representation of the hash or in a format such as those
> output by the 'sha*sum' suite of tools (example:
> https://www.systutorials.com/docs/linux/man/1-sha512sum/).
>  By "convenient file naming convention", I mean the artifact file name
> with an appended ".md5 or .sha\d*" or aggregated into a file such as
> CHECKSUMS, SHA1SUM, MD5SUM, etc. for the convenience of verifying multiple
> artifact files from a release.

   If I understand you correctly, the operational word here is "or"
   in "or aggregated into a file [...]".

   -- The scheme you propose is more complex than the current scheme ;
      therefor, a priori, it isn't better.
   -- In your scheme, the checksum of a given file can be in multiple
      places ; this error-prone, harder to use, check and report.
   -- Note that [3] /allows/ you to supply 'MD5SUM' and 'SHA*SUM' files,
      but [3] also says that in general checksum files shouldn't leak
      to the mirrors, so a list of excluded files is provided ;
      a short list is better than a long list.
      [the list appears to be incomplete, which proves my point ;-]
   -- You talk about a "standard file format" [for checksum files] ;
      note that the current scheme does /not/ specify anything
      about the contents of a checksum file ; yet it works (*).
      Look at the checksum files in /dist/ ; they vary wildly.
      Prescribing the format of checksum files is a bad idea.

   (*) The checker finds some bad checksum files in /dist/ :
       https://checker.apache.org/dist/sums.html
       AFAIK it doesn't report any false negatives or positives ;
       please send comments etc to me (henkp@apache.org).

   Regards,

   Henk Penning

------------------------------------------------------------   _
Henk P. Penning, ICT-beta                 R Uithof HFG-406   _/ \_
Faculty of Science, Utrecht University    T +31 30 253 4106 / \_/ \
Budapestlaan 6, 3584CD Utrecht, NL        F +31 30 253 4553 \_/ \_/
http://www.staff.science.uu.nl/~penni101/ M penning@uu.nl     \_/

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org