You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@community.apache.org by Mike Lissner <ml...@michaeljaylissner.com> on 2017/01/26 18:20:04 UTC
Use of MD5 and SHA1 for download verification
I filed a bug about this already, but I've been directed to email here
instead. The bug I filed is:
https://issues.apache.org/jira/browse/INFRA-12626
Basically, on download pages we provide obsolete hashes for our downloads
(MD5 and SHA1). These are meant, as I understand it, to serve two purposes.
First, they allow you to make sure that your download succeeded. Second,
they allow you to ensure that your download wasn't tampered with.
For the first purpose: Great. They work. For the second purpose, however,
we need to move away from MD5 and SHA1 hashes, both of which can now be
attacked with relatively modest hardware.
Browsers are moving away from SHA1 at a very fast pace. See:
https://security.googleblog.com/2014/09/gradually-sunsetting-sha-1.html
And:
https://blog.mozilla.org/security/2014/09/23/phasing-out-certificates-with-sha-1-based-signature-algorithms/
I don't know who's responsible for this, but my bug was closed because it's
not the infrastructure team, and so I'm trying here.
I suggest we move to SHA2 hashes for all verification purposes.
Thanks,
Mike
Re: Use of MD5 and SHA1 for download verification
Posted by sebb <se...@gmail.com>.
Yes, hashes etc are not replicated to mirrors; I think this is partly
to encourage people to download them from the ASF hardware.
However a rogue mirror could still provide its own hashes.
But hashes from ASF hardware still only provide a basic download
check; they don't provide authentication, because anyone can create a
hash for any file.
Of course, if you sign a hash with a GPG key, that would allow the
user to authenticate the hash if they trust the key.
AIUI that's basically what sigs are.
On 26 January 2017 at 21:26, Owen O'Malley <om...@apache.org> wrote:
> Infra does filter filenames that match (*.sha256) from the mirror
> replication, so it is possible
> to use such names and have matching behavior:
>
> Compare mirror: http://apache.cs.utah.edu/orc/orc-1.2.3/
> Apache version: http://www-eu.apache.org/dist/orc/orc-1.2.3/
>
> and you can see the sha256 files are dropped just like the *.asc files.
>
> .. Owen
>
> On Thu, Jan 26, 2017 at 12:01 PM, Christopher <ct...@apache.org> wrote:
>
>> To be clear, those "trusted signatures" should be using strong hash
>> algorithms themselves. (As well as sufficiently long keys.)
>> I raised the issue of weak hashes in GPG signatures for Maven projects at
>> ASF with https://issues.apache.org/jira/browse/MPOM-118 , but non-Maven
>> projects which manually sign releases should probably take care to ensure
>> their signatures are adequate. I consider this a release-voting quality
>> assurance step, and encourage projects to examine the signatures attached
>> to their release candidates as part of their release process.
>>
>> On Thu, Jan 26, 2017 at 2:27 PM Ted Dunning <te...@gmail.com> wrote:
>>
>> > SHA1 and MD5 have been individually compromised, but a combined hash has
>> > not been.
>> >
>> > Regardless, Sebb's comment that hashes are worthless for authentication
>> and
>> > tamper-detection is spot-on. You have to look to trusted signatures for
>> > that.
>> >
>> >
>> >
>> > On Thu, Jan 26, 2017 at 10:20 AM, Mike Lissner <
>> > mlissner@michaeljaylissner.com> wrote:
>> >
>> > > I filed a bug about this already, but I've been directed to email here
>> > > instead. The bug I filed is:
>> > > https://issues.apache.org/jira/browse/INFRA-12626
>> > >
>> > > Basically, on download pages we provide obsolete hashes for our
>> downloads
>> > > (MD5 and SHA1). These are meant, as I understand it, to serve two
>> > purposes.
>> > > First, they allow you to make sure that your download succeeded.
>> Second,
>> > > they allow you to ensure that your download wasn't tampered with.
>> > >
>> > > For the first purpose: Great. They work. For the second purpose,
>> however,
>> > > we need to move away from MD5 and SHA1 hashes, both of which can now be
>> > > attacked with relatively modest hardware.
>> > >
>> > > Browsers are moving away from SHA1 at a very fast pace. See:
>> > >
>> > > https://security.googleblog.com/2014/09/gradually-
>> sunsetting-sha-1.html
>> > >
>> > > And:
>> > >
>> > > https://blog.mozilla.org/security/2014/09/23/phasing-
>> > > out-certificates-with-sha-1-based-signature-algorithms/
>> > >
>> > > I don't know who's responsible for this, but my bug was closed because
>> > it's
>> > > not the infrastructure team, and so I'm trying here.
>> > >
>> > > I suggest we move to SHA2 hashes for all verification purposes.
>> > >
>> > > Thanks,
>> > >
>> > > Mike
>> > >
>> >
>> --
>> Christopher
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@community.apache.org
For additional commands, e-mail: dev-help@community.apache.org
Re: Use of MD5 and SHA1 for download verification
Posted by Owen O'Malley <om...@apache.org>.
Infra does filter filenames that match (*.sha256) from the mirror
replication, so it is possible
to use such names and have matching behavior:
Compare mirror: http://apache.cs.utah.edu/orc/orc-1.2.3/
Apache version: http://www-eu.apache.org/dist/orc/orc-1.2.3/
and you can see the sha256 files are dropped just like the *.asc files.
.. Owen
On Thu, Jan 26, 2017 at 12:01 PM, Christopher <ct...@apache.org> wrote:
> To be clear, those "trusted signatures" should be using strong hash
> algorithms themselves. (As well as sufficiently long keys.)
> I raised the issue of weak hashes in GPG signatures for Maven projects at
> ASF with https://issues.apache.org/jira/browse/MPOM-118 , but non-Maven
> projects which manually sign releases should probably take care to ensure
> their signatures are adequate. I consider this a release-voting quality
> assurance step, and encourage projects to examine the signatures attached
> to their release candidates as part of their release process.
>
> On Thu, Jan 26, 2017 at 2:27 PM Ted Dunning <te...@gmail.com> wrote:
>
> > SHA1 and MD5 have been individually compromised, but a combined hash has
> > not been.
> >
> > Regardless, Sebb's comment that hashes are worthless for authentication
> and
> > tamper-detection is spot-on. You have to look to trusted signatures for
> > that.
> >
> >
> >
> > On Thu, Jan 26, 2017 at 10:20 AM, Mike Lissner <
> > mlissner@michaeljaylissner.com> wrote:
> >
> > > I filed a bug about this already, but I've been directed to email here
> > > instead. The bug I filed is:
> > > https://issues.apache.org/jira/browse/INFRA-12626
> > >
> > > Basically, on download pages we provide obsolete hashes for our
> downloads
> > > (MD5 and SHA1). These are meant, as I understand it, to serve two
> > purposes.
> > > First, they allow you to make sure that your download succeeded.
> Second,
> > > they allow you to ensure that your download wasn't tampered with.
> > >
> > > For the first purpose: Great. They work. For the second purpose,
> however,
> > > we need to move away from MD5 and SHA1 hashes, both of which can now be
> > > attacked with relatively modest hardware.
> > >
> > > Browsers are moving away from SHA1 at a very fast pace. See:
> > >
> > > https://security.googleblog.com/2014/09/gradually-
> sunsetting-sha-1.html
> > >
> > > And:
> > >
> > > https://blog.mozilla.org/security/2014/09/23/phasing-
> > > out-certificates-with-sha-1-based-signature-algorithms/
> > >
> > > I don't know who's responsible for this, but my bug was closed because
> > it's
> > > not the infrastructure team, and so I'm trying here.
> > >
> > > I suggest we move to SHA2 hashes for all verification purposes.
> > >
> > > Thanks,
> > >
> > > Mike
> > >
> >
> --
> Christopher
>
Re: Use of MD5 and SHA1 for download verification
Posted by Christopher <ct...@apache.org>.
To be clear, those "trusted signatures" should be using strong hash
algorithms themselves. (As well as sufficiently long keys.)
I raised the issue of weak hashes in GPG signatures for Maven projects at
ASF with https://issues.apache.org/jira/browse/MPOM-118 , but non-Maven
projects which manually sign releases should probably take care to ensure
their signatures are adequate. I consider this a release-voting quality
assurance step, and encourage projects to examine the signatures attached
to their release candidates as part of their release process.
On Thu, Jan 26, 2017 at 2:27 PM Ted Dunning <te...@gmail.com> wrote:
> SHA1 and MD5 have been individually compromised, but a combined hash has
> not been.
>
> Regardless, Sebb's comment that hashes are worthless for authentication and
> tamper-detection is spot-on. You have to look to trusted signatures for
> that.
>
>
>
> On Thu, Jan 26, 2017 at 10:20 AM, Mike Lissner <
> mlissner@michaeljaylissner.com> wrote:
>
> > I filed a bug about this already, but I've been directed to email here
> > instead. The bug I filed is:
> > https://issues.apache.org/jira/browse/INFRA-12626
> >
> > Basically, on download pages we provide obsolete hashes for our downloads
> > (MD5 and SHA1). These are meant, as I understand it, to serve two
> purposes.
> > First, they allow you to make sure that your download succeeded. Second,
> > they allow you to ensure that your download wasn't tampered with.
> >
> > For the first purpose: Great. They work. For the second purpose, however,
> > we need to move away from MD5 and SHA1 hashes, both of which can now be
> > attacked with relatively modest hardware.
> >
> > Browsers are moving away from SHA1 at a very fast pace. See:
> >
> > https://security.googleblog.com/2014/09/gradually-sunsetting-sha-1.html
> >
> > And:
> >
> > https://blog.mozilla.org/security/2014/09/23/phasing-
> > out-certificates-with-sha-1-based-signature-algorithms/
> >
> > I don't know who's responsible for this, but my bug was closed because
> it's
> > not the infrastructure team, and so I'm trying here.
> >
> > I suggest we move to SHA2 hashes for all verification purposes.
> >
> > Thanks,
> >
> > Mike
> >
>
--
Christopher
Re: Use of MD5 and SHA1 for download verification
Posted by Ted Dunning <te...@gmail.com>.
SHA1 and MD5 have been individually compromised, but a combined hash has
not been.
Regardless, Sebb's comment that hashes are worthless for authentication and
tamper-detection is spot-on. You have to look to trusted signatures for
that.
On Thu, Jan 26, 2017 at 10:20 AM, Mike Lissner <
mlissner@michaeljaylissner.com> wrote:
> I filed a bug about this already, but I've been directed to email here
> instead. The bug I filed is:
> https://issues.apache.org/jira/browse/INFRA-12626
>
> Basically, on download pages we provide obsolete hashes for our downloads
> (MD5 and SHA1). These are meant, as I understand it, to serve two purposes.
> First, they allow you to make sure that your download succeeded. Second,
> they allow you to ensure that your download wasn't tampered with.
>
> For the first purpose: Great. They work. For the second purpose, however,
> we need to move away from MD5 and SHA1 hashes, both of which can now be
> attacked with relatively modest hardware.
>
> Browsers are moving away from SHA1 at a very fast pace. See:
>
> https://security.googleblog.com/2014/09/gradually-sunsetting-sha-1.html
>
> And:
>
> https://blog.mozilla.org/security/2014/09/23/phasing-
> out-certificates-with-sha-1-based-signature-algorithms/
>
> I don't know who's responsible for this, but my bug was closed because it's
> not the infrastructure team, and so I'm trying here.
>
> I suggest we move to SHA2 hashes for all verification purposes.
>
> Thanks,
>
> Mike
>
Re: Use of MD5 and SHA1 for download verification
Posted by Bertrand Delacretaz <bd...@apache.org>.
On Fri, Jan 27, 2017 at 6:04 PM, Rich Bowen <rb...@rcbowen.com> wrote:
> ...If you feel strongly about it, go make better hashes
> for the project(s) you care about. Show us how it's done...
+1, a blog post explaining what projects need to change would be
useful, as a single URL that people can be pointed to to understand
the issues and how to improve the situation.
-Bertrand
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@community.apache.org
For additional commands, e-mail: dev-help@community.apache.org
Re: Use of MD5 and SHA1 for download verification
Posted by Rich Bowen <rb...@rcbowen.com>.
On 01/26/2017 01:20 PM, Mike Lissner wrote:
> I filed a bug about this already, but I've been directed to email here
> instead. The bug I filed is:
> https://issues.apache.org/jira/browse/INFRA-12626
>
> Basically, on download pages we provide obsolete hashes for our downloads
> (MD5 and SHA1). These are meant, as I understand it, to serve two purposes.
> First, they allow you to make sure that your download succeeded. Second,
> they allow you to ensure that your download wasn't tampered with.
>
> For the first purpose: Great. They work. For the second purpose, however,
> we need to move away from MD5 and SHA1 hashes, both of which can now be
> attacked with relatively modest hardware.
>
> Browsers are moving away from SHA1 at a very fast pace. See:
>
> https://security.googleblog.com/2014/09/gradually-sunsetting-sha-1.html
>
> And:
>
> https://blog.mozilla.org/security/2014/09/23/phasing-out-certificates-with-sha-1-based-signature-algorithms/
>
> I don't know who's responsible for this, but my bug was closed because it's
> not the infrastructure team, and so I'm trying here.
>
> I suggest we move to SHA2 hashes for all verification purposes.
So ... what other folks said, for sure, but, two points.
1) each project community does this kind of thing largely independently
2) I encourage you to get involved with the communities themselves, and
make this happen. If you feel strongly about it, go make better hashes
for the project(s) you care about. Show us how it's done. Lead by
example, and the projects will pick up your example and do it. I presume
that mechanically it's a very simple step to add to our release
processes, right?
--
Rich Bowen - rbowen@rcbowen.com - @rbowen
http://apachecon.com/ - @apachecon
Re: Use of MD5 and SHA1 for download verification
Posted by sebb <se...@gmail.com>.
On 26 January 2017 at 18:20, Mike Lissner
<ml...@michaeljaylissner.com> wrote:
> I filed a bug about this already, but I've been directed to email here
> instead. The bug I filed is:
> https://issues.apache.org/jira/browse/INFRA-12626
>
> Basically, on download pages we provide obsolete hashes for our downloads
> (MD5 and SHA1). These are meant, as I understand it, to serve two purposes.
> First, they allow you to make sure that your download succeeded.
Agreed
> Second, they allow you to ensure that your download wasn't tampered with.
They aren't intended for that purpose.
> For the first purpose: Great. They work. For the second purpose, however,
> we need to move away from MD5 and SHA1 hashes, both of which can now be
> attacked with relatively modest hardware.
>
> Browsers are moving away from SHA1 at a very fast pace. See:
>
> https://security.googleblog.com/2014/09/gradually-sunsetting-sha-1.html
>
> And:
>
> https://blog.mozilla.org/security/2014/09/23/phasing-out-certificates-with-sha-1-based-signature-algorithms/
Those hashes serve a different purpose.
> I don't know who's responsible for this, but my bug was closed because it's
> not the infrastructure team, and so I'm trying here.
>
> I suggest we move to SHA2 hashes for all verification purposes.
And how do you verify that the SHA2 hash has not been tampered with?
With PGP signatures one can check the WoT (web of trust).
And the PGP public keys are published in the KEYS files which are
derived from a source code repo to which only ASF committers have
write access.
I think it would be a mistake to give the impression that hashes are
of use for anything other than a sophisticated download checksum.
> Thanks,
>
> Mike
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@community.apache.org
For additional commands, e-mail: dev-help@community.apache.org