Use of SHA, MD5 checksums on a release

[jan i <ja...@apache.org>]

Hi.

I just saw Daniel recommended we add checksums to our release. I admit it
is very common but I fail to understand the purpose.

We add a checksum file showing e.g. MD5 for the zip, to make sure the zip
is not manipulated....BUT

If someone can change the content of the zip in the location, what is
stopping them from
also generating a new MD5.

For a checksum to be effective (and likewise with the KEY) it needs to be
stored in a
different more safe place, so an offender would have to break 2 places.

Please help me understand where my argument is wrong ?

rgds
jan i.

RE: Use of SHA, MD5 checksums on a release

["Dennis E. Hamilton" <de...@acm.org>]

That is why the checksums and signatures do not go to the mirrors but stay in a location that Infra protects, along with the "official" release copy, ordinarily.  There is the threat that Jan mentions, but it is confined to a place where it should be detectable.  *Any* alteration after an approved RC was created and then after that was moved to the permanent release location should be detectable.  

 - Dennis

DETAILS

Remember, the checksums only accomplish one thing -- verification that transfer to a recipient was without damage.  It does not directly establish authenticity, except that it comes from a trusted location where presumably only authentic release hashes are found.  A download mirror could point to a different set of hash values, so there is also threat surface in the downstream delivery of release artifacts.

The signature being verified accomplishes both accuracy of delivery and provenance of the bits.  But not everyone is (1) in a position to check the signature (especially if they do not know how to find KEYS and also (2) how to satisfy themselves that the signature is to be relied upon.  There remains a downstream threat, so there needs to be a way to independently obtain the public key and agree on it being that of the release manager.

It is the case that the official location has hashes, signatures, and the release .zip (or .gz or whatever archive) in a single place that is read-only to the world.  The question is how to keep Infra and anyone else from noticing that a substitution has been made, especially since KEYS would also have to be altered and/or the private key used to sign the counterfeit have a compromised or even false committer key.  

PS: There is some circle-of-trust weirdness about Apache committer public keys too.  In that respect, neither of Jan's public keys appear to have been certified by anyone else, let alone anyone in the "Apache circle of trust" whoever that is.  However, the keys being found at <https://people.apache.org/keys/committer/jani.asc> is enough to satisfy me that I should trust that the private keys are in the possession of that Apache committer, so long as that account is not compromised and Jan is not careless with his private keys.  It always comes down to trusting.  
   This is one reason I have been saying the dots need to be connected to the release manager's Apache identifier and account, and ideally the key used will have the Apache User ID/email as one of its User IDs so someone could check on people.apache.org if they know enough to do that.  There are Apache seniors who also want to see a circle of trust, which means participating in key-signing parties among other Apache folk.
   A weakness of the KEYS file is that it is generally not by release and it can be stale. The key that was used may have been revoked subsequently.  It takes something for someone checking the signature to not only obtain the public key but to also acquire any certifications of it (including its revocation) and also check that it is the correct/current public key of the identified release manager.
   I suppose multiple committers could sign the release, but I haven't checked to see how well OpenPGP and utilities such as GnuPG handle that.

-----Original Message-----
From: jan i [mailto:jani@apache.org] 
Sent: Thursday, August 20, 2015 04:11
To: dev@corinthia.incubator.apache.org
Subject: Use of SHA, MD5 checksums on a release

Hi.

I just saw Daniel recommended we add checksums to our release. I admit it
is very common but I fail to understand the purpose.

We add a checksum file showing e.g. MD5 for the zip, to make sure the zip
is not manipulated....BUT

If someone can change the content of the zip in the location, what is
stopping them from
also generating a new MD5.

For a checksum to be effective (and likewise with the KEY) it needs to be
stored in a
different more safe place, so an offender would have to break 2 places.

Please help me understand where my argument is wrong ?

rgds
jan i.