You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@subversion.apache.org by Adam Jensen <ha...@riseup.net> on 2018/09/22 13:39:46 UTC
Subversion 1.10.2 Checksum (SHA512)
From the recommended release page:
https://subversion.apache.org/download.cgi#recommended-release
The Checksum (SHA512) file:
https://www.apache.org/dist/subversion/subversion-1.10.2.tar.bz2.sha512
Contains:
ccbe860ec93a198745e40620cb7e005a85797e344a99ddbc0e24c32ad846976eae35cf5b3d62ba5751b998f0d40bbebbba72f484d92c92693bbb2112c989b129
In order to check the hash of the file using a command like this:
sha512sum --check subversion-1.10.2.tar.bz2.sha512
Shouldn't the format of that file (subversion-1.10.2.tar.bz2.sha512) be
something like this:
ccbe860ec93a198745e40620cb7e005a85797e344a99ddbc0e24c32ad846976eae35cf5b3d62ba5751b998f0d40bbebbba72f484d92c92693bbb2112c989b129
subversion-1.10.2.tar.bz2
That is, it should contain the hash AND the file name.
Given the current situation, can anyone think of a shell script blurb to
download the files then verify with the hash? Normally, I would do it
with something like this:
#!/usr/bin/env bash
FILE=subversion-1.10.2.tar.bz2
if [ ! -f $FILE ]; then
echo "Fetching $FILE"
wget http://www-eu.apache.org/dist/subversion/subversion-1.10.2.tar.bz2
fi
FILE=subversion-1.10.2.tar.bz2.sha512
if [ ! -f $FILE ]; then
echo "Fetching $FILE"
wget
https://www.apache.org/dist/subversion/subversion-1.10.2.tar.bz2.sha512
fi
sha512sum --check subversion-1.10.2.tar.bz2.sha512 || exit 1
# and the script continues...
Re: Subversion 1.10.2 Checksum (SHA512)
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Daniel Shahaf wrote on Sat, 22 Sep 2018 14:13 +0000:
> Adam Jensen wrote on Sat, 22 Sep 2018 09:39 -0400:
> > Given the current situation, can anyone think of a shell script blurb to
> > download the files then verify with the hash?
>
> wget .../subversion-1.10.2.tar.bz2
> wget
> https://www.apache.org/dist/subversion/subversion-1.10.2.tar.bz2.sha512
> sha512sum subversion-1.10.2.tar.bz2 | grep -x -- "$(cat < subversion-1.10.2.tar.bz2.sha512)"
Sorry, change -x to -w. (Or omit it altogether, really.)
ezt [version] and [supported] placeholders in download.html [was:
Strange download link on the web page]
Posted by Julian Foad <ju...@apache.org>.
Branko Čibej wrote:
> I also noticed that publish/download.html now has explicit version
> numbers in links instead of using the [version] and [supported] ezt
> placeholders that are defined at the top of that file. Apparently this
> changed during the 1.10 release cycle. Again ... why? The log doesn't say.
That's because that HTML code block is produced by 'release.py' which substitutes the parameters at that time. (At least that's how I did it for 1.10. Maybe previous RMs used another method?)
If we wanted the result to use the ezt placeholders, perhaps to make it more amenable to hand-editing, we could change 'release.py' to not substitute them. I don't see a problem with that. I recall we even discussed this at the time ... ah yes, here:
http://colabti.org/irclogger/irclogger_log/svn-dev?date=2018-07-20#l75
--
- Julian
Re: Strange download link on the web page [was: Re: Subversion 1.10.2
Checksum (SHA512)]
Posted by Branko Čibej <br...@apache.org>.
On 22.09.2018 16:29, Branko Čibej wrote:
> On 22.09.2018 16:22, Branko Čibej wrote:
>> On 22.09.2018 16:13, Daniel Shahaf wrote:
>>> Please don't download the artifacts from www*.apache.org but from a
>>> mirror. I think there is a redirector CGI somewhere that automatically
>>> redirects you to a mirror close to you, but I can't find it :(
>> http://subversion.apache.org/download.cgi
>>
>> Linked from our main page.
> [The original thread is on users@]
>
> I just noticed that when I click the 'Source Download' link in the
> navigation tab on our web page, I get:
>
> http://subversion.apache.org/download.cgi?update=201708081800
>
> instead of plain
>
> http://subversion.apache.org/download.cgi
>
> Can anyone remember why that is? It seems wrong, and also doesn't appear
> to do anything, since the page contents and especially download links
> appear to be the same in both cases.
>
> It was done in r1804690, the log message is:
>
> Release Subversion 1.9.7 with a fix for CVE-2017-9800.
>
> So it's possible that we forgot to clean that up after the security fix
> release ... and also that the ?update= parameter doesn't appear to work
> properly (any more).
I also noticed that publish/download.html now has explicit version
numbers in links instead of using the [version] and [supported] ezt
placeholders that are defined at the top of that file. Apparently this
changed during the 1.10 release cycle. Again ... why? The log doesn't say.
-- Brane
Re: Strange download link on the web page [was: Re: Subversion 1.10.2
Checksum (SHA512)]
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Branko Čibej wrote on Sat, 22 Sep 2018 16:59 +0200:
> On 22.09.2018 16:43, Daniel Shahaf wrote:
> > Yes, we can remove it now.. but, frankly, I'd rather keep it, so we
> > don't have to look up the syntax the next time we need it. It sounds like
> > we could add a comment, though.
>
> Good idea; r1841687.
Thanks. I've filed SVN-4777 about documenting in HACKING the need for the RM to
update/set this parameter in security releases.
Re: Strange download link on the web page [was: Re: Subversion 1.10.2
Checksum (SHA512)]
Posted by Branko Čibej <br...@apache.org>.
On 22.09.2018 16:43, Daniel Shahaf wrote:
> Branko Čibej wrote on Sat, 22 Sep 2018 16:29 +0200:
>> On 22.09.2018 16:22, Branko Čibej wrote:
>>> On 22.09.2018 16:13, Daniel Shahaf wrote:
>>>> Please don't download the artifacts from www*.apache.org but from a
>>>> mirror. I think there is a redirector CGI somewhere that automatically
>>>> redirects you to a mirror close to you, but I can't find it :(
>>> http://subversion.apache.org/download.cgi
>>>
>>> Linked from our main page.
>> [The original thread is on users@]
>>
>> I just noticed that when I click the 'Source Download' link in the
>> navigation tab on our web page, I get:
>>
>> http://subversion.apache.org/download.cgi?update=201708081800
>>
>> instead of plain
>>
>> http://subversion.apache.org/download.cgi
>>
>> Can anyone remember why that is? It seems wrong, and also doesn't appear
>> to do anything, since the page contents and especially download links
>> appear to be the same in both cases.
>>
>> It was done in r1804690, the log message is:
>>
>> Release Subversion 1.9.7 with a fix for CVE-2017-9800.
>>
>> So it's possible that we forgot to clean that up after the security fix
>> release ... and also that the ?update= parameter doesn't appear to work
>> properly (any more).
> The ?update= parameter is used to only offer mirrors that have synced
> after the specified YYMMDDhhmm date. We use it after a security release when
> the email announcement is less than 24 hours after the upload to /dist/release,
> in order to prevent offering mirrors that don't carry the just-released artifacts.
>
> The reason the parameter seems to have no effect is that the threshold
> date it sets is over a year ago, and all mirrors have update since then,
> so it excludes no mirrors from the list.
>
> Yes, we can remove it now.. but, frankly, I'd rather keep it, so we
> don't have to look up the syntax the next time we need it. It sounds like
> we could add a comment, though.
Good idea; r1841687.
-- Brane
Re: Strange download link on the web page [was: Re: Subversion 1.10.2
Checksum (SHA512)]
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Branko Čibej wrote on Sat, 22 Sep 2018 16:29 +0200:
> On 22.09.2018 16:22, Branko Čibej wrote:
> > On 22.09.2018 16:13, Daniel Shahaf wrote:
> >> Please don't download the artifacts from www*.apache.org but from a
> >> mirror. I think there is a redirector CGI somewhere that automatically
> >> redirects you to a mirror close to you, but I can't find it :(
> > http://subversion.apache.org/download.cgi
> >
> > Linked from our main page.
>
> [The original thread is on users@]
>
> I just noticed that when I click the 'Source Download' link in the
> navigation tab on our web page, I get:
>
> http://subversion.apache.org/download.cgi?update=201708081800
>
> instead of plain
>
> http://subversion.apache.org/download.cgi
>
> Can anyone remember why that is? It seems wrong, and also doesn't appear
> to do anything, since the page contents and especially download links
> appear to be the same in both cases.
>
> It was done in r1804690, the log message is:
>
> Release Subversion 1.9.7 with a fix for CVE-2017-9800.
>
> So it's possible that we forgot to clean that up after the security fix
> release ... and also that the ?update= parameter doesn't appear to work
> properly (any more).
The ?update= parameter is used to only offer mirrors that have synced
after the specified YYMMDDhhmm date. We use it after a security release when
the email announcement is less than 24 hours after the upload to /dist/release,
in order to prevent offering mirrors that don't carry the just-released artifacts.
The reason the parameter seems to have no effect is that the threshold
date it sets is over a year ago, and all mirrors have update since then,
so it excludes no mirrors from the list.
Yes, we can remove it now.. but, frankly, I'd rather keep it, so we
don't have to look up the syntax the next time we need it. It sounds like
we could add a comment, though.
Cheers,
Daniel
Strange download link on the web page [was: Re: Subversion 1.10.2
Checksum (SHA512)]
Posted by Branko Čibej <br...@apache.org>.
On 22.09.2018 16:22, Branko Čibej wrote:
> On 22.09.2018 16:13, Daniel Shahaf wrote:
>> Please don't download the artifacts from www*.apache.org but from a
>> mirror. I think there is a redirector CGI somewhere that automatically
>> redirects you to a mirror close to you, but I can't find it :(
> http://subversion.apache.org/download.cgi
>
> Linked from our main page.
[The original thread is on users@]
I just noticed that when I click the 'Source Download' link in the
navigation tab on our web page, I get:
http://subversion.apache.org/download.cgi?update=201708081800
instead of plain
http://subversion.apache.org/download.cgi
Can anyone remember why that is? It seems wrong, and also doesn't appear
to do anything, since the page contents and especially download links
appear to be the same in both cases.
It was done in r1804690, the log message is:
Release Subversion 1.9.7 with a fix for CVE-2017-9800.
So it's possible that we forgot to clean that up after the security fix
release ... and also that the ?update= parameter doesn't appear to work
properly (any more).
-- Brane
Re: Subversion 1.10.2 Checksum (SHA512)
Posted by Branko Čibej <br...@apache.org>.
On 22.09.2018 17:26, Daniel Shahaf wrote:
> Branko Čibej wrote on Sat, 22 Sep 2018 16:54 +0200:
>> Ah, that would be this one:
>>
>> https://www.apache.org/dyn/closer.cgi?path=subversion/subversion-1.10.2.tar.bz2
> That still gives me an HTML page, whether I pass the relpath as a query
> parameter (as you have) or as PATH_INFO.
Huh. That must be new? Because I distinctly remember using it to
download our tarballs from mirrors with curl.
Homebrew on Mac relies on URLs of this format to download all Apache
sources, too.
-- Brane
Re: Subversion 1.10.2 Checksum (SHA512)
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Branko Čibej wrote on Sat, 22 Sep 2018 16:54 +0200:
> Ah, that would be this one:
>
> https://www.apache.org/dyn/closer.cgi?path=subversion/subversion-1.10.2.tar.bz2
That still gives me an HTML page, whether I pass the relpath as a query
parameter (as you have) or as PATH_INFO.
Re: Subversion 1.10.2 Checksum (SHA512)
Posted by Branko Čibej <br...@apache.org>.
On 22.09.2018 16:26, Daniel Shahaf wrote:
> Branko Čibej wrote on Sat, 22 Sep 2018 16:22 +0200:
>> On 22.09.2018 16:13, Daniel Shahaf wrote:
>>> Please don't download the artifacts from www*.apache.org but from a
>>> mirror. I think there is a redirector CGI somewhere that automatically
>>> redirects you to a mirror close to you, but I can't find it :(
>> http://subversion.apache.org/download.cgi
>>
>> Linked from our main page.
> I meant, a CGI that just gives a 302 redirect to the mirror. Not one
> that generates an HTML page that a shell script would need to parse.
>
> The actual CGI lives here: https://svn.apache.org/repos/asf/infrastructure/site/trunk/content/dyn/
Ah, that would be this one:
https://www.apache.org/dyn/closer.cgi?path=subversion/subversion-1.10.2.tar.bz2
-- Brane
Re: Subversion 1.10.2 Checksum (SHA512)
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Branko Čibej wrote on Sat, 22 Sep 2018 16:22 +0200:
> On 22.09.2018 16:13, Daniel Shahaf wrote:
> > Please don't download the artifacts from www*.apache.org but from a
> > mirror. I think there is a redirector CGI somewhere that automatically
> > redirects you to a mirror close to you, but I can't find it :(
>
> http://subversion.apache.org/download.cgi
>
> Linked from our main page.
I meant, a CGI that just gives a 302 redirect to the mirror. Not one
that generates an HTML page that a shell script would need to parse.
The actual CGI lives here: https://svn.apache.org/repos/asf/infrastructure/site/trunk/content/dyn/
Cheers,
Daniel
Re: Subversion 1.10.2 Checksum (SHA512)
Posted by Branko Čibej <br...@apache.org>.
On 22.09.2018 16:13, Daniel Shahaf wrote:
> Please don't download the artifacts from www*.apache.org but from a
> mirror. I think there is a redirector CGI somewhere that automatically
> redirects you to a mirror close to you, but I can't find it :(
http://subversion.apache.org/download.cgi
Linked from our main page.
-- Brane
Re: Subversion 1.10.2 Checksum (SHA512)
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Daniel Shahaf wrote on Sat, 22 Sep 2018 15:28 +0000:
> Adam Jensen wrote on Sat, 22 Sep 2018 11:12 -0400:
> > On 09/22/2018 10:13 AM, Daniel Shahaf wrote:
> > > One reason for not creating the file in the form "${hash}${two_spaces}${filename}"
> > > is that not everyone uses sha512sum from GNU coreutils. If there were
> > > additional information in the file besides the hash value, people not on
> > > linux would have had to sed it out.
> >
> > apr and apr-util each do it differently. Consequently, the
> > idiosyncrasies of each will need to explored and handled separately.
This will work unmodified for quite a few forms of .sha512 files.
grep -w "$(sha512sum subversion-1.10.2.tar.bz2 | sed -e 's/ .*//')" subversion-1.10.2.tar.bz2.sha512
Re: Subversion 1.10.2 Checksum (SHA512)
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Adam Jensen wrote on Sat, 22 Sep 2018 11:12 -0400:
> On 09/22/2018 10:13 AM, Daniel Shahaf wrote:
> > One reason for not creating the file in the form "${hash}${two_spaces}${filename}"
> > is that not everyone uses sha512sum from GNU coreutils. If there were
> > additional information in the file besides the hash value, people not on
> > linux would have had to sed it out.
>
> apr and apr-util each do it differently. Consequently, the
> idiosyncrasies of each will need to explored and handled separately.
The PGP signatures are presented more uniformly, I think. (The only variable
is the location of the public keys reference file.)
Re: Subversion 1.10.2 Checksum (SHA512)
Posted by Adam Jensen <ha...@riseup.net>.
On 09/22/2018 10:13 AM, Daniel Shahaf wrote:
> One reason for not creating the file in the form "${hash}${two_spaces}${filename}"
> is that not everyone uses sha512sum from GNU coreutils. If there were
> additional information in the file besides the hash value, people not on
> linux would have had to sed it out.
apr and apr-util each do it differently. Consequently, the
idiosyncrasies of each will need to explored and handled separately.
Re: Subversion 1.10.2 Checksum (SHA512)
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Adam Jensen wrote on Sat, 22 Sep 2018 09:39 -0400:
> Given the current situation, can anyone think of a shell script blurb to
> download the files then verify with the hash?
wget .../subversion-1.10.2.tar.bz2
wget https://www.apache.org/dist/subversion/subversion-1.10.2.tar.bz2.sha512
sha512sum subversion-1.10.2.tar.bz2 | grep -x -- "$(cat < subversion-1.10.2.tar.bz2.sha512)"
> Normally, I would do it with something like this:
>
> #!/usr/bin/env bash
> FILE=subversion-1.10.2.tar.bz2
> if [ ! -f $FILE ]; then
> echo "Fetching $FILE"
> wget http://www-eu.apache.org/dist/subversion/subversion-1.10.2.tar.bz2
Please don't download the artifacts from www*.apache.org but from a
mirror. I think there is a redirector CGI somewhere that automatically
redirects you to a mirror close to you, but I can't find it :(
(The checksum files, in contrast, are intentionally available *only*
from www*.apache.org.)
> sha512sum --check subversion-1.10.2.tar.bz2.sha512 || exit 1
One reason for not creating the file in the form "${hash}${two_spaces}${filename}"
is that not everyone uses sha512sum from GNU coreutils. If there were
additional information in the file besides the hash value, people not on
linux would have had to sed it out.
Cheers,
Daniel
Re: Subversion 1.10.2 Checksum (SHA512)
Posted by Adam Jensen <ha...@riseup.net>.
The script was patched with a line like this:
echo " subversion-1.10.2.tar.bz2" >> subversion-1.10.2.tar.bz2.sha512
On 09/22/2018 09:39 AM, Adam Jensen wrote:
>>From the recommended release page:
> https://subversion.apache.org/download.cgi#recommended-release
>
> The Checksum (SHA512) file:
> https://www.apache.org/dist/subversion/subversion-1.10.2.tar.bz2.sha512
>
> Contains:
> ccbe860ec93a198745e40620cb7e005a85797e344a99ddbc0e24c32ad846976eae35cf5b3d62ba5751b998f0d40bbebbba72f484d92c92693bbb2112c989b129
>
> In order to check the hash of the file using a command like this:
> sha512sum --check subversion-1.10.2.tar.bz2.sha512
>
> Shouldn't the format of that file (subversion-1.10.2.tar.bz2.sha512) be
> something like this:
> ccbe860ec93a198745e40620cb7e005a85797e344a99ddbc0e24c32ad846976eae35cf5b3d62ba5751b998f0d40bbebbba72f484d92c92693bbb2112c989b129
> subversion-1.10.2.tar.bz2
>
> That is, it should contain the hash AND the file name.
>
> Given the current situation, can anyone think of a shell script blurb to
> download the files then verify with the hash? Normally, I would do it
> with something like this:
>
> #!/usr/bin/env bash
> FILE=subversion-1.10.2.tar.bz2
> if [ ! -f $FILE ]; then
> echo "Fetching $FILE"
> wget http://www-eu.apache.org/dist/subversion/subversion-1.10.2.tar.bz2
> fi
> FILE=subversion-1.10.2.tar.bz2.sha512
> if [ ! -f $FILE ]; then
> echo "Fetching $FILE"
> wget
> https://www.apache.org/dist/subversion/subversion-1.10.2.tar.bz2.sha512
> fi
> sha512sum --check subversion-1.10.2.tar.bz2.sha512 || exit 1
> # and the script continues...
>
>