Automated mirrors and checksum usage for downloads in machine readable format

[Anthony Bryan <an...@gmail.com>]

Hi,

Have you looked into using Metalinks for your downloads? It's an XML
format used by download apps, and contains the ways to get a file
(mirrors/P2P) along with info for automatic error detection/recovery
(checksums) and other metadata.

It can simplify the download process for the user, since they don't
have to manually try a bunch of servers that could be down, can
automatically use local mirrors first, and can repair downloads.

About 15 download managers & P2P apps support it so far, including
aria2, DownThemAll! (Firefox extension), KGet2 (part of KDE4), and
popular DMs on Windows and OS X like GetRight, Free Download Manager,
Orbit, wxDownload Fast, Speed Download, and TheWorld web browser. Over
20 other distros use Metalink, along with OpenOffice.org, cURL, and
LugRadio.

Here's what a metalink file (httpd-2.2.6.tar.gz.metalink) looks like:

  <file name="httpd-2.2.6.tar.gz">
   <verification>
    <hash type="md5">d050a49bd7532ec21c6bb593b3473a5d</hash>
   </verification>
   <resources>
     <url type="http"
          location="us"
          preference="90">
      http://apache.oregonstate.edu/httpd/httpd-2.2.6.tar.gz
     </url>
     <url type="http"
          location="us"
          preference="100">
      http://www.axint.net/apache/httpd/httpd-2.2.6.tar.gz
     </url>
   </resources>
  </file>

(And so on for all 50+ mirrors).

I've filed an issue on it: https://issues.apache.org/jira/browse/INFRA-1387

Reading "Verify the integrity of the files" at the bottom of
http://httpd.apache.org/download.cgi , shows the importance of
integrity verification, and Metalink makes this automatic. (You can
also include PGP signatures).

If you're interested in implementing this, the Metalink project would
like to help.

thanks,
-- 
(( Anthony Bryan ... Metalink [ http://www.metalinker.org ]
  )) Easier, More Reliable, Self Healing Downloads