You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by Kevin Pilch-Bisson <ke...@pilch-bisson.net> on 2001/11/07 19:06:25 UTC
checksums on http transfers?
A while ago, when we were discussing checksums, we were thinking of adding
checksums to the repository, to ensure that we could accurately reconstruct
old versions. At the same time there was a little bit of discussion about
checksums on the network traffic as well.
IIRC it was decided that checksums weren't really important for network
traffic, since tcp was already checksummed. However, I have been thinking.
my recent working copy problems seem to have stemmed from the following:
My ISP performs NAT somewhere in its network, which requires re-caculating
the tcp checksum of packets as they pass through. It seems that the
problem is that they weren't _verifying_ the original checksum before
recalculating, therefore allowing corrupted packets to be received by
my machine.
In light of this, I thought I would bring this subject up again. I was
wondering does http have an md5sum header or something, and how would we
go about making mod_dav_svn generate and send this header, and have
neon verify it. gstein, joe, any pointers? I'd be willing to take on
this challenge, just to make sure I don't get any more wonky working copies
if my ISP's NAT goes flaky again.
--
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Kevin Pilch-Bisson http://www.pilch-bisson.net
"Historically speaking, the presences of wheels in Unix
has never precluded their reinvention." - Larry Wall
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Re: checksums on http transfers?
Posted by Joe Orton <jo...@manyfish.co.uk>.
On Wed, Nov 07, 2001 at 02:06:25PM -0500, Kevin Pilch-Bisson wrote:
...
> In light of this, I thought I would bring this subject up again. I was
> wondering does http have an md5sum header or something, and how would we
> go about making mod_dav_svn generate and send this header, and have
> neon verify it. gstein, joe, any pointers? I'd be willing to take on
> this challenge, just to make sure I don't get any more wonky working copies
> if my ISP's NAT goes flaky again.
Yup, this could be done with the Content-MD5 header (section 14.15 of
RFC 2616, just a base64 of the MD5 digest as you'd expect). Sending the
header and picking it up in ra_dav would be easy: getting the checksum
value is the problem. Choices I can see:
- the checksum is stored in the repository (or cheaply accessible from it)
- before sending the file, read it through once to precompute the checksum
- force the file to be sent using chunked transfer-encoding, calculate
the digest while sending it, and send the header in a chunked
response trailer
(the last option probably requires a PhD in Apache 2.0 filters ;)
joe
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: checksums on http transfers?
Posted by Daniel Stenberg <da...@haxx.se>.
On Thu, 8 Nov 2001, Kevin Pilch-Bisson wrote:
> Having just written a NAT server for vxworks, I know that RFC 0761 (TCP),
> states that the TCP checksum includes the tcp header, data, and a 96 bit
> pseudo header including source and dest addresses, 8 zero bits, the
> protocol number, and the length of the tcp data + tcp header.
I'll try to think more before I post.
Thanks for correcting my errors!
--
Daniel Stenberg - http://daniel.haxx.se - +46-705-44 31 77
ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: checksums on http transfers?
Posted by Kevin Pilch-Bisson <ke...@pilch-bisson.net>.
On Thu, Nov 08, 2001 at 08:26:21AM +0100, Daniel Stenberg wrote:
> On Wed, 7 Nov 2001, Kevin Pilch-Bisson wrote:
>
> > My ISP performs NAT somewhere in its network, which requires
> > re-caculating the tcp checksum of packets as they pass through. It seems
> > that the problem is that they weren't _verifying_ the original checksum
> > before recalculating, therefore allowing corrupted packets to be received
> > by my machine
>
> Just a minor detail, but NATs don't mess with TCP, they mess with IP. So they
> change the IP headers and that is separately checksummed (and very commonly
> ignored). Thus, the TCP checksum and data shall remain unmodified even after
> having passed through a NAT.
>
> IPv6 doesn't even have a checksum for the IP header, as both the underlying
> layer (mostly ethernet) and the upper layer (mostly TCP) usually already
> checksum the data.
>
> So, I don't think the NAT explains how your machine could accept broken TCP
> packets.
>
Having just written a NAT server for vxworks, I know that RFC 0761 (TCP),
states that the TCP checksum includes the tcp header, data, and a 96 bit
pseudo header including source and dest addresses, 8 zero bits, the protocol
number, and the length of the tcp data + tcp header. Thus anytime the
source or destination port or address is modified the tcp checksum _DOES_ need
to be recalculated. (Plus if any port changes are involved, the port number
is specified at the TCP level, not the IP level.)
--
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Kevin Pilch-Bisson http://www.pilch-bisson.net
"Historically speaking, the presences of wheels in Unix
has never precluded their reinvention." - Larry Wall
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Re: checksums on http transfers?
Posted by Daniel Stenberg <da...@haxx.se>.
On Wed, 7 Nov 2001, Kevin Pilch-Bisson wrote:
> My ISP performs NAT somewhere in its network, which requires
> re-caculating the tcp checksum of packets as they pass through. It seems
> that the problem is that they weren't _verifying_ the original checksum
> before recalculating, therefore allowing corrupted packets to be received
> by my machine
Just a minor detail, but NATs don't mess with TCP, they mess with IP. So they
change the IP headers and that is separately checksummed (and very commonly
ignored). Thus, the TCP checksum and data shall remain unmodified even after
having passed through a NAT.
IPv6 doesn't even have a checksum for the IP header, as both the underlying
layer (mostly ethernet) and the upper layer (mostly TCP) usually already
checksum the data.
So, I don't think the NAT explains how your machine could accept broken TCP
packets.
--
Daniel Stenberg - http://daniel.haxx.se - +46-705-44 31 77
ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org