You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by Garrett Rooney <ro...@electricjellyfish.net> on 2006/02/16 01:17:30 UTC
svndiff1 and compressBound
We're still having trouble building trunk with older versions of zlib
due to the lack of the compressBound function. Rumor has it there
were patches floating around to fix this, is that the case? It would
be awfully nice to be able to compile on Solaris 10 without having to
manually copy functions from newer versions of zlib into our code...
-garrett
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svndiff1 and compressBound
Posted by Daniel Berlin <db...@dberlin.org>.
On Sat, 2006-02-25 at 10:31 +0000, Max Bowsher wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Daniel Berlin wrote:
> > A configure test where we select either the zlib.h version or our
> > version would be silly, because then, in the remote possibility it ever
> > did change, your subversion would break depending on whether you had a
> > new enough zlib around it could pull compressBound from or not.
>
> Uhh, what?
>
> That's not right: If all new versions of zlib supply compressBound,
Well, since we are already arguing about something with a very remote
probability, I will just point out that zlib doesn't have a policy of
providing these API guarantees, so they might disappear on a whim, just
the same as compressBound changing.
> and
> we have a fallback which is correct for the versions which do not,
> then
> we should always be using the correct version.
> This is exactly the kind of situation autoconf is meant for.
>
I still claim it is a complete waste of time in this case, for the other
reasons I've posited.
There is a limit to the amount of defensiveness needed, and i think
anything more than noting where the macro comes from is overkill.
If you want to do it, go for it, i won't stop you :)
You could also simply make the number 13.5% of input size + 11 bytes,
and call it a day, because without changing the algorithm zlib uses in
an incompatible way, that is the absolute worst case you could generate
by toggling the parameters compressBound is using as constants.
See http://www.zlib.net/zlib_tech.html about deflateInit2
Unless you want to also try to protect against zlib making completely
incompatible changes to their format that is used widely and specified
by RFC, plus removing parts of their API, and ....
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svndiff1 and compressBound
Posted by Max Bowsher <ma...@ukf.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Daniel Berlin wrote:
> A configure test where we select either the zlib.h version or our
> version would be silly, because then, in the remote possibility it ever
> did change, your subversion would break depending on whether you had a
> new enough zlib around it could pull compressBound from or not.
Uhh, what?
That's not right: If all new versions of zlib supply compressBound, and
we have a fallback which is correct for the versions which do not, then
we should always be using the correct version.
This is exactly the kind of situation autoconf is meant for.
Max.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (Cygwin)
iD8DBQFEADH1fFNSmcDyxYARAvJnAJ9tMNcwhqzdbncqQzlIOUb7/oRaXACgkf7o
gCgw23dbZ57jl9pgnc2UnNU=
=N1Bh
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svndiff1 and compressBound
Posted by "Peter N. Lundblad" <pe...@famlundblad.se>.
On Thu, 23 Feb 2006, Daniel Berlin wrote:
>
> > Works for me. Add the #define.
>
> Can someone commit the following for me?
>
r18609.
//Peter
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svndiff1 and compressBound
Posted by Daniel Berlin <db...@dberlin.org>.
> Works for me. Add the #define.
Can someone commit the following for me?
I'm on a really crappy internet connection at a hotel for the next week,
and their proxy seems to be screwing me over.
yay!
You can also feel free to call the macro svn_compress_bound if we want
to be anal about it. :P
[[[
Add private copy of zlib's compressBound function to avoid missing
function issues on systems using zlib 1.1.x
* subversion/libsvn_delta/svndiff.c
(svnCompressBound): New macro
(zlib_encode): Use it
]]]
Re: svndiff1 and compressBound
Posted by Garrett Rooney <ro...@electricjellyfish.net>.
On 2/21/06, Daniel Berlin <db...@dberlin.org> wrote:
> Errr, not if you are keeping up to date on your redhat updates you
> didn't.
>
> -bash-3.00$ cat /etc/redhat-release
> Red Hat Enterprise Linux AS release 4 (Nahant Update 2)
> -bash-3.00$ grep compressBound /usr/include/zlib.h
> ...
> ZEXTERN uLong ZEXPORT compressBound OF((uLong sourceLen));
I am perfectly willing to stipulate that my RHEL install is out of date ;-)
> Anyhoo, i believe the correct solution is to simply paste the definition
> of compressBound into the file as a macro, note where it came from, and
> be done with it.
>
> compressBound is defined as:
>
> /* If the default memLevel or windowBits for deflateInit() is changed,
> then this function needs to be updated. */
> uLong ZEXPORT compressBound (sourceLen)
> uLong sourceLen;
> {
> return sourceLen + (sourceLen >> 12) + (sourceLen >> 14) + 11;
> }
>
>
>
> When i suggested this on IRC, there was some acrimony over the idea that
> this calculation may sometime change but
> 1. It hasn't changed since zlib has started
> 2. They have no plans to ever change it, AFAIK
> 3. The docs above it simply say that *if you change the window bits,
> etc, you need to change these numbers*. They didn't even make the
> numbers a define.
> 4. If it ever broke, you'd just get errors from zlib on compress about
> the output buffer not being large enough and we'd fix it rather quickly.
>
> " compress2 returns Z_OK if success, Z_MEM_ERROR if there was not enough
> memory, Z_BUF_ERROR if there was not enough room in the output buffer,
> Z_STREAM_ERROR if the level parameter is invalid."
>
> There is no solution to "not using compressBound" in some form, whether
> we inline it or not.
>
> A configure test where we select either the zlib.h version or our
> version would be silly, because then, in the remote possibility it ever
> did change, your subversion would break depending on whether you had a
> new enough zlib around it could pull compressBound from or not.
>
> If we inline it, and zlib changes the output buffer size requirement, it
> will at least fail consistently :)
Works for me. Add the #define.
-garrett
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svndiff1 and compressBound
Posted by Daniel Berlin <db...@dberlin.org>.
On Tue, 2006-02-21 at 08:20 -0800, Garrett Rooney wrote:
> On 2/20/06, Branko Čibej <br...@xbc.nu> wrote:
> > Garrett Rooney wrote:
> > > We're still having trouble building trunk with older versions of zlib
> > >
> > What's an "older version"? IIRC, anything before 1.2.3 (at least the
> > 1.1.x and 1.2.[012] line) has a security hole which 1.2.3 fixed.
>
> I'm not in front of the machines in question at the moment, but I saw
> the problem in whatever zlib versions are shipped with Redhat
> Enterprise 4 and Solaris 10.
>
Errr, not if you are keeping up to date on your redhat updates you
didn't.
-bash-3.00$ cat /etc/redhat-release
Red Hat Enterprise Linux AS release 4 (Nahant Update 2)
-bash-3.00$ grep compressBound /usr/include/zlib.h
...
ZEXTERN uLong ZEXPORT compressBound OF((uLong sourceLen));
-bash-3.00$ rpm -qi zlib-devel
Name : zlib-devel Relocations: /usr
Version : 1.2.1.2 Vendor: Red Hat, Inc.
Release : 1.2 Build Date: Tue 12 Jul 2005
03:06:22 PM GMT
Install Date: Wed 10 Aug 2005 02:41:04 PM GMT Build Host:
decompose.build.redhat.com
Group : Development/Libraries Source RPM:
zlib-1.2.1.2-1.2.src.rpm
Size : 218767 License: BSD
Signature : DSA/SHA1, Tue 12 Jul 2005 08:37:35 PM GMT, Key ID
219180cddb42a60e
Packager : Red Hat, Inc. <http://bugzilla.redhat.com/bugzilla>
URL : http://www.gzip.org/zlib/
Summary : Header files and libraries for Zlib development.
Description :
The zlib-devel package contains the header files and libraries needed
to develop programs that use the zlib compression and decompression
library.
Anyhoo, i believe the correct solution is to simply paste the definition
of compressBound into the file as a macro, note where it came from, and
be done with it.
compressBound is defined as:
/* If the default memLevel or windowBits for deflateInit() is changed,
then this function needs to be updated. */
uLong ZEXPORT compressBound (sourceLen)
uLong sourceLen;
{
return sourceLen + (sourceLen >> 12) + (sourceLen >> 14) + 11;
}
When i suggested this on IRC, there was some acrimony over the idea that
this calculation may sometime change but
1. It hasn't changed since zlib has started
2. They have no plans to ever change it, AFAIK
3. The docs above it simply say that *if you change the window bits,
etc, you need to change these numbers*. They didn't even make the
numbers a define.
4. If it ever broke, you'd just get errors from zlib on compress about
the output buffer not being large enough and we'd fix it rather quickly.
" compress2 returns Z_OK if success, Z_MEM_ERROR if there was not enough
memory, Z_BUF_ERROR if there was not enough room in the output buffer,
Z_STREAM_ERROR if the level parameter is invalid."
There is no solution to "not using compressBound" in some form, whether
we inline it or not.
A configure test where we select either the zlib.h version or our
version would be silly, because then, in the remote possibility it ever
did change, your subversion would break depending on whether you had a
new enough zlib around it could pull compressBound from or not.
If we inline it, and zlib changes the output buffer size requirement, it
will at least fail consistently :)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svndiff1 and compressBound
Posted by Branko Čibej <br...@xbc.nu>.
Garrett Rooney wrote:
> On 2/20/06, Branko Čibej <br...@xbc.nu> wrote:
>
>> Garrett Rooney wrote:
>>
>>> We're still having trouble building trunk with older versions of zlib
>>>
>>>
>> What's an "older version"? IIRC, anything before 1.2.3 (at least the
>> 1.1.x and 1.2.[012] line) has a security hole which 1.2.3 fixed.
>>
>
> I'm not in front of the machines in question at the moment, but I saw
> the problem in whatever zlib versions are shipped with Redhat
> Enterprise 4 and Solaris 10.
>
Ah, yes. At least RHEL4 nominally has a 1.1.x version of ZLib, with the
security patch back-ported.
-- Brane
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svndiff1 and compressBound
Posted by Garrett Rooney <ro...@electricjellyfish.net>.
On 2/20/06, Branko Čibej <br...@xbc.nu> wrote:
> Garrett Rooney wrote:
> > We're still having trouble building trunk with older versions of zlib
> >
> What's an "older version"? IIRC, anything before 1.2.3 (at least the
> 1.1.x and 1.2.[012] line) has a security hole which 1.2.3 fixed.
I'm not in front of the machines in question at the moment, but I saw
the problem in whatever zlib versions are shipped with Redhat
Enterprise 4 and Solaris 10.
-garrett
Re: svndiff1 and compressBound
Posted by Branko Čibej <br...@xbc.nu>.
Garrett Rooney wrote:
> We're still having trouble building trunk with older versions of zlib
>
What's an "older version"? IIRC, anything before 1.2.3 (at least the
1.1.x and 1.2.[012] line) has a security hole which 1.2.3 fixed.
-- Brane
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svndiff1 and compressBound
Posted by Justin Erenkrantz <ju...@erenkrantz.com>.
On 2/15/06, Garrett Rooney <ro...@electricjellyfish.net> wrote:
> We're still having trouble building trunk with older versions of zlib
> due to the lack of the compressBound function. Rumor has it there
> were patches floating around to fix this, is that the case? It would
> be awfully nice to be able to compile on Solaris 10 without having to
> manually copy functions from newer versions of zlib into our code...
IIRC, DannyB said he was just going to commit that compressBound
function into the tree.
*shrug* -- justin
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org