You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by rb...@covalent.net on 2000/07/06 17:40:39 UTC
Encoding code in APR (Was: Re: hooks)
I thought of another reason why the md5/sha1 encoding code needs to be in
APR. It may be VERY portable code, but the results aren't portable. I
know that makes very little sense. Think about why we decided to always
use the Apache MD5 code on all platforms. We got the code originally from
FreeBSD, so we know they have the code, we could have used the crypt
function on FreeBSD.
If you review the archives, you'll discover that later versions of FreeBSD
modified their version of crypt to produce different results. They are a
bit more secure IIRC, but the results are also just not portable. This
means that if we don't put the encoding stuff into APR, and somebody uses
an MD5 algorithm on a FreeBSD box, that code will not be portable to a
Windows machine using APR with the lib/util MD5 algorithm.
By putting the MD5 algorithm in APR, we remove that problem, and can
provide a common interface that always implements the same alpgorithm
across all platforms.
With this argument and the argument laid out yesterday about not all
platforms having encoding API's, does anybody still have a problem with
creating lib/apr/encode, and putting ap_(md5/sha1/base64) there?
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
Re: Encoding code in APR
Posted by Greg Stein <gs...@lyra.org>.
On Fri, Jul 07, 2000 at 08:28:44PM -0700, rbb@covalent.net wrote:
>...
> > For the SHA1 and MD5 hashing stuff, I'd suggest putting them under
> > apr/crypto/ ("cryptographic algorithms"). That leaves the directory open to
> > other crypto stuff (i.e interfaces to SSL, TLS, or whatever).
>
> Either apr/crypto or apr/hash, either way. crypto is probably better,
> because it can't be confused with hash tables.
Right. That was my thought process, too.
Tonite, I also realized that the /dev/random and truerand stuff in APR
should go in there. Those are used only for true, "cryptographically safe"
random numbers. (they're too slow for basic random number gen)
I'll get that stuff moved at some point, as I'd like to use it for the UUID
generation on non-Win32 platforms.
Cheers,
-g
--
Greg Stein, http://www.lyra.org/
Re: Encoding code in APR
Posted by "Life is hard, and then you die" <ro...@innovation.ch>.
On Sun, Jul 09, 2000 at 07:38:57PM -0700, rbb@covalent.net wrote:
> On Sun, 9 Jul 2000, Greg Stein wrote:
> > On Sun, Jul 09, 2000 at 04:09:46PM -0700, Life is hard, and then you die wrote:
> > > On Sun, Jul 09, 2000 at 07:29:37AM -0700, rbb@covalent.net wrote:
> > >...
> > > > But apr_validate_password is a VERY useful function. I would personally
> > > > rather move ap_validate_password out of apr_md5, and put it in it's own
> > > > file. If it requires base64, then base64 goes into the crypt directory
> > > > too.
> >
> > Hmm. base64 isn't crypto :-) If/when base64 goes into APR, that is where
> > your "encode" directory would pop up. Or:
> >
> > encoder
> > encoding
> > encode
> >
> > Dunno. The problem is how to properly classify base64 because we don't have
> > any siblings for it (yet).
>
> Am I the only person who doesn't understand this. MD5 is used to encode
> passwords in our scheme, so is SHA1. The algorithms themselves, may not
> be meant for this, but this is what Apache uses these algorithms
> for.
No. As Greg already said, encodings/encoders are usually understood to
be reversible processes; hashes/digests are not reversible. So no,
Apache does not encode passwords, it hashes them - the password is lost
after the process. But this is nit-picking.
> Yes, we also take MD5 checksums at times. But, the reason APR needs
> MD5 is to encode passwords. This is also why we need to ap_checkpasswd
> function.
>
> Why can't we put all of the algorithms in the same directory, along with a
> simple function to check the passwords? This just makes sense to me, so
> please explain it to me.
Fine with me. But we should call not call it crypto (which none of this stuff
is), or encoders (which only base64 is). misc? utils? security? (though
base64 would be stretching that last term a bit).
Cheers,
Ronald
Re: Encoding code in APR
Posted by Greg Stein <gs...@lyra.org>.
On Tue, Jul 11, 2000 at 11:05:09PM +0100, Ben Laurie wrote:
> Greg Stein wrote:
> >
> > On Sun, Jul 09, 2000 at 04:09:46PM -0700, Life is hard, and then you die wrote:
> > > On Sun, Jul 09, 2000 at 07:29:37AM -0700, rbb@covalent.net wrote:
> > >...
> > > > But apr_validate_password is a VERY useful function. I would personally
> > > > rather move ap_validate_password out of apr_md5, and put it in it's own
> > > > file. If it requires base64, then base64 goes into the crypt directory
> > > > too.
> >
> > Hmm. base64 isn't crypto :-) If/when base64 goes into APR, that is where
> > your "encode" directory would pop up. Or:
> >
> > encoder
> > encoding
> > encode
> >
> > Dunno. The problem is how to properly classify base64 because we don't have
> > any siblings for it (yet).
>
> No encoding at all is a sibling, of course. Hex is another.
Eh? How does that help to classify where to place base64?
-g
--
Greg Stein, http://www.lyra.org/
Re: Encoding code in APR
Posted by Ben Laurie <be...@algroup.co.uk>.
Greg Stein wrote:
>
> On Sun, Jul 09, 2000 at 04:09:46PM -0700, Life is hard, and then you die wrote:
> > On Sun, Jul 09, 2000 at 07:29:37AM -0700, rbb@covalent.net wrote:
> >...
> > > But apr_validate_password is a VERY useful function. I would personally
> > > rather move ap_validate_password out of apr_md5, and put it in it's own
> > > file. If it requires base64, then base64 goes into the crypt directory
> > > too.
>
> Hmm. base64 isn't crypto :-) If/when base64 goes into APR, that is where
> your "encode" directory would pop up. Or:
>
> encoder
> encoding
> encode
>
> Dunno. The problem is how to properly classify base64 because we don't have
> any siblings for it (yet).
No encoding at all is a sibling, of course. Hex is another.
Cheers,
Ben.
--
http://www.apache-ssl.org/ben.html
Coming to ApacheCon Europe 2000? http://apachecon.com/
Re: Encoding code in APR
Posted by rb...@covalent.net.
On Sun, 9 Jul 2000, Greg Stein wrote:
> On Sun, Jul 09, 2000 at 04:09:46PM -0700, Life is hard, and then you die wrote:
> > On Sun, Jul 09, 2000 at 07:29:37AM -0700, rbb@covalent.net wrote:
> >...
> > > But apr_validate_password is a VERY useful function. I would personally
> > > rather move ap_validate_password out of apr_md5, and put it in it's own
> > > file. If it requires base64, then base64 goes into the crypt directory
> > > too.
>
> Hmm. base64 isn't crypto :-) If/when base64 goes into APR, that is where
> your "encode" directory would pop up. Or:
>
> encoder
> encoding
> encode
>
> Dunno. The problem is how to properly classify base64 because we don't have
> any siblings for it (yet).
Am I the only person who doesn't understand this. MD5 is used to encode
passwords in our scheme, so is SHA1. The algorithms themselves, may not
be meant for this, but this is what Apache uses these algorithms
for. Yes, we also take MD5 checksums at times. But, the reason APR needs
MD5 is to encode passwords. This is also why we need to ap_checkpasswd
function.
Why can't we put all of the algorithms in the same directory, along with a
simple function to check the passwords? This just makes sense to me, so
please explain it to me.
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
Re: Encoding code in APR
Posted by Greg Stein <gs...@lyra.org>.
On Sun, Jul 09, 2000 at 04:09:46PM -0700, Life is hard, and then you die wrote:
> On Sun, Jul 09, 2000 at 07:29:37AM -0700, rbb@covalent.net wrote:
>...
> > But apr_validate_password is a VERY useful function. I would personally
> > rather move ap_validate_password out of apr_md5, and put it in it's own
> > file. If it requires base64, then base64 goes into the crypt directory
> > too.
Hmm. base64 isn't crypto :-) If/when base64 goes into APR, that is where
your "encode" directory would pop up. Or:
encoder
encoding
encode
Dunno. The problem is how to properly classify base64 because we don't have
any siblings for it (yet).
Cheers,
-g
--
Greg Stein, http://www.lyra.org/
Re: Encoding code in APR
Posted by "Life is hard, and then you die" <ro...@innovation.ch>.
On Sun, Jul 09, 2000 at 07:29:37AM -0700, rbb@covalent.net wrote:
> On Sat, 8 Jul 2000, Life is hard, and then you die wrote:
>
> > On Sat, Jul 08, 2000 at 09:08:47AM -0700, rbb@covalent.net wrote:
> > >
> > > > ap_base64, ap_checkpass -> lib/utils/
> > >
> > > ap_base64 should be moved. ap_checkpass still bothers me. The function
> > > it implements is in APR already.
> >
> > But it's incomplete (it's missing Netscape's SHA-1 based password).
> > This means apr_validate_password has to move out of apr_md5. Also, the
> > Netscape password scheme needs base64 (in addition to SHA-1). So, if
> > apr_checkpass were to go into apr then ap_base64 would have to too.
>
> But apr_validate_password is a VERY useful function. I would personally
> rather move ap_validate_password out of apr_md5, and put it in it's own
> file. If it requires base64, then base64 goes into the crypt directory
> too.
Of course it's useful - it's part of apr_checkpass, i.e. it's in its own
file. So we're really in complete agreement here.
Cheers,
Ronald
Re: Encoding code in APR
Posted by rb...@covalent.net.
On Sat, 8 Jul 2000, Life is hard, and then you die wrote:
> On Sat, Jul 08, 2000 at 09:08:47AM -0700, rbb@covalent.net wrote:
> >
> > > ap_base64, ap_checkpass -> lib/utils/
> >
> > ap_base64 should be moved. ap_checkpass still bothers me. The function
> > it implements is in APR already.
>
> But it's incomplete (it's missing Netscape's SHA-1 based password).
> This means apr_validate_password has to move out of apr_md5. Also, the
> Netscape password scheme needs base64 (in addition to SHA-1). So, if
> apr_checkpass were to go into apr then ap_base64 would have to too.
But apr_validate_password is a VERY useful function. I would personally
rather move ap_validate_password out of apr_md5, and put it in it's own
file. If it requires base64, then base64 goes into the crypt directory
too.
Although I know that will upset some people.
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
Re: Encoding code in APR
Posted by "Life is hard, and then you die" <ro...@innovation.ch>.
On Sat, Jul 08, 2000 at 09:08:47AM -0700, rbb@covalent.net wrote:
>
> > ap_base64, ap_checkpass -> lib/utils/
>
> ap_base64 should be moved. ap_checkpass still bothers me. The function
> it implements is in APR already.
But it's incomplete (it's missing Netscape's SHA-1 based password).
This means apr_validate_password has to move out of apr_md5. Also, the
Netscape password scheme needs base64 (in addition to SHA-1). So, if
apr_checkpass were to go into apr then ap_base64 would have to too.
> I would rather just see that function
> moved into the crypt directory in APR.
>
> > ap/ap_cache, ap/ap_hooks -> lib/utils/
> >
> > I'm wondering about main/util_* - util_md5.c and util_date.c could be
> > moved to lib/utils/, but the other stuff references things like request_rec
> > and therefore aren't suitable for putting in lib/utils/ unless we make that
> > lib tightly bound to apache (i.e. unusable outside of it).
>
> Take a VERY close look at main/utils_*. For the most part, the references
> to request_rec's can be replaced with one or two fields from the
> request_rec. Regardless, I am VERY hesitant to move anything into
> lib/utils right now. That library will need it's own build system,
> because we don't want to tie it to Apache. The hooks are tied to APR VERY
> tightly, I haven't looked at ap_cache.
Ok, I'll leave that stuff alone since you have a much better handle on it
than I.
Cheers,
Ronald
Re: Encoding code in APR
Posted by rb...@covalent.net.
> > Either apr/crypto or apr/hash, either way. crypto is probably better,
> > because it can't be confused with hash tables.
>
> Ok, so
>
> apr_md5, apr_sha1 -> lib/apr/crypto/
Yes.
> ap_base64, ap_checkpass -> lib/utils/
ap_base64 should be moved. ap_checkpass still bothers me. The function
it implements is in APR already. I would rather just see that function
moved into the crypt directory in APR.
> ap/ap_cache, ap/ap_hooks -> lib/utils/
>
> I'm wondering about main/util_* - util_md5.c and util_date.c could be
> moved to lib/utils/, but the other stuff references things like request_rec
> and therefore aren't suitable for putting in lib/utils/ unless we make that
> lib tightly bound to apache (i.e. unusable outside of it).
Take a VERY close look at main/utils_*. For the most part, the references
to request_rec's can be replaced with one or two fields from the
request_rec. Regardless, I am VERY hesitant to move anything into
lib/utils right now. That library will need it's own build system,
because we don't want to tie it to Apache. The hooks are tied to APR VERY
tightly, I haven't looked at ap_cache.
I say go for it.
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
Re: Encoding code in APR
Posted by "Life is hard, and then you die" <ro...@innovation.ch>.
On Fri, Jul 07, 2000 at 08:28:44PM -0700, rbb@covalent.net wrote:
> >
> > For the SHA1 and MD5 hashing stuff, I'd suggest putting them under
> > apr/crypto/ ("cryptographic algorithms"). That leaves the directory open to
> > other crypto stuff (i.e interfaces to SSL, TLS, or whatever).
>
> Either apr/crypto or apr/hash, either way. crypto is probably better,
> because it can't be confused with hash tables.
Ok, so
apr_md5, apr_sha1 -> lib/apr/crypto/
ap_base64, ap_checkpass -> lib/utils/
ap/ap_cache, ap/ap_hooks -> lib/utils/
I'm wondering about main/util_* - util_md5.c and util_date.c could be
moved to lib/utils/, but the other stuff references things like request_rec
and therefore aren't suitable for putting in lib/utils/ unless we make that
lib tightly bound to apache (i.e. unusable outside of it).
Cheers,
Ronald
Re: Encoding code in APR
Posted by rb...@covalent.net.
> They hash data into a fixed-size block. It is an irreversible process. That
> is not encoding.
>
> If the code says "encoding", then it should be changed. It is an improper
> characterization of the process.
Then lets put them in apr/hash.
> No problem, I agree with you on this. You beat me to moving the pool
> functions to apr_pools.h. I was getting pissed off at searching through
> apr_pools.h, apr_lib.h, and apr_general.h. (why the latter two?) I was just
> about to do it, then saw your checkin :-)
apr_lib.h was what all of the lib functions were put in. I always hated
this, and never got around to fixing it until I had to document
everything (something else that was long overdue). Apr_general was a
trash bin for a while, until I was able to get everything sorted
out. That took longer than I expected. :-)
> For the SHA1 and MD5 hashing stuff, I'd suggest putting them under
> apr/crypto/ ("cryptographic algorithms"). That leaves the directory open to
> other crypto stuff (i.e interfaces to SSL, TLS, or whatever).
Either apr/crypto or apr/hash, either way. crypto is probably better,
because it can't be confused with hash tables.
I will get to this next week unless somebody beats me to it.
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
Re: Encoding code in APR
Posted by Greg Stein <gs...@lyra.org>.
On Fri, Jul 07, 2000 at 07:52:36AM -0700, rbb@covalent.net wrote:
>...
> > *) MD5 and SHA1 are *hash* algorithms. not encodings. base64 is an encoding.
>
> Read the code please.
Doesn't matter what the code says :-)
http://www.mach5.com/crypto/algorithms.html -- look at the bottom part of
the page under "hash algorithms"
> As I said before ALL of the code refers to
> encoding data. That sure makes it sound to me like they can be used as
> encoding algorithms. Whether that was their purpose or not is
> irrelevant. They can be used to encode data.
They hash data into a fixed-size block. It is an irreversible process. That
is not encoding.
If the code says "encoding", then it should be changed. It is an improper
characterization of the process.
> > May as well toss the SHA1 hash in there. The two files would simply go into
> > apr/lib/.
>
> No. I am trying to remove apr/lib, because it is a horrible mess of
> unrelated functions. I would like for it to be easy to find functions in
> APR, and currently it isn't inside of lib. Whenever I have to give a
> presentation about APR, I have a section in my talk about "Functions moved
> from Apache 1.3 to APR". That is horrible. I am moving things out of lib
> sometime next week, and making new directories for them.
No problem, I agree with you on this. You beat me to moving the pool
functions to apr_pools.h. I was getting pissed off at searching through
apr_pools.h, apr_lib.h, and apr_general.h. (why the latter two?) I was just
about to do it, then saw your checkin :-)
For the SHA1 and MD5 hashing stuff, I'd suggest putting them under
apr/crypto/ ("cryptographic algorithms"). That leaves the directory open to
other crypto stuff (i.e interfaces to SSL, TLS, or whatever).
Cheers,
-g
--
Greg Stein, http://www.lyra.org/
Re: Encoding code in APR
Posted by rb...@covalent.net.
> > If you review the archives, you'll discover that later versions of FreeBSD
> > modified their version of crypt to produce different results. They are a
> > bit more secure IIRC, but the results are also just not portable. This
> > means that if we don't put the encoding stuff into APR, and somebody uses
> > an MD5 algorithm on a FreeBSD box, that code will not be portable to a
> > Windows machine using APR with the lib/util MD5 algorithm.
>
> You're mixing your arguments here.
>
> *) crypt() is non-portable
> *) the MD5 hash algorithm is totally portable
I am not mixing arguments, your not paying attention. :-)
The MD5 algorithm is only portable if everybody implements it as
stock. Not every platform does. This was proven on FreeBSD, where we
couldn't GET a stock version of MD5 so that the same passwords could be
used on Unix and Windows, or even two Unix boxes that weren't running the
same version of FreeBSD
> *) the above argument doesn't hold
> *) not all platforms have these hash algorithms, but that doesn't matter --
> we already said that we will never attempt to rely on the platform's hash
> functions (we have our own, so use it)
The problem is Apache versus APR. If MD5 isn't in APR, then we don't have
our own implementation unless we are Apache. It's that simple.
> *) MD5 and SHA1 are *hash* algorithms. not encodings. base64 is an encoding.
Read the code please. As I said before ALL of the code refers to
encoding data. That sure makes it sound to me like they can be used as
encoding algorithms. Whether that was their purpose or not is
irrelevant. They can be used to encode data.
> May as well toss the SHA1 hash in there. The two files would simply go into
> apr/lib/.
No. I am trying to remove apr/lib, because it is a horrible mess of
unrelated functions. I would like for it to be easy to find functions in
APR, and currently it isn't inside of lib. Whenever I have to give a
presentation about APR, I have a section in my talk about "Functions moved
from Apache 1.3 to APR". That is horrible. I am moving things out of lib
sometime next week, and making new directories for them.
> I don't see an argument for base64 encoding, though. I would disagree with
> moving that into APR.
Fine. I'm not going to argue about this anymore.
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
Re: Encoding code in APR
Posted by Greg Stein <gs...@lyra.org>.
On Thu, Jul 06, 2000 at 08:40:39AM -0700, rbb@covalent.net wrote:
>
> I thought of another reason why the md5/sha1 encoding code needs to be in
> APR. It may be VERY portable code, but the results aren't portable. I
> know that makes very little sense. Think about why we decided to always
> use the Apache MD5 code on all platforms. We got the code originally from
> FreeBSD, so we know they have the code, we could have used the crypt
> function on FreeBSD.
>
> If you review the archives, you'll discover that later versions of FreeBSD
> modified their version of crypt to produce different results. They are a
> bit more secure IIRC, but the results are also just not portable. This
> means that if we don't put the encoding stuff into APR, and somebody uses
> an MD5 algorithm on a FreeBSD box, that code will not be portable to a
> Windows machine using APR with the lib/util MD5 algorithm.
You're mixing your arguments here.
*) crypt() is non-portable
*) the MD5 hash algorithm is totally portable
The problem that you refer to is trying to use crypt() to do MD5 hashing.
That just doesn't work.
>...
> With this argument and the argument laid out yesterday about not all
> platforms having encoding API's, does anybody still have a problem with
> creating lib/apr/encode, and putting ap_(md5/sha1/base64) there?
*) the above argument doesn't hold
*) not all platforms have these hash algorithms, but that doesn't matter --
we already said that we will never attempt to rely on the platform's hash
functions (we have our own, so use it)
*) MD5 and SHA1 are *hash* algorithms. not encodings. base64 is an encoding.
All that said, I have discovered a reason for MD5 to be in APR, and it has
nothing to do with the above :-) ... APR needs to do UUID generation which
is non-portable. Win32 has UuidCreate(), but other platforms need to roll an
algorithm. Part of the UUID is random data. This can be fetched using
something like the "truerand" library (which is already in APR?). But when a
good random number generator isn't present, then a great way to generate
random bytes is to apply the MD5 hash to a block of data. By definition, the
result is random :-) ... in any case, this means that APR itself will have a
dependency upon MD5 hashing. Therefore, it needs to include it.
May as well toss the SHA1 hash in there. The two files would simply go into
apr/lib/.
I don't see an argument for base64 encoding, though. I would disagree with
moving that into APR.
Cheers,
-g
--
Greg Stein, http://www.lyra.org/