You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by Sam Magnuson <sm...@aventail.com> on 2000/06/20 23:58:37 UTC
hooks
What is the reason we have hooks (2.0) based on function name (ala
ap_hook_fudge, ap_run_fudge) rather than some key (ie
ap_add_callback(fudge_hook,...), ap_call_hook(fudge_hook))? I'm sure there
is some good reason this has been done, but more than once I've needed to
remind myself what the macro generated (ick) functions were going to look
like, seems if we had a single function that dispatched these hooks it
would be easier to maintain/lookup.
It would also seem to be a good design to have _dynamic_ modules use
runtime lookups of these hooks, rather than _static_ information. Doing
this would remove some dependency of a dynamic module on the apache it's
being used with.
The DECLARE_HOOK stuff is good, however it doesn't lend itself well to
having modules (protocol modules?) ap_run_*()'ing hooks (ie I must see an
AP_DECLARE_HOOK for fudge, before I can actually bind to it). It also means
I can't really "reuse" a hook if I come along with a module to replace the
old functionality...
//Sam
--
Sam Magnuson <sm...@aventail.com>
RE: hooks
Posted by "William A. Rowe, Jr." <wr...@lnd.com>.
> From: smagnuson@aventail.com [mailto:smagnuson@aventail.com]
> Sent: Tuesday, June 20, 2000 6:42 PM
>
> Ie, how will linking issues be solved in Windows (or AIX) where all
> symbols must be resolved in the dll (or .so)? It seems now the mod_* will
> need to have the AP_IMPLEMENT_HOOK_* and AP_HOOK_STRUCT in its shared
> library, or am I missing something?
I think so. The Apache core modules implement the hooks to extend
functionallity to the modules, not visa-versa.
So if you want to hook into a specific Apache event, you simply call
ap_hook_whatIwillDo (exported, declared 'import' in the header for a
win32 so/dll) and off you are to the races. Apache will call your
hook, in it's turn, from the ap_run_whatIwillDo() function and your
function will be invoked. Not easily as bad as we are seeing.
I don't see the day where the dynamic so/dll exports a hook that the
module will implement (excepting the core, which I don't classify as
your usual so/dll.) A patch to even allow this behavior was soundly
blasted as unnessesary. But we could certainly put it back if you
can prove a module that must export a hook itself, instead of simply
providing a function to be hooked.
Re: hooks
Posted by rb...@covalent.net.
> > I was mainly thinking of the MD5 stuff with regards to portability,
> > because some platforms have it and others don't. We actually took our
> > code from FreeBSD, so we know they have this function.
>
> I believe the result of that conversation was just to always use our
> functions rather than trying to rely on per-platform MD5 libraries.
Yes, but this doesn't reduce the need for portability here, and it doesn't
reduce the argument that not all platforms have an encryption/encoding
routine.
> > The other argument is that this is a bunch of VERY portable encryption
> > code. Anybody who is trying to write a secure piece of portable software
> > would most likely like to use some of these functions. Isn't this what
> > APR is supposed to provide? The ability to write very portable software
> > with one library. We can put all of these functions in lib/apr/encrypt or
> > lib/apr/enocde (may be more precise) and be able to enable/disable them at
> > compile time. I really think these are useful portable routines.
>
> MD5 and SHA1 are "hash" functions. Very far from encryption or even
> encoding. We should avoid calling them anything close to encryption because
> that would open a huge can of worms :-)
Considering that both the md5 and sha1 files refer to them encoding data
multiple times, I think we are safe calling them encoding functions.
> Both hash functions are inherently portable, which makes their placement in
> APR dubious.
The placement in APR isn't dubious. They are very useful portable
functions. These are also functions that we have needed, and used
extensively. I really don't see what the problem is as long as their
compilation can be turned off easily. It can, by adding a simple flag to
the configure stage.
> Given our move towards shifting some functionality over to lib/util, then I
> believe that is a good location for these generic, inherently-portable
> suites of functionality.
We don't even have lib/util yet. We have been talking about it for
months, but nothing has ever been done, so I don't see how we can say we
have made that move. Putting a new library in lib/util also implies that
this library will have it's own build system. We can not just put a bunch
of files into a lib subdirectory and tie it into Apache's build system.
> Correct. APR should be our portability-enabler rather than a kitchen sink of
> useful functions. Keeping a tight, clean focus on that will keep APR on
> track to being an excellent tool.
APR is a very focused project. It's goal is to provide a set of functions
to help make programs portable. This does not mean that all code that is
in APR must be inherently non-portable. It just means that all code in
APR must be portable.
> [ note: Subversion (subversion.tigris.org) has definitively selected APR as
> a base library. their first milestone is September, where we'll see a
> non-Apache application ship using APR. whee! ]
Sorry, they are too late. :-) I know of at least one closed source
application already using APR. No, I'm not the one writing it. I haven't
announced that it is being used or who is writing it, because the project
isn't finished. :-)
> Now if lib/util becomes "interesting" to package, then we can make that
> choice later. IMO, there are already too many "kitchen sink" libraries out
> there. Lets avoid that space. APR can be incredibly successful by simply
> offering portability to apps.
But offering a small set of encoding functions could be argued to provide
portability. Consider the trouble Apache had getting encoded passwords
into the Windows version. Not all platforms provide ANY scheme to
encrypt/encode data. This is a useful function, and it belongs in APR.
> > > Etc referred to stuff in ap and main: ap/ap_cache, ap/ap_hooks, and the
> > > main/util* stuff.
>
> What is the rule for moving something into lib/util? What is the rule for
> NOT moving it? I am unclear on this. Do we simply say anything that happened
> to have been named "util_*" gets moved? Why not mpm_common? It is a neat
> little set of common functions that an MPM might use. Is util_uri really a
> non-web-server thing that is destined for lib/util?
Things that are moved to lib/util, should be those functions that would be
useful to have outside of the webserver. If they only make sense inside
of Apache, then the function doesn't belong in lib/util. It belongs in
ap/. As far as the util_* functions, if we remove the reliance on the
request_rec and ap_log_error* from the few functions that require it, then
yes those functions become VERY useful outside of the webserver. Imagine
a small app wanting to make a web request or to parse a URI. Having
util_uri would go a long way to making that easier. Some of the util_*
files shouldn't be moved, they should be removed from the server
completely. For example, util_script. This file needs to start being
merged into mod_cgi[d], and the remaining parts need to be moved into
modules/common (a nonexistant directory, for functions needed by multiple
modules. This would always be compiled into the server).
> Meta-point here: if we're going to start creating directories for this and
> that, then *please* make sure that a directory has a well-defined meaning.
> Otherwise, it could create confusion when people think, "dang. where does
> that module live? in the utility directory? oh, wait, that is an HTTP
> utility, so it must be in the main directory. oops. but it applies to any
> networking protocol, so it must be in the netutil directory."
Of course. :-)
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
Re: hooks
Posted by Greg Stein <gs...@lyra.org>.
On Tue, Jul 04, 2000 at 05:10:26PM -0700, rbb@covalent.net wrote:
> > > > If everybody is happy with lib/utils then I'll attempt to beat you to
> > > > it for the md5, sha1, base64, getpass, etc.
> > >
>
> > > I would still really like to see all of these files in APR. They all use
> > > APR types, and although the code is all common for all platforms, they are
> > > portability issues for most programs. By moving them into different
> > > directories within APR, we can enable and disable them at compile time.
> >
> > What portability issues? The ebcdic stuff is now in apr (ap_xlate, or so
> > I assume), and I can't think of any others.
>
> The EBCDIC stuff is still in ap_base64.c, but that is another issue.
>
> I was mainly thinking of the MD5 stuff with regards to portability,
> because some platforms have it and others don't. We actually took our
> code from FreeBSD, so we know they have this function.
I believe the result of that conversation was just to always use our
functions rather than trying to rely on per-platform MD5 libraries.
> The other argument is that this is a bunch of VERY portable encryption
> code. Anybody who is trying to write a secure piece of portable software
> would most likely like to use some of these functions. Isn't this what
> APR is supposed to provide? The ability to write very portable software
> with one library. We can put all of these functions in lib/apr/encrypt or
> lib/apr/enocde (may be more precise) and be able to enable/disable them at
> compile time. I really think these are useful portable routines.
MD5 and SHA1 are "hash" functions. Very far from encryption or even
encoding. We should avoid calling them anything close to encryption because
that would open a huge can of worms :-)
Both hash functions are inherently portable, which makes their placement in
APR dubious.
Given our move towards shifting some functionality over to lib/util, then I
believe that is a good location for these generic, inherently-portable
suites of functionality.
> > Last time I asked about
> > this, Greg was against moving them into apr and other-Bill was for it -
> > other than that nobody cared. I'm equally split on whether to put it in
> > apr or lib/utils.
Correct. APR should be our portability-enabler rather than a kitchen sink of
useful functions. Keeping a tight, clean focus on that will keep APR on
track to being an excellent tool.
[ note: Subversion (subversion.tigris.org) has definitively selected APR as
a base library. their first milestone is September, where we'll see a
non-Apache application ship using APR. whee! ]
Now if lib/util becomes "interesting" to package, then we can make that
choice later. IMO, there are already too many "kitchen sink" libraries out
there. Lets avoid that space. APR can be incredibly successful by simply
offering portability to apps.
>...
> > > Which things are included in the etc? I would like to be VERY careful
> > > about what we move out of APR, because some of the functions in the lib
> > > directory are portability issues, and not cruft for APR.
> >
> > Etc referred to stuff in ap and main: ap/ap_cache, ap/ap_hooks, and the
> > main/util* stuff.
What is the rule for moving something into lib/util? What is the rule for
NOT moving it? I am unclear on this. Do we simply say anything that happened
to have been named "util_*" gets moved? Why not mpm_common? It is a neat
little set of common functions that an MPM might use. Is util_uri really a
non-web-server thing that is destined for lib/util?
Meta-point here: if we're going to start creating directories for this and
that, then *please* make sure that a directory has a well-defined meaning.
Otherwise, it could create confusion when people think, "dang. where does
that module live? in the utility directory? oh, wait, that is an HTTP
utility, so it must be in the main directory. oops. but it applies to any
networking protocol, so it must be in the netutil directory."
Cheers,
-g
--
Greg Stein, http://www.lyra.org/
Re: hooks
Posted by "Life is hard, and then you die" <ro...@innovation.ch>.
On Thu, Jul 06, 2000 at 09:53:15AM -0400, Jeff Trawick wrote:
> > Date: Wed, 5 Jul 2000 13:12:21 -0700
> > From: "Life is hard, and then you die" <ro...@innovation.ch>
>
> > Btw., does anybody disagree with changing the update functions in
> > md5 and sha1 to ap_MD5Update_ascii(), ap_MD5Update_binary(),
> > ap_SHA1Update_ascii(), and ap_SHA1Update_binary()? I.e. to better
> > reflect which are ascii and which binary (the ascii forms first
> > translate the characters as necessary to ascii). And similarly
> > for ap_base64. Currently we have ap_SHA1Update() and
> > ap_SHA1Update_binary() in 1.3.
>
> A. MD5:
>
> Currently:
>
> single call to ap_MD5Init()
> optional single call to ap_MD5SetXlate()
> multiple calls to ap_MD5Update()
> single call to ap_MD5Final()
>
> Your proposal (I guess):
>
> single call to ap_MD5Init()
> multiple calls to either ap_MD5Update_ascii() or ap_MD5Update_binary()
> single call to ap_MD5Final()
Yes.
> 1) _ascii is a mis-nomer for this feature; we don't know or care what
> type of translation is performed (if it is done); since this is
> content, it could be anything
Agreed - I wasn't thinking too far. So _text would probably be better.
> 2) I like the current API better because once the app sets up the
> translation properly it doesn't have to have dual paths later
> (Hmmm... do I call ap_MD5Update_ascii() or ap_MD5Update_binary()?)
>
> You could get rid of ap_MD5SetXlate() and provide an additional
> parameter to ap_MD5Init() if you want to simplify it.
The problem with the current setup and with ap_MD5Init(ap_xlate,...) is that
they aren't mt-safe. The later could be made so if the ap_xlate is stored
as part of the context, not in a global variable as is done in
ap_SHA1InitEBCDIC. Passing the ap_xlate in _text provides for a little
more flexibility, but I honestly don't know if that's needed or useful.
> B. SHA1
>
> This is definitely hard-wired for EBCDIC currently. The API set could
> be changed to be more similar to the MD5 one, but you'll need to
> change more code in the utilities to be EBCDIC-aware. It is ugly
> either way, but until some app needs a different type of translation
> for SHA1, I think the code savings (and the related smaller amount of
> EBCDIC awareness in Apache) with the current API set is worth the
> ugliness. #ifdef EBCDIC is pretty ugly to me, and I are EBCDIC for at
> least part of the day.
This brings up a second point that I'm having problems with: if we have
ap_xlate, then why the heck do we have #ifdef CHARSET_EBCDIC everywhere?
We should just be able to pass an ap_xlate everywhere, and if that's
NULL then the ap_xlate_conv_buffer should be a no-op. I don't think that
extra function call and test is going to affect performance noticeably,
considering all the other code in the sha/md5/base64/etc functions.
> C. base64
>
> no real opinion... I don't have a notion of how much app-level code
> would get changed (and possibly uglified). You'll find out soon
> enough.
Basically same as above, i.e. supply _text and _binary versions. Since these
are not multipart operations we have to supply the ap_xlate to the _text.
The only problem is the pr2six table - not sure what to do about it. One
possibility would be to use a cache of pr2six tables, and compute them on
the fly if not in the cache.
Cheers,
Ronald
Re: hooks
Posted by Jeff Trawick <tr...@ibm.net>.
> Date: Wed, 5 Jul 2000 13:12:21 -0700
> From: "Life is hard, and then you die" <ro...@innovation.ch>
> Btw., does anybody disagree with changing the update functions in
> md5 and sha1 to ap_MD5Update_ascii(), ap_MD5Update_binary(),
> ap_SHA1Update_ascii(), and ap_SHA1Update_binary()? I.e. to better
> reflect which are ascii and which binary (the ascii forms first
> translate the characters as necessary to ascii). And similarly
> for ap_base64. Currently we have ap_SHA1Update() and
> ap_SHA1Update_binary() in 1.3.
A. MD5:
Currently:
single call to ap_MD5Init()
optional single call to ap_MD5SetXlate()
multiple calls to ap_MD5Update()
single call to ap_MD5Final()
Your proposal (I guess):
single call to ap_MD5Init()
multiple calls to either ap_MD5Update_ascii() or ap_MD5Update_binary()
single call to ap_MD5Final()
1) _ascii is a mis-nomer for this feature; we don't know or care what
type of translation is performed (if it is done); since this is
content, it could be anything
2) I like the current API better because once the app sets up the
translation properly it doesn't have to have dual paths later
(Hmmm... do I call ap_MD5Update_ascii() or ap_MD5Update_binary()?)
You could get rid of ap_MD5SetXlate() and provide an additional
parameter to ap_MD5Init() if you want to simplify it.
B. SHA1
This is definitely hard-wired for EBCDIC currently. The API set could
be changed to be more similar to the MD5 one, but you'll need to
change more code in the utilities to be EBCDIC-aware. It is ugly
either way, but until some app needs a different type of translation
for SHA1, I think the code savings (and the related smaller amount of
EBCDIC awareness in Apache) with the current API set is worth the
ugliness. #ifdef EBCDIC is pretty ugly to me, and I are EBCDIC for at
least part of the day.
C. base64
no real opinion... I don't have a notion of how much app-level code
would get changed (and possibly uglified). You'll find out soon
enough.
Best wishes,
Jeff
--
Jeff Trawick | trawick@ibm.net | PGP public key at web site:
http://www.geocities.com/SiliconValley/Park/9289/
Born in Roswell... married an alien...
Re: hooks
Posted by "Life is hard, and then you die" <ro...@innovation.ch>.
On Wed, Jul 05, 2000 at 02:06:23PM -0700, rbb@covalent.net wrote:
>
> > > Btw., does anybody disagree with changing the update functions in
> > > md5 and sha1 to ap_MD5Update_ascii(), ap_MD5Update_binary(),
> > > ap_SHA1Update_ascii(), and ap_SHA1Update_binary()? I.e. to better
> > > reflect which are ascii and which binary (the ascii forms first
> > > translate the characters as necessary to ascii). And similarly
> > > for ap_base64. Currently we have ap_SHA1Update() and
> > > ap_SHA1Update_binary() in 1.3.
> >
> > Euh... why not call it ap_MD5Update_xlate() and pass an xlate pointer? Seems
> > a bit better than hard-wiring to ascii/ebcdic and using just one of those
> > two charsets.
We could do that. But the conversion will always be <default-charset> ->
<ascii>. Hmm, wait, maybe it will be <local-charset> -> <ascii> in some
cases, in which case your suggestion would make sense. Umm, what's the
behaviour when writing headers? Oh, I think I see it: ap_hdrs_to_ascii
in main/util_charset.c. So it ends up being a fixed translation, at least
for Apache. OTOH if these are supposed to be generic libraries then
passing ap_xlate, while a little uglier for the rest of code, may be the
right thing.
I guess I'm not too happy with the current setup in which ap_MD5InitEBCDIC
is called once to setup the ap_xlate.
> Huh. The translation is already done. I was under the impression that we
> currently have two functions, ap_SHA1Update and ap_SHA1Update_binary. I
> thought the suggestion was to just change ap_SHA1Update to
> ap_SHA1Update_ascii.
Correct.
> The translation only makes sense if we are dealing
> with ascii data IMO, and that is setup before hand and is taken care of
> automagically. What am I missing?
Not sure if I understand you correctly. The point of the _ascii version
is to assume that the input data is text and translate it to ascii before
doing the update (actually, 8859_1 would be more precise). The point of
this is that when you're sticking a hash value or base64-encoded data
into the headers you first need to make sure the input to the hash or
base64 is ascii, because that's what the other side will assume.
base64_encode will also translate the result back, btw, because that's
also text.
Cheers,
Ronald
Re: hooks
Posted by rb...@covalent.net.
> > Btw., does anybody disagree with changing the update functions in
> > md5 and sha1 to ap_MD5Update_ascii(), ap_MD5Update_binary(),
> > ap_SHA1Update_ascii(), and ap_SHA1Update_binary()? I.e. to better
> > reflect which are ascii and which binary (the ascii forms first
> > translate the characters as necessary to ascii). And similarly
> > for ap_base64. Currently we have ap_SHA1Update() and
> > ap_SHA1Update_binary() in 1.3.
>
> Euh... why not call it ap_MD5Update_xlate() and pass an xlate pointer? Seems
> a bit better than hard-wiring to ascii/ebcdic and using just one of those
> two charsets.
Huh. The translation is already done. I was under the impression that we
currently have two functions, ap_SHA1Update and ap_SHA1Update_binary. I
thought the suggestion was to just change ap_SHA1Update to
ap_SHA1Update_ascii. The translation only makes sense if we are dealing
with ascii data IMO, and that is setup before hand and is taken care of
automagically. What am I missing?
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
Re: hooks
Posted by Greg Stein <gs...@lyra.org>.
On Wed, Jul 05, 2000 at 01:12:21PM -0700, Life is hard, and then you die wrote:
>...
> Btw., does anybody disagree with changing the update functions in
> md5 and sha1 to ap_MD5Update_ascii(), ap_MD5Update_binary(),
> ap_SHA1Update_ascii(), and ap_SHA1Update_binary()? I.e. to better
> reflect which are ascii and which binary (the ascii forms first
> translate the characters as necessary to ascii). And similarly
> for ap_base64. Currently we have ap_SHA1Update() and
> ap_SHA1Update_binary() in 1.3.
Euh... why not call it ap_MD5Update_xlate() and pass an xlate pointer? Seems
a bit better than hard-wiring to ascii/ebcdic and using just one of those
two charsets.
Cheers,
-g
--
Greg Stein, http://www.lyra.org/
Re: hooks
Posted by "Life is hard, and then you die" <ro...@innovation.ch>.
On Thu, Jul 06, 2000 at 10:21:51AM -0400, Jeff Trawick wrote:
> > Date: Thu, 6 Jul 2000 06:26:36 -0700
> > From: Greg Stein <gs...@lyra.org>
> >
> > On Thu, Jul 06, 2000 at 08:59:14AM -0400, Jeff Trawick wrote:
> > > > From: rbb@covalent.net
> > > > Date: Wed, 5 Jul 2000 13:35:04 -0700 (PDT)
> > > >
> > > > > > The EBCDIC stuff is still in ap_base64.c, but that is another issue.
> > > > >
> > > > > That has to change anyway to use ap_xlate.
> > > >
> > > > Agreed.
> > >
> > > I guess I'm supposed to know what you're talking about here, but I
> > > don't. Can you clarify?
> >
> > Read the thread a bit later... <20...@innovation.ch> is a
> > good terminus. Ryan's response dropped the authors of the quoted material,
> > but it is basically Ronald on the innermost, myself, Ryan, then Ronald
> > again. :-)
>
> Yep, I went back to the archive and later responded to one of the
> earlier messages. I still don't understand "change anyway to use
> ap_xlate()." All EBCDIC code in 2.0 has been solely APR-based (i.e.,
> ap_xlate_*()) for many moons.
Sorry, you're right, I must have been smoking something weird. I think
the #ifdef CHARSET_EBCDIC confused me.
Cheers,
Ronald
Re: hooks
Posted by Jeff Trawick <tr...@ibm.net>.
> Date: Thu, 6 Jul 2000 06:26:36 -0700
> From: Greg Stein <gs...@lyra.org>
>
> On Thu, Jul 06, 2000 at 08:59:14AM -0400, Jeff Trawick wrote:
> > > From: rbb@covalent.net
> > > Date: Wed, 5 Jul 2000 13:35:04 -0700 (PDT)
> > >
> > > > > The EBCDIC stuff is still in ap_base64.c, but that is another issue.
> > > >
> > > > That has to change anyway to use ap_xlate.
> > >
> > > Agreed.
> >
> > I guess I'm supposed to know what you're talking about here, but I
> > don't. Can you clarify?
>
> Read the thread a bit later... <20...@innovation.ch> is a
> good terminus. Ryan's response dropped the authors of the quoted material,
> but it is basically Ronald on the innermost, myself, Ryan, then Ronald
> again. :-)
Yep, I went back to the archive and later responded to one of the
earlier messages. I still don't understand "change anyway to use
ap_xlate()." All EBCDIC code in 2.0 has been solely APR-based (i.e.,
ap_xlate_*()) for many moons.
Have fun...
--
Jeff Trawick | trawick@ibm.net | PGP public key at web site:
http://www.geocities.com/SiliconValley/Park/9289/
Born in Roswell... married an alien...
Re: hooks
Posted by Greg Stein <gs...@lyra.org>.
On Thu, Jul 06, 2000 at 08:59:14AM -0400, Jeff Trawick wrote:
> > From: rbb@covalent.net
> > Date: Wed, 5 Jul 2000 13:35:04 -0700 (PDT)
> >
> > > > The EBCDIC stuff is still in ap_base64.c, but that is another issue.
> > >
> > > That has to change anyway to use ap_xlate.
> >
> > Agreed.
>
> I guess I'm supposed to know what you're talking about here, but I
> don't. Can you clarify?
Read the thread a bit later... <20...@innovation.ch> is a
good terminus. Ryan's response dropped the authors of the quoted material,
but it is basically Ronald on the innermost, myself, Ryan, then Ronald
again. :-)
Cheers,
-g
--
Greg Stein, http://www.lyra.org/
Re: hooks
Posted by Jeff Trawick <tr...@ibm.net>.
> From: rbb@covalent.net
> Date: Wed, 5 Jul 2000 13:35:04 -0700 (PDT)
>
> > > The EBCDIC stuff is still in ap_base64.c, but that is another issue.
> >
> > That has to change anyway to use ap_xlate.
>
> Agreed.
I guess I'm supposed to know what you're talking about here, but I
don't. Can you clarify?
--
Jeff Trawick | trawick@ibm.net | PGP public key at web site:
http://www.geocities.com/SiliconValley/Park/9289/
Born in Roswell... married an alien...
Re: Encoding code in APR
Posted by Greg Stein <gs...@lyra.org>.
On Fri, Jul 07, 2000 at 08:28:44PM -0700, rbb@covalent.net wrote:
>...
> > For the SHA1 and MD5 hashing stuff, I'd suggest putting them under
> > apr/crypto/ ("cryptographic algorithms"). That leaves the directory open to
> > other crypto stuff (i.e interfaces to SSL, TLS, or whatever).
>
> Either apr/crypto or apr/hash, either way. crypto is probably better,
> because it can't be confused with hash tables.
Right. That was my thought process, too.
Tonite, I also realized that the /dev/random and truerand stuff in APR
should go in there. Those are used only for true, "cryptographically safe"
random numbers. (they're too slow for basic random number gen)
I'll get that stuff moved at some point, as I'd like to use it for the UUID
generation on non-Win32 platforms.
Cheers,
-g
--
Greg Stein, http://www.lyra.org/
Re: Encoding code in APR
Posted by "Life is hard, and then you die" <ro...@innovation.ch>.
On Sun, Jul 09, 2000 at 07:38:57PM -0700, rbb@covalent.net wrote:
> On Sun, 9 Jul 2000, Greg Stein wrote:
> > On Sun, Jul 09, 2000 at 04:09:46PM -0700, Life is hard, and then you die wrote:
> > > On Sun, Jul 09, 2000 at 07:29:37AM -0700, rbb@covalent.net wrote:
> > >...
> > > > But apr_validate_password is a VERY useful function. I would personally
> > > > rather move ap_validate_password out of apr_md5, and put it in it's own
> > > > file. If it requires base64, then base64 goes into the crypt directory
> > > > too.
> >
> > Hmm. base64 isn't crypto :-) If/when base64 goes into APR, that is where
> > your "encode" directory would pop up. Or:
> >
> > encoder
> > encoding
> > encode
> >
> > Dunno. The problem is how to properly classify base64 because we don't have
> > any siblings for it (yet).
>
> Am I the only person who doesn't understand this. MD5 is used to encode
> passwords in our scheme, so is SHA1. The algorithms themselves, may not
> be meant for this, but this is what Apache uses these algorithms
> for.
No. As Greg already said, encodings/encoders are usually understood to
be reversible processes; hashes/digests are not reversible. So no,
Apache does not encode passwords, it hashes them - the password is lost
after the process. But this is nit-picking.
> Yes, we also take MD5 checksums at times. But, the reason APR needs
> MD5 is to encode passwords. This is also why we need to ap_checkpasswd
> function.
>
> Why can't we put all of the algorithms in the same directory, along with a
> simple function to check the passwords? This just makes sense to me, so
> please explain it to me.
Fine with me. But we should call not call it crypto (which none of this stuff
is), or encoders (which only base64 is). misc? utils? security? (though
base64 would be stretching that last term a bit).
Cheers,
Ronald
Re: Encoding code in APR
Posted by Greg Stein <gs...@lyra.org>.
On Tue, Jul 11, 2000 at 11:05:09PM +0100, Ben Laurie wrote:
> Greg Stein wrote:
> >
> > On Sun, Jul 09, 2000 at 04:09:46PM -0700, Life is hard, and then you die wrote:
> > > On Sun, Jul 09, 2000 at 07:29:37AM -0700, rbb@covalent.net wrote:
> > >...
> > > > But apr_validate_password is a VERY useful function. I would personally
> > > > rather move ap_validate_password out of apr_md5, and put it in it's own
> > > > file. If it requires base64, then base64 goes into the crypt directory
> > > > too.
> >
> > Hmm. base64 isn't crypto :-) If/when base64 goes into APR, that is where
> > your "encode" directory would pop up. Or:
> >
> > encoder
> > encoding
> > encode
> >
> > Dunno. The problem is how to properly classify base64 because we don't have
> > any siblings for it (yet).
>
> No encoding at all is a sibling, of course. Hex is another.
Eh? How does that help to classify where to place base64?
-g
--
Greg Stein, http://www.lyra.org/
Re: Encoding code in APR
Posted by Ben Laurie <be...@algroup.co.uk>.
Greg Stein wrote:
>
> On Sun, Jul 09, 2000 at 04:09:46PM -0700, Life is hard, and then you die wrote:
> > On Sun, Jul 09, 2000 at 07:29:37AM -0700, rbb@covalent.net wrote:
> >...
> > > But apr_validate_password is a VERY useful function. I would personally
> > > rather move ap_validate_password out of apr_md5, and put it in it's own
> > > file. If it requires base64, then base64 goes into the crypt directory
> > > too.
>
> Hmm. base64 isn't crypto :-) If/when base64 goes into APR, that is where
> your "encode" directory would pop up. Or:
>
> encoder
> encoding
> encode
>
> Dunno. The problem is how to properly classify base64 because we don't have
> any siblings for it (yet).
No encoding at all is a sibling, of course. Hex is another.
Cheers,
Ben.
--
http://www.apache-ssl.org/ben.html
Coming to ApacheCon Europe 2000? http://apachecon.com/
Re: Encoding code in APR
Posted by rb...@covalent.net.
On Sun, 9 Jul 2000, Greg Stein wrote:
> On Sun, Jul 09, 2000 at 04:09:46PM -0700, Life is hard, and then you die wrote:
> > On Sun, Jul 09, 2000 at 07:29:37AM -0700, rbb@covalent.net wrote:
> >...
> > > But apr_validate_password is a VERY useful function. I would personally
> > > rather move ap_validate_password out of apr_md5, and put it in it's own
> > > file. If it requires base64, then base64 goes into the crypt directory
> > > too.
>
> Hmm. base64 isn't crypto :-) If/when base64 goes into APR, that is where
> your "encode" directory would pop up. Or:
>
> encoder
> encoding
> encode
>
> Dunno. The problem is how to properly classify base64 because we don't have
> any siblings for it (yet).
Am I the only person who doesn't understand this. MD5 is used to encode
passwords in our scheme, so is SHA1. The algorithms themselves, may not
be meant for this, but this is what Apache uses these algorithms
for. Yes, we also take MD5 checksums at times. But, the reason APR needs
MD5 is to encode passwords. This is also why we need to ap_checkpasswd
function.
Why can't we put all of the algorithms in the same directory, along with a
simple function to check the passwords? This just makes sense to me, so
please explain it to me.
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
Re: Encoding code in APR
Posted by Greg Stein <gs...@lyra.org>.
On Sun, Jul 09, 2000 at 04:09:46PM -0700, Life is hard, and then you die wrote:
> On Sun, Jul 09, 2000 at 07:29:37AM -0700, rbb@covalent.net wrote:
>...
> > But apr_validate_password is a VERY useful function. I would personally
> > rather move ap_validate_password out of apr_md5, and put it in it's own
> > file. If it requires base64, then base64 goes into the crypt directory
> > too.
Hmm. base64 isn't crypto :-) If/when base64 goes into APR, that is where
your "encode" directory would pop up. Or:
encoder
encoding
encode
Dunno. The problem is how to properly classify base64 because we don't have
any siblings for it (yet).
Cheers,
-g
--
Greg Stein, http://www.lyra.org/
Re: Encoding code in APR
Posted by "Life is hard, and then you die" <ro...@innovation.ch>.
On Sun, Jul 09, 2000 at 07:29:37AM -0700, rbb@covalent.net wrote:
> On Sat, 8 Jul 2000, Life is hard, and then you die wrote:
>
> > On Sat, Jul 08, 2000 at 09:08:47AM -0700, rbb@covalent.net wrote:
> > >
> > > > ap_base64, ap_checkpass -> lib/utils/
> > >
> > > ap_base64 should be moved. ap_checkpass still bothers me. The function
> > > it implements is in APR already.
> >
> > But it's incomplete (it's missing Netscape's SHA-1 based password).
> > This means apr_validate_password has to move out of apr_md5. Also, the
> > Netscape password scheme needs base64 (in addition to SHA-1). So, if
> > apr_checkpass were to go into apr then ap_base64 would have to too.
>
> But apr_validate_password is a VERY useful function. I would personally
> rather move ap_validate_password out of apr_md5, and put it in it's own
> file. If it requires base64, then base64 goes into the crypt directory
> too.
Of course it's useful - it's part of apr_checkpass, i.e. it's in its own
file. So we're really in complete agreement here.
Cheers,
Ronald
Re: Encoding code in APR
Posted by rb...@covalent.net.
On Sat, 8 Jul 2000, Life is hard, and then you die wrote:
> On Sat, Jul 08, 2000 at 09:08:47AM -0700, rbb@covalent.net wrote:
> >
> > > ap_base64, ap_checkpass -> lib/utils/
> >
> > ap_base64 should be moved. ap_checkpass still bothers me. The function
> > it implements is in APR already.
>
> But it's incomplete (it's missing Netscape's SHA-1 based password).
> This means apr_validate_password has to move out of apr_md5. Also, the
> Netscape password scheme needs base64 (in addition to SHA-1). So, if
> apr_checkpass were to go into apr then ap_base64 would have to too.
But apr_validate_password is a VERY useful function. I would personally
rather move ap_validate_password out of apr_md5, and put it in it's own
file. If it requires base64, then base64 goes into the crypt directory
too.
Although I know that will upset some people.
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
Re: Encoding code in APR
Posted by "Life is hard, and then you die" <ro...@innovation.ch>.
On Sat, Jul 08, 2000 at 09:08:47AM -0700, rbb@covalent.net wrote:
>
> > ap_base64, ap_checkpass -> lib/utils/
>
> ap_base64 should be moved. ap_checkpass still bothers me. The function
> it implements is in APR already.
But it's incomplete (it's missing Netscape's SHA-1 based password).
This means apr_validate_password has to move out of apr_md5. Also, the
Netscape password scheme needs base64 (in addition to SHA-1). So, if
apr_checkpass were to go into apr then ap_base64 would have to too.
> I would rather just see that function
> moved into the crypt directory in APR.
>
> > ap/ap_cache, ap/ap_hooks -> lib/utils/
> >
> > I'm wondering about main/util_* - util_md5.c and util_date.c could be
> > moved to lib/utils/, but the other stuff references things like request_rec
> > and therefore aren't suitable for putting in lib/utils/ unless we make that
> > lib tightly bound to apache (i.e. unusable outside of it).
>
> Take a VERY close look at main/utils_*. For the most part, the references
> to request_rec's can be replaced with one or two fields from the
> request_rec. Regardless, I am VERY hesitant to move anything into
> lib/utils right now. That library will need it's own build system,
> because we don't want to tie it to Apache. The hooks are tied to APR VERY
> tightly, I haven't looked at ap_cache.
Ok, I'll leave that stuff alone since you have a much better handle on it
than I.
Cheers,
Ronald
Re: Encoding code in APR
Posted by rb...@covalent.net.
> > Either apr/crypto or apr/hash, either way. crypto is probably better,
> > because it can't be confused with hash tables.
>
> Ok, so
>
> apr_md5, apr_sha1 -> lib/apr/crypto/
Yes.
> ap_base64, ap_checkpass -> lib/utils/
ap_base64 should be moved. ap_checkpass still bothers me. The function
it implements is in APR already. I would rather just see that function
moved into the crypt directory in APR.
> ap/ap_cache, ap/ap_hooks -> lib/utils/
>
> I'm wondering about main/util_* - util_md5.c and util_date.c could be
> moved to lib/utils/, but the other stuff references things like request_rec
> and therefore aren't suitable for putting in lib/utils/ unless we make that
> lib tightly bound to apache (i.e. unusable outside of it).
Take a VERY close look at main/utils_*. For the most part, the references
to request_rec's can be replaced with one or two fields from the
request_rec. Regardless, I am VERY hesitant to move anything into
lib/utils right now. That library will need it's own build system,
because we don't want to tie it to Apache. The hooks are tied to APR VERY
tightly, I haven't looked at ap_cache.
I say go for it.
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
Re: Encoding code in APR
Posted by "Life is hard, and then you die" <ro...@innovation.ch>.
On Fri, Jul 07, 2000 at 08:28:44PM -0700, rbb@covalent.net wrote:
> >
> > For the SHA1 and MD5 hashing stuff, I'd suggest putting them under
> > apr/crypto/ ("cryptographic algorithms"). That leaves the directory open to
> > other crypto stuff (i.e interfaces to SSL, TLS, or whatever).
>
> Either apr/crypto or apr/hash, either way. crypto is probably better,
> because it can't be confused with hash tables.
Ok, so
apr_md5, apr_sha1 -> lib/apr/crypto/
ap_base64, ap_checkpass -> lib/utils/
ap/ap_cache, ap/ap_hooks -> lib/utils/
I'm wondering about main/util_* - util_md5.c and util_date.c could be
moved to lib/utils/, but the other stuff references things like request_rec
and therefore aren't suitable for putting in lib/utils/ unless we make that
lib tightly bound to apache (i.e. unusable outside of it).
Cheers,
Ronald
Re: Encoding code in APR
Posted by rb...@covalent.net.
> They hash data into a fixed-size block. It is an irreversible process. That
> is not encoding.
>
> If the code says "encoding", then it should be changed. It is an improper
> characterization of the process.
Then lets put them in apr/hash.
> No problem, I agree with you on this. You beat me to moving the pool
> functions to apr_pools.h. I was getting pissed off at searching through
> apr_pools.h, apr_lib.h, and apr_general.h. (why the latter two?) I was just
> about to do it, then saw your checkin :-)
apr_lib.h was what all of the lib functions were put in. I always hated
this, and never got around to fixing it until I had to document
everything (something else that was long overdue). Apr_general was a
trash bin for a while, until I was able to get everything sorted
out. That took longer than I expected. :-)
> For the SHA1 and MD5 hashing stuff, I'd suggest putting them under
> apr/crypto/ ("cryptographic algorithms"). That leaves the directory open to
> other crypto stuff (i.e interfaces to SSL, TLS, or whatever).
Either apr/crypto or apr/hash, either way. crypto is probably better,
because it can't be confused with hash tables.
I will get to this next week unless somebody beats me to it.
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
Re: Encoding code in APR
Posted by Greg Stein <gs...@lyra.org>.
On Fri, Jul 07, 2000 at 07:52:36AM -0700, rbb@covalent.net wrote:
>...
> > *) MD5 and SHA1 are *hash* algorithms. not encodings. base64 is an encoding.
>
> Read the code please.
Doesn't matter what the code says :-)
http://www.mach5.com/crypto/algorithms.html -- look at the bottom part of
the page under "hash algorithms"
> As I said before ALL of the code refers to
> encoding data. That sure makes it sound to me like they can be used as
> encoding algorithms. Whether that was their purpose or not is
> irrelevant. They can be used to encode data.
They hash data into a fixed-size block. It is an irreversible process. That
is not encoding.
If the code says "encoding", then it should be changed. It is an improper
characterization of the process.
> > May as well toss the SHA1 hash in there. The two files would simply go into
> > apr/lib/.
>
> No. I am trying to remove apr/lib, because it is a horrible mess of
> unrelated functions. I would like for it to be easy to find functions in
> APR, and currently it isn't inside of lib. Whenever I have to give a
> presentation about APR, I have a section in my talk about "Functions moved
> from Apache 1.3 to APR". That is horrible. I am moving things out of lib
> sometime next week, and making new directories for them.
No problem, I agree with you on this. You beat me to moving the pool
functions to apr_pools.h. I was getting pissed off at searching through
apr_pools.h, apr_lib.h, and apr_general.h. (why the latter two?) I was just
about to do it, then saw your checkin :-)
For the SHA1 and MD5 hashing stuff, I'd suggest putting them under
apr/crypto/ ("cryptographic algorithms"). That leaves the directory open to
other crypto stuff (i.e interfaces to SSL, TLS, or whatever).
Cheers,
-g
--
Greg Stein, http://www.lyra.org/
Re: Encoding code in APR
Posted by rb...@covalent.net.
> > If you review the archives, you'll discover that later versions of FreeBSD
> > modified their version of crypt to produce different results. They are a
> > bit more secure IIRC, but the results are also just not portable. This
> > means that if we don't put the encoding stuff into APR, and somebody uses
> > an MD5 algorithm on a FreeBSD box, that code will not be portable to a
> > Windows machine using APR with the lib/util MD5 algorithm.
>
> You're mixing your arguments here.
>
> *) crypt() is non-portable
> *) the MD5 hash algorithm is totally portable
I am not mixing arguments, your not paying attention. :-)
The MD5 algorithm is only portable if everybody implements it as
stock. Not every platform does. This was proven on FreeBSD, where we
couldn't GET a stock version of MD5 so that the same passwords could be
used on Unix and Windows, or even two Unix boxes that weren't running the
same version of FreeBSD
> *) the above argument doesn't hold
> *) not all platforms have these hash algorithms, but that doesn't matter --
> we already said that we will never attempt to rely on the platform's hash
> functions (we have our own, so use it)
The problem is Apache versus APR. If MD5 isn't in APR, then we don't have
our own implementation unless we are Apache. It's that simple.
> *) MD5 and SHA1 are *hash* algorithms. not encodings. base64 is an encoding.
Read the code please. As I said before ALL of the code refers to
encoding data. That sure makes it sound to me like they can be used as
encoding algorithms. Whether that was their purpose or not is
irrelevant. They can be used to encode data.
> May as well toss the SHA1 hash in there. The two files would simply go into
> apr/lib/.
No. I am trying to remove apr/lib, because it is a horrible mess of
unrelated functions. I would like for it to be easy to find functions in
APR, and currently it isn't inside of lib. Whenever I have to give a
presentation about APR, I have a section in my talk about "Functions moved
from Apache 1.3 to APR". That is horrible. I am moving things out of lib
sometime next week, and making new directories for them.
> I don't see an argument for base64 encoding, though. I would disagree with
> moving that into APR.
Fine. I'm not going to argue about this anymore.
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
Re: Encoding code in APR
Posted by Greg Stein <gs...@lyra.org>.
On Thu, Jul 06, 2000 at 08:40:39AM -0700, rbb@covalent.net wrote:
>
> I thought of another reason why the md5/sha1 encoding code needs to be in
> APR. It may be VERY portable code, but the results aren't portable. I
> know that makes very little sense. Think about why we decided to always
> use the Apache MD5 code on all platforms. We got the code originally from
> FreeBSD, so we know they have the code, we could have used the crypt
> function on FreeBSD.
>
> If you review the archives, you'll discover that later versions of FreeBSD
> modified their version of crypt to produce different results. They are a
> bit more secure IIRC, but the results are also just not portable. This
> means that if we don't put the encoding stuff into APR, and somebody uses
> an MD5 algorithm on a FreeBSD box, that code will not be portable to a
> Windows machine using APR with the lib/util MD5 algorithm.
You're mixing your arguments here.
*) crypt() is non-portable
*) the MD5 hash algorithm is totally portable
The problem that you refer to is trying to use crypt() to do MD5 hashing.
That just doesn't work.
>...
> With this argument and the argument laid out yesterday about not all
> platforms having encoding API's, does anybody still have a problem with
> creating lib/apr/encode, and putting ap_(md5/sha1/base64) there?
*) the above argument doesn't hold
*) not all platforms have these hash algorithms, but that doesn't matter --
we already said that we will never attempt to rely on the platform's hash
functions (we have our own, so use it)
*) MD5 and SHA1 are *hash* algorithms. not encodings. base64 is an encoding.
All that said, I have discovered a reason for MD5 to be in APR, and it has
nothing to do with the above :-) ... APR needs to do UUID generation which
is non-portable. Win32 has UuidCreate(), but other platforms need to roll an
algorithm. Part of the UUID is random data. This can be fetched using
something like the "truerand" library (which is already in APR?). But when a
good random number generator isn't present, then a great way to generate
random bytes is to apply the MD5 hash to a block of data. By definition, the
result is random :-) ... in any case, this means that APR itself will have a
dependency upon MD5 hashing. Therefore, it needs to include it.
May as well toss the SHA1 hash in there. The two files would simply go into
apr/lib/.
I don't see an argument for base64 encoding, though. I would disagree with
moving that into APR.
Cheers,
-g
--
Greg Stein, http://www.lyra.org/
Encoding code in APR (Was: Re: hooks)
Posted by rb...@covalent.net.
I thought of another reason why the md5/sha1 encoding code needs to be in
APR. It may be VERY portable code, but the results aren't portable. I
know that makes very little sense. Think about why we decided to always
use the Apache MD5 code on all platforms. We got the code originally from
FreeBSD, so we know they have the code, we could have used the crypt
function on FreeBSD.
If you review the archives, you'll discover that later versions of FreeBSD
modified their version of crypt to produce different results. They are a
bit more secure IIRC, but the results are also just not portable. This
means that if we don't put the encoding stuff into APR, and somebody uses
an MD5 algorithm on a FreeBSD box, that code will not be portable to a
Windows machine using APR with the lib/util MD5 algorithm.
By putting the MD5 algorithm in APR, we remove that problem, and can
provide a common interface that always implements the same alpgorithm
across all platforms.
With this argument and the argument laid out yesterday about not all
platforms having encoding API's, does anybody still have a problem with
creating lib/apr/encode, and putting ap_(md5/sha1/base64) there?
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
Re: hooks
Posted by rb...@covalent.net.
> > The EBCDIC stuff is still in ap_base64.c, but that is another issue.
>
> That has to change anyway to use ap_xlate.
Agreed.
> As has been said, these are completely portable functions across any
> platform/compiler/etc, with the exception of the non-binary versions
> of update - those need to use ap_xlate, at which point the stuff
> becomes fully portable.
The problem is what do we mean by "portable". Yes, the C code is
portable. The problem is that there isn't currently an encoding function
on all platforms. If we take the stance that any inheritantly portable
code does not belong in APR, then we basically have to remove all of the
apr/lib directory. All of that code is inherantly portable. But wait,
look at what that code is.
pools -- can't remove those, APR relies on those.
string functions -- things like ap_pstrdup and ap_pstrncpy. We re-wrote
those to be more portable, and to make more
sense. According to the definition that is keeping
the encoding stuff out of APR, these aren't portable
functions.
signals -- not all platforms used the same API or had the same
semantics. We re-implemented to make it all common.
apr_getopt (from misc) -- all platforms didn't have the same semantics,
we re-implemented.
APR wasn't designed to provide only functions that couldn't be written
once for all platforms. It was designed to provide a common API for all
platforms. Not all platforms have crypt, not all platforms have md5, not
all platforms have sha1. Shoot, Windows doesn't have any of them.
These belong in APR, because they implement the same feature on all
platforms. That makes them portable, and it makes them a candidate for a
portability run-time. This does not make APR a kitchen-sink, it doesn't
even expand APR's design goals.
If we want APR to decide on one encoding algorithm, and use it on all
platforms, then fine, put one encoding algorithm in APR and the rest in
lib/util. This sounds stupid to me though. APR needs a way to encode
data on all platforms without relying on another library. Unless crypt is
available on all platforms (it isn't), then this is something APR should
provide.
> Btw., does anybody disagree with changing the update functions in
> md5 and sha1 to ap_MD5Update_ascii(), ap_MD5Update_binary(),
> ap_SHA1Update_ascii(), and ap_SHA1Update_binary()? I.e. to better
> reflect which are ascii and which binary (the ascii forms first
> translate the characters as necessary to ascii). And similarly
> for ap_base64. Currently we have ap_SHA1Update() and
> ap_SHA1Update_binary() in 1.3.
Nope.
> I agree. This is boiling down to the previous (and I think undecided)
> discussion of what should go into APR. I have nothing against putting
> md5, sha1, and base64 into APR - I'm just noting that I think your
> portability argument isn't valid.
I think I explained the portability issue a bit better above. Do you
think it's valid and makes sense now?
> I quite aware of what ap_checkpass contains :-)
> A number of changes were made in the 1.3 code base last July, but those
> changes never got brought forward to 2.0 (the snapshot was taken just
> before). As part of the move I intended to bring those changes forward.
> Btw., apr_validate.c is too generic, IMHO - validate what?
> apr_checkpass.c is clearer. Or apr_validate_pw.c . But that's a nit.
Names are a VERY minor thing to me. I usually don't come up with the
right one, so choose whatever you want. I wasn't aware that there were
changes made to validate_passwd in 1.3 that weren't in 2.0. By all means,
this needs to be updated. From your original note it looked like we were
about to duplicate some logic. :-)
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
Re: hooks
Posted by "Life is hard, and then you die" <ro...@innovation.ch>.
On Tue, Jul 04, 2000 at 05:10:26PM -0700, rbb@covalent.net wrote:
> > > I would still really like to see all of these files in APR. They all use
> > > APR types, and although the code is all common for all platforms, they are
> > > portability issues for most programs. By moving them into different
> > > directories within APR, we can enable and disable them at compile time.
> >
> > What portability issues? The ebcdic stuff is now in apr (ap_xlate, or so
> > I assume), and I can't think of any others.
>
> The EBCDIC stuff is still in ap_base64.c, but that is another issue.
That has to change anyway to use ap_xlate.
> I was mainly thinking of the MD5 stuff with regards to portability,
> because some platforms have it and others don't. We actually took our
> code from FreeBSD, so we know they have this function.
As has been said, these are completely portable functions across any
platform/compiler/etc, with the exception of the non-binary versions
of update - those need to use ap_xlate, at which point the stuff
becomes fully portable.
Btw., does anybody disagree with changing the update functions in
md5 and sha1 to ap_MD5Update_ascii(), ap_MD5Update_binary(),
ap_SHA1Update_ascii(), and ap_SHA1Update_binary()? I.e. to better
reflect which are ascii and which binary (the ascii forms first
translate the characters as necessary to ascii). And similarly
for ap_base64. Currently we have ap_SHA1Update() and
ap_SHA1Update_binary() in 1.3.
> The other argument is that this is a bunch of VERY portable encryption
> code. Anybody who is trying to write a secure piece of portable software
> would most likely like to use some of these functions. Isn't this what
> APR is supposed to provide? The ability to write very portable software
> with one library. We can put all of these functions in lib/apr/encrypt or
> lib/apr/enocde (may be more precise) and be able to enable/disable them at
> compile time. I really think these are useful portable routines.
I agree. This is boiling down to the previous (and I think undecided)
discussion of what should go into APR. I have nothing against putting
md5, sha1, and base64 into APR - I'm just noting that I think your
portability argument isn't valid.
> > > At the very least getpass is a portability issue, and it doesn't always
> > > have common code, so it definately belongs in APR.
> >
> > Sorry, confused getpass with checkpass (which needs to be brought in from
> > apache-1.3) - you're right, getpass shouldn't be touched.
>
> ap_checkpass is validate_password, which can be found in
> apr/lib/apr_md5.c. :-) See above for why this function shouldn't be
> moved out of APR. It should be moved to apr/encode/apr_validate.c IMHO
> though.
I quite aware of what ap_checkpass contains :-)
A number of changes were made in the 1.3 code base last July, but those
changes never got brought forward to 2.0 (the snapshot was taken just
before). As part of the move I intended to bring those changes forward.
Btw., apr_validate.c is too generic, IMHO - validate what?
apr_checkpass.c is clearer. Or apr_validate_pw.c . But that's a nit.
Cheers,
Ronald
RE: hooks
Posted by "William A. Rowe, Jr." <wr...@lnd.com>.
> From: Life is hard, and then you die [mailto:ronald@innovation.ch]
> Sent: Tuesday, July 04, 2000 6:51 PM
>
> On Tue, Jul 04, 2000 at 04:10:21PM -0700, rbb@covalent.net wrote:
> >
> > > If everybody is happy with lib/utils then I'll attempt to
> > > beat you to it for the md5, sha1, base64, getpass, etc.
> >
> > I would still really like to see all of these files in APR. They all use
> > APR types, and although the code is all common for all platforms, they are
> > portability issues for most programs. By moving them into different
> > directories within APR, we can enable and disable them at
> compile time.
>
> What portability issues? The ebcdic stuff is now in apr (ap_xlate, or so
> I assume), and I can't think of any others. Last time I asked about
> this, Greg was against moving them into apr and other-Bill was for it -
> other than that nobody cared. I'm equally split on whether to put it in
> apr or lib/utils.
Ok... APR being APR, and we had very, very little in the way of ap code
left, I strongly felt that should move. I believed it should become part
of APR.
In hindsite, I realize why many were against this. And I think, given
what everyone is suggesting for all the "module helper" code, that there
is room for a lib/aputil style library. The existing ap stuff belongs
there, and so should all the xml stuff.
The difference? No lib/aputil stuff should EVER change from processor
to processor, compiler to compiler, or os to os.
If it isn't that generic, then it's a portability problem (al la APR).
And if it is truly platform process or security model dependent, than
it belongs in the MPM or the potential future SMM.
Bill
Re: hooks
Posted by rb...@covalent.net.
> > > If everybody is happy with lib/utils then I'll attempt to beat you to
> > > it for the md5, sha1, base64, getpass, etc.
> >
> > I would still really like to see all of these files in APR. They all use
> > APR types, and although the code is all common for all platforms, they are
> > portability issues for most programs. By moving them into different
> > directories within APR, we can enable and disable them at compile time.
>
> What portability issues? The ebcdic stuff is now in apr (ap_xlate, or so
> I assume), and I can't think of any others.
The EBCDIC stuff is still in ap_base64.c, but that is another issue.
I was mainly thinking of the MD5 stuff with regards to portability,
because some platforms have it and others don't. We actually took our
code from FreeBSD, so we know they have this function.
The other argument is that this is a bunch of VERY portable encryption
code. Anybody who is trying to write a secure piece of portable software
would most likely like to use some of these functions. Isn't this what
APR is supposed to provide? The ability to write very portable software
with one library. We can put all of these functions in lib/apr/encrypt or
lib/apr/enocde (may be more precise) and be able to enable/disable them at
compile time. I really think these are useful portable routines.
> Last time I asked about
> this, Greg was against moving them into apr and other-Bill was for it -
> other than that nobody cared. I'm equally split on whether to put it in
> apr or lib/utils.
Count me on the side of APR. :-) I believe Manoj is against moving them
into APR as well, but I don't want to speak for him. :-)
> > At the very least getpass is a portability issue, and it doesn't always
> > have common code, so it definately belongs in APR.
>
> Sorry, confused getpass with checkpass (which needs to be brought in from
> apache-1.3) - you're right, getpass shouldn't be touched.
ap_checkpass is validate_password, which can be found in
apr/lib/apr_md5.c. :-) See above for why this function shouldn't be
moved out of APR. It should be moved to apr/encode/apr_validate.c IMHO
though.
> > Which things are included in the etc? I would like to be VERY careful
> > about what we move out of APR, because some of the functions in the lib
> > directory are portability issues, and not cruft for APR.
>
> Etc referred to stuff in ap and main: ap/ap_cache, ap/ap_hooks, and the
> main/util* stuff.
This stuff I would like to see moved to lib/util. This is code that is
very suited for Apache, but isn't really written to be useful to things
other than web servers. That is where I see the biggest
difference.
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
Re: hooks
Posted by "Life is hard, and then you die" <ro...@innovation.ch>.
On Tue, Jul 04, 2000 at 04:10:21PM -0700, rbb@covalent.net wrote:
>
> > > .... I keep meaning to move files around and modify
> > > Makefiles to untie these, but other things keep coming up. I think we
> > > need modules/protocol/echo and modules/protocol/http directories. I also
> > > would like to see lib/utils. I plan on doing some of this work this week
> > > unless somebody beats me to it. :-)
> >
> > If everybody is happy with lib/utils then I'll attempt to beat you to
> > it for the md5, sha1, base64, getpass, etc.
>
> I would still really like to see all of these files in APR. They all use
> APR types, and although the code is all common for all platforms, they are
> portability issues for most programs. By moving them into different
> directories within APR, we can enable and disable them at compile time.
What portability issues? The ebcdic stuff is now in apr (ap_xlate, or so
I assume), and I can't think of any others. Last time I asked about
this, Greg was against moving them into apr and other-Bill was for it -
other than that nobody cared. I'm equally split on whether to put it in
apr or lib/utils.
> At the very least getpass is a portability issue, and it doesn't always
> have common code, so it definately belongs in APR.
Sorry, confused getpass with checkpass (which needs to be brought in from
apache-1.3) - you're right, getpass shouldn't be touched.
> Which things are included in the etc? I would like to be VERY careful
> about what we move out of APR, because some of the functions in the lib
> directory are portability issues, and not cruft for APR.
Etc referred to stuff in ap and main: ap/ap_cache, ap/ap_hooks, and the
main/util* stuff.
Cheers,
Ronald
Re: hooks
Posted by Greg Stein <gs...@lyra.org>.
On Tue, Jul 04, 2000 at 03:51:00PM -0700, Life is hard, and then you die wrote:
> On Tue, Jul 04, 2000 at 02:57:08PM -0700, rbb@covalent.net wrote:
> >
> > .... I keep meaning to move files around and modify
> > Makefiles to untie these, but other things keep coming up. I think we
> > need modules/protocol/echo and modules/protocol/http directories. I also
> > would like to see lib/utils. I plan on doing some of this work this week
> > unless somebody beats me to it. :-)
>
> If everybody is happy with lib/utils then I'll attempt to beat you to
> it for the md5, sha1, base64, getpass, etc.
+1
--
Greg Stein, http://www.lyra.org/
Re: hooks
Posted by rb...@covalent.net.
> > .... I keep meaning to move files around and modify
> > Makefiles to untie these, but other things keep coming up. I think we
> > need modules/protocol/echo and modules/protocol/http directories. I also
> > would like to see lib/utils. I plan on doing some of this work this week
> > unless somebody beats me to it. :-)
>
> If everybody is happy with lib/utils then I'll attempt to beat you to
> it for the md5, sha1, base64, getpass, etc.
I would still really like to see all of these files in APR. They all use
APR types, and although the code is all common for all platforms, they are
portability issues for most programs. By moving them into different
directories within APR, we can enable and disable them at compile time.
At the very least getpass is a portability issue, and it doesn't always
have common code, so it definately belongs in APR.
Which things are included in the etc? I would like to be VERY careful
about what we move out of APR, because some of the functions in the lib
directory are portability issues, and not cruft for APR.
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
Re: hooks
Posted by "Life is hard, and then you die" <ro...@innovation.ch>.
On Tue, Jul 04, 2000 at 02:57:08PM -0700, rbb@covalent.net wrote:
>
> .... I keep meaning to move files around and modify
> Makefiles to untie these, but other things keep coming up. I think we
> need modules/protocol/echo and modules/protocol/http directories. I also
> would like to see lib/utils. I plan on doing some of this work this week
> unless somebody beats me to it. :-)
If everybody is happy with lib/utils then I'll attempt to beat you to
it for the md5, sha1, base64, getpass, etc.
Cheers,
Ronald
Re: hooks
Posted by rb...@covalent.net.
> > All of this is fine if the abstraction of protocols is to give an easy way
> > of plugging in an all new monolithic module that incorporates all the stuff
> > for another protocol (ala the apache core stuff). If it is there to provide
> > the ability to have a very generic framework and I can easily plug in
> > different protocols and those different protocol modules can use some
> > common modules to do certain aspects then it seems what exists isn't going
> > to work well.
>
> Catching up with this rather late, but I really can't see why you think
> this. New protocol modules can call the hooks just like old protocols do
> - how does that not work well?
>
> And like Ryan said - give a real-world example of how this doesn't work,
> and we'll figure out how to make it better.
I think we have convinced him that this is true, but Sam does bring up an
interesting point. Currently, the http protocol module is tied into
Apache very tightly. I keep meaning to move files around and modify
Makefiles to untie these, but other things keep coming up. I think we
need modules/protocol/echo and modules/protocol/http directories. I also
would like to see lib/utils. I plan on doing some of this work this week
unless somebody beats me to it. :-)
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
Re: hooks
Posted by Ben Laurie <be...@algroup.co.uk>.
Sam Magnuson wrote:
> All of this is fine if the abstraction of protocols is to give an easy way
> of plugging in an all new monolithic module that incorporates all the stuff
> for another protocol (ala the apache core stuff). If it is there to provide
> the ability to have a very generic framework and I can easily plug in
> different protocols and those different protocol modules can use some
> common modules to do certain aspects then it seems what exists isn't going
> to work well.
Catching up with this rather late, but I really can't see why you think
this. New protocol modules can call the hooks just like old protocols do
- how does that not work well?
And like Ryan said - give a real-world example of how this doesn't work,
and we'll figure out how to make it better.
Cheers,
Ben.
--
http://www.apache-ssl.org/ben.html
Coming to ApacheCon Europe 2000? http://apachecon.com/
RE: hooks
Posted by "William A. Rowe, Jr." <wr...@lnd.com>.
> From: rbb@covalent.net [mailto:rbb@covalent.net]
> Sent: Tuesday, June 20, 2000 9:24 PM
>
> > Sorry for confusion, when I said it needs to include AP_* I really meant the
> > stuff generated BY the AP_* macros :)
> >
> > Anyway.. you are right the ap_run_(hook_name) symbol must be exported so
> > that the module can call it however..... ap_run_* uses _hooks, _hooks comes
> > from AP_HOOK_STRUCT, which is static, which isn't exported to the module, I
> > guess this is where my confusion comes from. I am very probably wrong - or
> > misunderstand something about windows (or AIX) linker, but I certainly
> > thought this was a no-no.
>
> Nope, this is fine to do. If we really want to, it would be very possible
> to export the ap_run_* functions, and call them from anywhere. In fact,
> if you look at the Windows .def file, we explicitly export the
> ap_run_pre_config for http_main.c. This is done in some magical way that
> I don't understand so that just http_main.c can use it (OtherBill, could
> you clear this up?)
Yes... if you look -deep- inside the macros, they now have the API_EXPORT...
stuff as appropriate, so no, all hook calls are acceptable for dynamic linkage,
as is the ap_run_ config stuff.
Let me try to answer Ryan's question. We will no longer build http_main.c into
the ApacheCore.dll (our so version of Apache on Win32.) We used to have a very
silly thunk that would simply call something called apache_main, and that was
nonesense. We really -never- again call http_main.c and it's functions.
So http_main.c is a static module, that dynamically binds in the entire
Apache server engine. Simple, really :)
Re: hooks
Posted by rb...@covalent.net.
> Sorry for confusion, when I said it needs to include AP_* I really meant the
> stuff generated BY the AP_* macros :)
>
> Anyway.. you are right the ap_run_(hook_name) symbol must be exported so
> that the module can call it however..... ap_run_* uses _hooks, _hooks comes
> from AP_HOOK_STRUCT, which is static, which isn't exported to the module, I
> guess this is where my confusion comes from. I am very probably wrong - or
> misunderstand something about windows (or AIX) linker, but I certainly
> thought this was a no-no.
Nope, this is fine to do. If we really want to, it would be very possible
to export the ap_run_* functions, and call them from anywhere. In fact,
if you look at the Windows .def file, we explicitly export the
ap_run_pre_config for http_main.c. This is done in some magical way that
I don't understand so that just http_main.c can use it (OtherBill, could
you clear this up?)
> This is definetly a good goal, however adding to the hooking that was in
> 1.3 so it had order fu in the API might have been a good thing, if for no
> other reason to ease migration from 1.3. As is obvious I've come along (to
> hacking on apache) late in the 2.0 time - so I may be missing key
> discussions, I will go to an archive and start perusing, but it seems the
> old API had some very good features (minus the ordering troubles)..
Adding to the 1.3 hooking would have been hard and ugly. The current
implementation is relatively easy to understand and clean. Also, as
somebody who has ported more than his fair share of standard modules, I
can say that the port is very straight forward and easy to do.
> I'm sorry - what I was referring to was suppose there was a foobar hook
> (ap_hook_foobar) and it is used by the core, then I write a module and I
> want to go behind the core's back (for some reason) and trigger a foobar
> (ap_run_foobar) too, to do this I would either
>
> a) have to hack up something (as you said big hack above) to do this, but
> supposedly it's "bad"
> b) have a new hook, and any module that wants to do foobar for my protocol,
> or http would have to know about both and be sure to hook both.
-Or-
c) Change the linkage for ap_run_foobar, so that anybody can run
it. This is not impossible to do, we just haven't had a request for
it yet with a valid reason behind it. What I assumed you wanted to
do was have the core implement ap_run_foobar, and mod_baz implement
ap_run_baz, and everytime a module registered for foobar, it
automagically registered for baz. That requires a hack. Letting
mod_baz call mod_foobar requires a good case for it to be made, and
the linkage to be changed. I do not see this as an insurmountable
issue at all.
> I'm looking at it more from a standpoint of "Why are we limiting it in this
> way?", it is also reasonable to decide these limitations are acceptable -
> the original questions where asked while I was still in throught process,
> explanation below (1)...
We are limiting this way because it is an easy place to draw the
line. Prove that the line should be moved, and it will be. :-)
> (1) Ok real world has to do with....
>
> I was doing some thinking about protocol modules. I was thinking the http
> stuff could be split into a separate module, and we can have a generic
> server framework (as most of the work is really there and this would just
> be tying up loose ends). I was thinking about the next protocol modules
> that might come along, and thought there were a bunch of useful modules for
> the http side of things, perhaps new protocol will have a use for
> them. Then it started to occur to me the hooking mechanisms probably aren't
> going to be sufficient for inter-module hook relay, to boot it wouldn't
> really provide new protocols with a way to use hooks that exist for http
> things.
Actually, a lot of this work has been done. Mod_echo is a good example of
a protocol module that does work with the current server. Somebody else
on the list has recently been posting that he too has written his own
protocol module. I can't remember right now what protocol it was. I
suspect that the multi-protocol stuff will change drastically at some
point in the future. I have been toying with the idea of writing a
compiler server (I'm a sick man). The compiler server would take C code
and return a binary for whatever platform you asked for (using
cross-compiling). So, the server framework idea has some people who want
to make it work. I also know that right now, I am working far too many
hours on Apache 2.0, and I am unlikely to work on the framework side until
2.0 is released. I expect many others feel the same way, because we want
to see 2.0 a reality soon.
> All of this is fine if the abstraction of protocols is to give an easy way
> of plugging in an all new monolithic module that incorporates all the stuff
> for another protocol (ala the apache core stuff). If it is there to provide
> the ability to have a very generic framework and I can easily plug in
> different protocols and those different protocol modules can use some
> common modules to do certain aspects then it seems what exists isn't going
> to work well.
I think it will work just fine once we change some linkages. I really
don't think this is a big issue. If it is, we just fix it for 2.1 or
3.0. But, I also think throwing away a very good hook mechanism because
it isn't perfect yet is a bit much. Take a look at the ap_run_pre_config
and ap_run_open_logs stuff, and tell me if that is close to what you want
to do.
> I understand 2.0 might not have all that in it, but since 2.0 is new enough
> not to have all modules ported to it, I figured it may be worthwhile coming
> up with something (the more I look at it the more the 1.3 hooking facility
> seems to be it) to correct these hooking issues.
I think you would be surprised how many modules have been ported to 2.0
already. The 1.3 hooking facility has some major drawbacks beyond just
the ordering. Whenever a new hook is added (much more likely with more
protocol modules) every module needs to be re-compiled and possibly
modified. This is a big hit for some people. Some people don't have the
time/desire to keep up with Apache development. Some companies have
modules that are not released as source. The new hook mechanism allows
Apache 2.0 to not have to bump the MMN whenever a new hook is added. I
suggest playing with it a bit more. I think you will find that it can do
everything you want it to, it just needs a littel carressing and fine
tuning.
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
Re: hooks
Posted by Sam Magnuson <sm...@aventail.com>.
rbb@covalent.net writes:
>
> I believe you are missing something. :-) I assume you are talking now
> about a module that wants to provide hooks for other modules to register
> for.
>
> The AP_IMPLEMENT_HOOK is a macro, and it will never need to be exposed
> through a .def or .exp file. The symbol that is exposed is the generated
> symbol. This means, for example, that ap_hook_check_user_id is exposed,
> but not AP_IMPLEMENT_HOOK (which is the macro that generates the
> ap_hook_check_user_id function).
>
> AP_HOOK_STRUCT is also not exposed, because it is a static variable. It
> took me a long time to see how this all fits together, but here it goes:
>
> when you declare AP_HOOK_STRUCT at the top of a file, that defines:
>
> static struct {
> ap_array_header_t link_(first_hook_name)
> ap_array_header_t link_(second_hook_name)
> ap_array_header_t link_(nth_hook_name)
>
> }
>
> Then, at the bottom of the file you use the AP_IMPLEMENT_HOOK_* macros to
> actually generate the relevant functions. There are two functions
> generated. The first basically add a function pointer to the hook, and
> the second loops through the hook calling the functions. The first is
> ap_hook_(hook_name) and the second is ap_run_(hook_name). The first is
> public and needs to be put into the .def or .exp file, but the second is
> static and should not be exposed.
>
Sorry for confusion, when I said it needs to include AP_* I really meant the
stuff generated BY the AP_* macros :)
Anyway.. you are right the ap_run_(hook_name) symbol must be exported so
that the module can call it however..... ap_run_* uses _hooks, _hooks comes
from AP_HOOK_STRUCT, which is static, which isn't exported to the module, I
guess this is where my confusion comes from. I am very probably wrong - or
misunderstand something about windows (or AIX) linker, but I certainly
thought this was a no-no.
> The final piece to all of this is the AP_DECLARE_HOOK that goes into the
> header file. This just declares a couple of prototypes, and sets up the
> LINK_(hook_name) structure to be used for inserting new hooks.
>
> This should explain how things are shared or not shared in a relatively
> straight forward manner. Please if this doesn't help, let me know. Ben,
> if I have this completely wrong, please correct me.
>
This does help, I have a doc sitting around from a gcc -E of one of these
just so I can refer to it when I get lost in the macro evilness :)
> > Yes the programmer needs to decide where it needs to be hooked, but since
> > we're using hooks instead of single function pointers, one can assume how
> > the call is directed to be transparent.
>
> It would really help me out if you could give an example of what you are
> thinking here. I am having a VERY hard time parsing this.
>
I apologize, I will clarify below (1)..
> > But this hooking mechanism seems to be less than useful when modules want
> > to create hooks for other modules. I guess what I'm wondering is why it has
> > changed so significantly from 1.3?
>
> If a module creates a hook for other modules, then this implies that the
> first module is always in the server. This is just not the case with
> Apache, and we do not code for that case. For example, say mod_foo
> implements a hook and mod_bar registers a function for it. if mod_foo is
> not in the server, then mod_bar has no place to be registered. This is a
> bad thing.
>
> It has changed so siginificantly from 1.3, because in 2.0 we didn't want
> the order of the hooks to be dependant on the module. The hooks needed to
> be independant of each other.
>
This is definetly a good goal, however adding to the hooking that was in
1.3 so it had order fu in the API might have been a good thing, if for no
other reason to ease migration from 1.3. As is obvious I've come along (to
hacking on apache) late in the 2.0 time - so I may be missing key
discussions, I will go to an archive and start perusing, but it seems the
old API had some very good features (minus the ordering troubles)..
> > Yes I guess I mean the latter, if a module came along and wanted to reuse a
> > hook for something "similar" to what the hook was originally for - I'm not
> > sure why that would be a bad thing, worse is having each module that
> > supports foobar needs to have ap_hook_this_foobar and ap_hook_that_foobar
> > to get itself to work right..
>
> First of all, because the _hooks struct (declared by AP_HOOK_STRUCT) is a
> static, I take it back. It is not possible to share a hook between two
> modules.
>
Exactly.
> But why are multiple modules supporting foobar? If your module needs the
> same function that the core implements, don't duplicate the core's work,
> take advantage of it. Throw your function in as a REALLY_LAST function
> and use what the other modules have already done.
>
I'm sorry - what I was referring to was suppose there was a foobar hook
(ap_hook_foobar) and it is used by the core, then I write a module and I
want to go behind the core's back (for some reason) and trigger a foobar
(ap_run_foobar) too, to do this I would either
a) have to hack up something (as you said big hack above) to do this, but
supposedly it's "bad"
b) have a new hook, and any module that wants to do foobar for my protocol,
or http would have to know about both and be sure to hook both.
I'm looking at it more from a standpoint of "Why are we limiting it in this
way?", it is also reasonable to decide these limitations are acceptable -
the original questions where asked while I was still in throught process,
explanation below (1)...
> What this really comes down to, is we need a real-world example of what
> you are trying to do before we can even hope to help. My problem is that
> I don't really see what you are trying. William Wrowe just responded
> saying that a patch to allow some of what you are trying was shot
> down. It was shot down because nobody saw the need.
>
> Apache takes a very hard line against just including function for the sake
> of including function. If there is a real need for an ability (the
> ability to have a module declare it's own hooks, for example), the group
> is perfectly willing to help implement it. If it is just something that
> sounds like it could be useful, then we don't in general implement
> it. Give a real-world example of what you want, and we can and will work
> with you to make it happen.
>
(1) Ok real world has to do with....
I was doing some thinking about protocol modules. I was thinking the http
stuff could be split into a separate module, and we can have a generic
server framework (as most of the work is really there and this would just
be tying up loose ends). I was thinking about the next protocol modules
that might come along, and thought there were a bunch of useful modules for
the http side of things, perhaps new protocol will have a use for
them. Then it started to occur to me the hooking mechanisms probably aren't
going to be sufficient for inter-module hook relay, to boot it wouldn't
really provide new protocols with a way to use hooks that exist for http
things.
All of this is fine if the abstraction of protocols is to give an easy way
of plugging in an all new monolithic module that incorporates all the stuff
for another protocol (ala the apache core stuff). If it is there to provide
the ability to have a very generic framework and I can easily plug in
different protocols and those different protocol modules can use some
common modules to do certain aspects then it seems what exists isn't going
to work well.
I understand 2.0 might not have all that in it, but since 2.0 is new enough
not to have all modules ported to it, I figured it may be worthwhile coming
up with something (the more I look at it the more the 1.3 hooking facility
seems to be it) to correct these hooking issues.
//Sam
> Ryan
>
> _______________________________________________________________________________
> Ryan Bloom rbb@apache.org
> 406 29th St.
> San Francisco, CA 94131
> -------------------------------------------------------------------------------
>
>
--
Sam Magnuson <sm...@aventail.com>
Re: hooks
Posted by rb...@covalent.net.
> > > What is the reason we have hooks (2.0) based on function name (ala
> > > ap_hook_fudge, ap_run_fudge) rather than some key (ie
> > > ap_add_callback(fudge_hook,...), ap_call_hook(fudge_hook))? I'm sure there
> >
> > Because each hook has a different prototype and we need to be sure that
> > modules use the correct prototype. Having a single function that just
> > calls the right stuff under the covers means that we can't do real
> > argument checking at build time.
>
> Type safety at build time can be a good thing, but if its at the cost of
> having modules exposing hooks to other modules it might not really be worth
> it. Ie, how will linking issues be solved in Windows (or AIX) where all
> symbols must be resolved in the dll (or .so)? It seems now the mod_* will
> need to have the AP_IMPLEMENT_HOOK_* and AP_HOOK_STRUCT in its shared
> library, or am I missing something?
I believe you are missing something. :-) I assume you are talking now
about a module that wants to provide hooks for other modules to register
for.
The AP_IMPLEMENT_HOOK is a macro, and it will never need to be exposed
through a .def or .exp file. The symbol that is exposed is the generated
symbol. This means, for example, that ap_hook_check_user_id is exposed,
but not AP_IMPLEMENT_HOOK (which is the macro that generates the
ap_hook_check_user_id function).
AP_HOOK_STRUCT is also not exposed, because it is a static variable. It
took me a long time to see how this all fits together, but here it goes:
when you declare AP_HOOK_STRUCT at the top of a file, that defines:
static struct {
ap_array_header_t link_(first_hook_name)
ap_array_header_t link_(second_hook_name)
ap_array_header_t link_(nth_hook_name)
}
Then, at the bottom of the file you use the AP_IMPLEMENT_HOOK_* macros to
actually generate the relevant functions. There are two functions
generated. The first basically add a function pointer to the hook, and
the second loops through the hook calling the functions. The first is
ap_hook_(hook_name) and the second is ap_run_(hook_name). The first is
public and needs to be put into the .def or .exp file, but the second is
static and should not be exposed.
The final piece to all of this is the AP_DECLARE_HOOK that goes into the
header file. This just declares a couple of prototypes, and sets up the
LINK_(hook_name) structure to be used for inserting new hooks.
This should explain how things are shared or not shared in a relatively
straight forward manner. Please if this doesn't help, let me know. Ben,
if I have this completely wrong, please correct me.
> > The fact is that an Apache Module will ALWAYS be dependant on which Apache
> > it is running on. It is unrealistic to think that a module can figure out
> > where it should add a hook at run-time. Those kinds of decisions need to
> > be made by the programmer. For example, if I have a module that provides
> > the auth_checker_hook, how would you propose that module would determine
> > which hook to use?
>
> Yes the programmer needs to decide where it needs to be hooked, but since
> we're using hooks instead of single function pointers, one can assume how
> the call is directed to be transparent.
It would really help me out if you could give an example of what you are
thinking here. I am having a VERY hard time parsing this.
> But this hooking mechanism seems to be less than useful when modules want
> to create hooks for other modules. I guess what I'm wondering is why it has
> changed so significantly from 1.3?
If a module creates a hook for other modules, then this implies that the
first module is always in the server. This is just not the case with
Apache, and we do not code for that case. For example, say mod_foo
implements a hook and mod_bar registers a function for it. if mod_foo is
not in the server, then mod_bar has no place to be registered. This is a
bad thing.
It has changed so siginificantly from 1.3, because in 2.0 we didn't want
the order of the hooks to be dependant on the module. The hooks needed to
be independant of each other.
> > Are you saying the proxy implements a hook that closely approximates what
> > the server is doing, and you want all of the modules that registered for
> > the server's hook to also register for yours? This is IMHO a bad thing to
> > do, but it is possible to do it. It requires a big hack, because it a bad
> > thing.
> Yes I guess I mean the latter, if a module came along and wanted to reuse a
> hook for something "similar" to what the hook was originally for - I'm not
> sure why that would be a bad thing, worse is having each module that
> supports foobar needs to have ap_hook_this_foobar and ap_hook_that_foobar
> to get itself to work right..
First of all, because the _hooks struct (declared by AP_HOOK_STRUCT) is a
static, I take it back. It is not possible to share a hook between two
modules.
But why are multiple modules supporting foobar? If your module needs the
same function that the core implements, don't duplicate the core's work,
take advantage of it. Throw your function in as a REALLY_LAST function
and use what the other modules have already done.
What this really comes down to, is we need a real-world example of what
you are trying to do before we can even hope to help. My problem is that
I don't really see what you are trying. William Wrowe just responded
saying that a patch to allow some of what you are trying was shot
down. It was shot down because nobody saw the need.
Apache takes a very hard line against just including function for the sake
of including function. If there is a real need for an ability (the
ability to have a module declare it's own hooks, for example), the group
is perfectly willing to help implement it. If it is just something that
sounds like it could be useful, then we don't in general implement
it. Give a real-world example of what you want, and we can and will work
with you to make it happen.
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
Re: hooks
Posted by Sam Magnuson <sm...@aventail.com>.
rbb@covalent.net writes:
> > What is the reason we have hooks (2.0) based on function name (ala
> > ap_hook_fudge, ap_run_fudge) rather than some key (ie
> > ap_add_callback(fudge_hook,...), ap_call_hook(fudge_hook))? I'm sure there
>
> Because each hook has a different prototype and we need to be sure that
> modules use the correct prototype. Having a single function that just
> calls the right stuff under the covers means that we can't do real
> argument checking at build time.
>
Type safety at build time can be a good thing, but if its at the cost of
having modules exposing hooks to other modules it might not really be worth
it. Ie, how will linking issues be solved in Windows (or AIX) where all
symbols must be resolved in the dll (or .so)? It seems now the mod_* will
need to have the AP_IMPLEMENT_HOOK_* and AP_HOOK_STRUCT in its shared
library, or am I missing something?
> > is some good reason this has been done, but more than once I've needed to
> > remind myself what the macro generated (ick) functions were going to look
> > like, seems if we had a single function that dispatched these hooks it
> > would be easier to maintain/lookup.
>
> The macro generated functions are ugly, but IMHO they are better than use
> having to duplicate code over and over again to do the same basic thing
> with different parameters.
>
True - for what it does now this is perfectly reasonable, if only ugly :)
> > It would also seem to be a good design to have _dynamic_ modules use
> > runtime lookups of these hooks, rather than _static_ information. Doing
> > this would remove some dependency of a dynamic module on the apache it's
> > being used with.
>
> I am very confused about what you are asking here, but I'll try to answer
> anyway. :-)
>
> The fact is that an Apache Module will ALWAYS be dependant on which Apache
> it is running on. It is unrealistic to think that a module can figure out
> where it should add a hook at run-time. Those kinds of decisions need to
> be made by the programmer. For example, if I have a module that provides
> the auth_checker_hook, how would you propose that module would determine
> which hook to use?
>
Yes the programmer needs to decide where it needs to be hooked, but since
we're using hooks instead of single function pointers, one can assume how
the call is directed to be transparent.
But this hooking mechanism seems to be less than useful when modules want
to create hooks for other modules. I guess what I'm wondering is why it has
changed so significantly from 1.3?
> > The DECLARE_HOOK stuff is good, however it doesn't lend itself well to
> > having modules (protocol modules?) ap_run_*()'ing hooks (ie I must see an
> > AP_DECLARE_HOOK for fudge, before I can actually bind to it). It also means
> > I can't really "reuse" a hook if I come along with a module to replace the
> > old functionality...
>
> A protocol module should not be trying to ap_run_*() the hooks. Each
> protocol module will need to implement it's own set of hooks. The
> protocol module needs to know about the hooks it is allowing modules to
> implement.
>
> What do you mean you want to "reuse" a hook if you come along with a
> module to replace the old functionality. I assume that since you are
> working on the proxy, this is a proxy question. It would help a lot if
> you could explain exactly what you want to do.
>
Not really a proxy question - just a general one.. :)
> Are you saying you want to basically replace a hook that the server
> implements with one the proxy implements? You can't do that.
>
> Are you saying the proxy implements a hook that closely approximates what
> the server is doing, and you want all of the modules that registered for
> the server's hook to also register for yours? This is IMHO a bad thing to
> do, but it is possible to do it. It requires a big hack, because it a bad
> thing.
>
Yes I guess I mean the latter, if a module came along and wanted to reuse a
hook for something "similar" to what the hook was originally for - I'm not
sure why that would be a bad thing, worse is having each module that
supports foobar needs to have ap_hook_this_foobar and ap_hook_that_foobar
to get itself to work right..
Thanks for the quick reply Ryan - I may be misunderstanding the intent of
the current hook stuff, hence the questions :)
//Sam
> Ryan
>
>
> _______________________________________________________________________________
> Ryan Bloom rbb@apache.org
> 406 29th St.
> San Francisco, CA 94131
> -------------------------------------------------------------------------------
--
Sam Magnuson <sm...@aventail.com>
RE: hooks
Posted by Charles Cutler <ch...@ruadh.com>.
please unsubscribe this user from this group
___________________________________________
Charles Cutler
Ruadh Ltd
Tel - 00 44 (0)1334 654300
charles@ruadh.com
http://www.ruadh.com
-----Original Message-----
From: rbb@covalent.net [mailto:rbb@covalent.net]
Sent: 20 June 2000 23:18
To: new-httpd@apache.org
Subject: Re: hooks
> What is the reason we have hooks (2.0) based on function name (ala
> ap_hook_fudge, ap_run_fudge) rather than some key (ie
> ap_add_callback(fudge_hook,...), ap_call_hook(fudge_hook))? I'm sure there
Because each hook has a different prototype and we need to be sure that
modules use the correct prototype. Having a single function that just
calls the right stuff under the covers means that we can't do real
argument checking at build time.
> is some good reason this has been done, but more than once I've needed to
> remind myself what the macro generated (ick) functions were going to look
> like, seems if we had a single function that dispatched these hooks it
> would be easier to maintain/lookup.
The macro generated functions are ugly, but IMHO they are better than use
having to duplicate code over and over again to do the same basic thing
with different parameters.
> It would also seem to be a good design to have _dynamic_ modules use
> runtime lookups of these hooks, rather than _static_ information. Doing
> this would remove some dependency of a dynamic module on the apache it's
> being used with.
I am very confused about what you are asking here, but I'll try to answer
anyway. :-)
The fact is that an Apache Module will ALWAYS be dependant on which Apache
it is running on. It is unrealistic to think that a module can figure out
where it should add a hook at run-time. Those kinds of decisions need to
be made by the programmer. For example, if I have a module that provides
the auth_checker_hook, how would you propose that module would determine
which hook to use?
> The DECLARE_HOOK stuff is good, however it doesn't lend itself well to
> having modules (protocol modules?) ap_run_*()'ing hooks (ie I must see an
> AP_DECLARE_HOOK for fudge, before I can actually bind to it). It also
means
> I can't really "reuse" a hook if I come along with a module to replace the
> old functionality...
A protocol module should not be trying to ap_run_*() the hooks. Each
protocol module will need to implement it's own set of hooks. The
protocol module needs to know about the hooks it is allowing modules to
implement.
What do you mean you want to "reuse" a hook if you come along with a
module to replace the old functionality. I assume that since you are
working on the proxy, this is a proxy question. It would help a lot if
you could explain exactly what you want to do.
Are you saying you want to basically replace a hook that the server
implements with one the proxy implements? You can't do that.
Are you saying the proxy implements a hook that closely approximates what
the server is doing, and you want all of the modules that registered for
the server's hook to also register for yours? This is IMHO a bad thing to
do, but it is possible to do it. It requires a big hack, because it a bad
thing.
Ryan
____________________________________________________________________________
___
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
----------------------------------------------------------------------------
---
Re: ap_pool support for shared memory?
Posted by rb...@covalent.net.
> > We looked at re-vamping our memory allocation model at the beginning of
> > 2.0, and we never really came to a good conclusion of how to do it
> > right. If you want to submit I patch, I will answer any questions that I
> > can to help.
>
> Thanks. I don't have a patch ready yet, but wanted some feedback on
> whether such a change would possibly be accepted back into the
> codebase.
I think a patch would be VERY welcome. I would suggest that you look at
the archives from about February of last year. You could try the hypermail
archives. Look for the thread on which Dean was talking about using pools
to simulate malloc and shared memory allocation. I can't remember the
thread subject, but it was around February of last year I think.
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
Re: ap_pool support for shared memory?
Posted by David Williams <dw...@cisco.com>.
On Tue, 27 Jun 2000 rbb@covalent.net wrote:
> On Mon, 26 Jun 2000, David Williams wrote:
>
> > I may have a need to use shared memory, but would also like to use the
> > pool routines found in apr_pools.c, since they simplify memory
> > management. The current obstacle being that these routines only call
> > malloc/free. Has any one else had a similiar need, or see reasons
> > they could not be enhanced? I was thinking along the lines of
> > something similiar to:
>
> Take a look at the ap_shm_* functions. They are actually using their own
> pool implementation. I think this will be enough for what you are looking
> for.
These do help, but I would like to have the ability to call existing
functions such as ap_psprint, ap_make_table, ap_palloc, etc. and have
the low-level memory allocation call use ap_shm_malloc (or similiar)
instead of standard malloc. Then have free_blocks call ap_shm_free
when the pool is destroyed.
To gain the memory management benefits of pools, plus the ability to
control their placement.
>
> > A) In addition to the existing ap_create_pool(), add a companion
> > ap_set_pool_allocator( void *opaque_allocator_object,
> > void (*pool_malloc)(void *opaque, size_t size),
> > void (*pool_free)(void *opaque, void *ptr) );
> > Or combine with ap_create_pool() to create a new
> > ap_create_pool_with_allocator() type call. These 3 parameters would be
> > stored in ap_pool_t for later retrieval
> >
> > B) Update malloc_block() and other apr_pools.c routines to call the
> > pool_malloc()/pool_free() when they have been set for that pool.
> > Subpools would inherit these allocate/deallocate routines.
>
> We looked at re-vamping our memory allocation model at the beginning of
> 2.0, and we never really came to a good conclusion of how to do it
> right. If you want to submit I patch, I will answer any questions that I
> can to help.
Thanks. I don't have a patch ready yet, but wanted some feedback on
whether such a change would possibly be accepted back into the
codebase.
-david
>
> > Also I was curious:
> >
> > 1. Is ap_create_pool() located in lib/apr/misc/*/start.c in order to
> > be platform dependent? The code looks the nearly same between
> > platforms, with the beos version slightly different from the other
> > two, though it is not clear why.
>
> I think you need to update your code. Currently, the only start.c is in
> lib/apr/misc/unix. The beos and Windows versions were removed because
> they were just duplicate code.
>
> > 2. ap_shm_init() has a ap_pool_t parameter that is not used. As if
> > someone had thought of tying the shared mem to a pool (instead of the
> > other way around)?
>
> One overriding feature of APR is that all APR functions take a pool
> *. This is because it is impossible for the APR developers to be able to
> predict when a new platform will be added that will need to be able to
> allocate memory in one of the APR functions. This would be much more
> useful with a new memory management model, but it is useful now
> none-the-less.
>
> Ryan
>
> _______________________________________________________________________________
> Ryan Bloom rbb@apache.org
> 406 29th St.
> San Francisco, CA 94131
> -------------------------------------------------------------------------------
>
>
>
Re: ap_pool support for shared memory?
Posted by rb...@covalent.net.
On Mon, 26 Jun 2000, David Williams wrote:
> I may have a need to use shared memory, but would also like to use the
> pool routines found in apr_pools.c, since they simplify memory
> management. The current obstacle being that these routines only call
> malloc/free. Has any one else had a similiar need, or see reasons
> they could not be enhanced? I was thinking along the lines of
> something similiar to:
Take a look at the ap_shm_* functions. They are actually using their own
pool implementation. I think this will be enough for what you are looking
for.
> A) In addition to the existing ap_create_pool(), add a companion
> ap_set_pool_allocator( void *opaque_allocator_object,
> void (*pool_malloc)(void *opaque, size_t size),
> void (*pool_free)(void *opaque, void *ptr) );
> Or combine with ap_create_pool() to create a new
> ap_create_pool_with_allocator() type call. These 3 parameters would be
> stored in ap_pool_t for later retrieval
>
> B) Update malloc_block() and other apr_pools.c routines to call the
> pool_malloc()/pool_free() when they have been set for that pool.
> Subpools would inherit these allocate/deallocate routines.
We looked at re-vamping our memory allocation model at the beginning of
2.0, and we never really came to a good conclusion of how to do it
right. If you want to submit I patch, I will answer any questions that I
can to help.
> Also I was curious:
>
> 1. Is ap_create_pool() located in lib/apr/misc/*/start.c in order to
> be platform dependent? The code looks the nearly same between
> platforms, with the beos version slightly different from the other
> two, though it is not clear why.
I think you need to update your code. Currently, the only start.c is in
lib/apr/misc/unix. The beos and Windows versions were removed because
they were just duplicate code.
> 2. ap_shm_init() has a ap_pool_t parameter that is not used. As if
> someone had thought of tying the shared mem to a pool (instead of the
> other way around)?
One overriding feature of APR is that all APR functions take a pool
*. This is because it is impossible for the APR developers to be able to
predict when a new platform will be added that will need to be able to
allocate memory in one of the APR functions. This would be much more
useful with a new memory management model, but it is useful now
none-the-less.
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
ap_pool support for shared memory?
Posted by David Williams <dw...@cisco.com>.
I may have a need to use shared memory, but would also like to use the
pool routines found in apr_pools.c, since they simplify memory
management. The current obstacle being that these routines only call
malloc/free. Has any one else had a similiar need, or see reasons
they could not be enhanced? I was thinking along the lines of
something similiar to:
A) In addition to the existing ap_create_pool(), add a companion
ap_set_pool_allocator( void *opaque_allocator_object,
void (*pool_malloc)(void *opaque, size_t size),
void (*pool_free)(void *opaque, void *ptr) );
Or combine with ap_create_pool() to create a new
ap_create_pool_with_allocator() type call. These 3 parameters would be
stored in ap_pool_t for later retrieval
B) Update malloc_block() and other apr_pools.c routines to call the
pool_malloc()/pool_free() when they have been set for that pool.
Subpools would inherit these allocate/deallocate routines.
Also I was curious:
1. Is ap_create_pool() located in lib/apr/misc/*/start.c in order to
be platform dependent? The code looks the nearly same between
platforms, with the beos version slightly different from the other
two, though it is not clear why.
2. ap_shm_init() has a ap_pool_t parameter that is not used. As if
someone had thought of tying the shared mem to a pool (instead of the
other way around)?
thanks,
-david
Re: hooks
Posted by rb...@covalent.net.
> What is the reason we have hooks (2.0) based on function name (ala
> ap_hook_fudge, ap_run_fudge) rather than some key (ie
> ap_add_callback(fudge_hook,...), ap_call_hook(fudge_hook))? I'm sure there
Because each hook has a different prototype and we need to be sure that
modules use the correct prototype. Having a single function that just
calls the right stuff under the covers means that we can't do real
argument checking at build time.
> is some good reason this has been done, but more than once I've needed to
> remind myself what the macro generated (ick) functions were going to look
> like, seems if we had a single function that dispatched these hooks it
> would be easier to maintain/lookup.
The macro generated functions are ugly, but IMHO they are better than use
having to duplicate code over and over again to do the same basic thing
with different parameters.
> It would also seem to be a good design to have _dynamic_ modules use
> runtime lookups of these hooks, rather than _static_ information. Doing
> this would remove some dependency of a dynamic module on the apache it's
> being used with.
I am very confused about what you are asking here, but I'll try to answer
anyway. :-)
The fact is that an Apache Module will ALWAYS be dependant on which Apache
it is running on. It is unrealistic to think that a module can figure out
where it should add a hook at run-time. Those kinds of decisions need to
be made by the programmer. For example, if I have a module that provides
the auth_checker_hook, how would you propose that module would determine
which hook to use?
> The DECLARE_HOOK stuff is good, however it doesn't lend itself well to
> having modules (protocol modules?) ap_run_*()'ing hooks (ie I must see an
> AP_DECLARE_HOOK for fudge, before I can actually bind to it). It also means
> I can't really "reuse" a hook if I come along with a module to replace the
> old functionality...
A protocol module should not be trying to ap_run_*() the hooks. Each
protocol module will need to implement it's own set of hooks. The
protocol module needs to know about the hooks it is allowing modules to
implement.
What do you mean you want to "reuse" a hook if you come along with a
module to replace the old functionality. I assume that since you are
working on the proxy, this is a proxy question. It would help a lot if
you could explain exactly what you want to do.
Are you saying you want to basically replace a hook that the server
implements with one the proxy implements? You can't do that.
Are you saying the proxy implements a hook that closely approximates what
the server is doing, and you want all of the modules that registered for
the server's hook to also register for yours? This is IMHO a bad thing to
do, but it is possible to do it. It requires a big hack, because it a bad
thing.
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------