You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@apr.apache.org by "William A. Rowe, Jr." <wr...@rowe-clan.net> on 2008/05/16 00:17:00 UTC
Disconnecting -lldap from utilizing apr-util
http://people.apache.org/~wrowe/ldap/apr-util-1.x-ldap.patch
is the very first draft of a proposal which begins to break apart the
libldap (& liblber) bindings from the core libaprutil-1 and consuming
applications.
You can see, for reference, an earlier patch in that tree that broke ldap
from httpd 2.0. Remember than in libaprutil-0, there was no code that
actually consumed the ldap libs in apr-util; this was all the problem of
the user who linked to apr-util. So the earlier set of patches allowed
httpd to bind -lldap into mod_ldap and mod_auth_ldap on an explicit basis.
http://people.apache.org/~wrowe/ldap/httpd-2.x-ldap.patch illustrates how
this translates for such a patched apr-util-1.
What is interesting is that a patched apr-1.3.x remains binary compatible.
Simply ./configuring --enable-dbd-dso (really should be --enable-dsos) in
combination with --with-ldap will toggle the behavior.
What breaks in terms of ***build*** behavior is that it becomes the user's
responsibility to add the result of `apu-1-config --ldap-libs` to the app's
ld bindings. Otherwise, the non-ldap-consuming app won't have a direct
binding to libldap. Which IMHO is how it always should be.
The question is - is this a permitted change in 1.3.0? I believe it is, we
should not have been in the business of providing libraries in the manner
that happened for ldap. The other accessors, expat, iconv, etc don't carry
this sort of communicative property for our util users. In short, it was
a failed contract that should be harmonized without disrupting our users
too terribly. The --ldap-libs query lets them resume.
But this is not far enough, and only phase 1 of a patch. The complete
patch should abstract the rest of the ldap provider such that they do not
need to bind to -lldap in the first place. If this is not the goal, I'd
argue that this ldap binding really wasn't appropriate in the first place.
[Of course, that's an argument for 2.0 sans ldap and not a solution for 1.3
next release ;-] But I didn't want to hold up discussing a patch just to
add stubs to apr_ldap_err2string etc etc.
I also didn't hold up to add the parsing of LD_LIBRARY_PATH etc to find the
libraries (ldap, and dbd as well). That's something to discuss as well, and
again not a reason to hold up this patch.
Folks had hoped that httpd-2.2.9 would be compatible with either 1.2 or 1.3
apr flavors. We can get closer by improving the config.m4 patches, e.g.
MOD_AUTHNZ_LDAP_LDADD="`$apu_config --ldap-libs`"
becomes
MOD_AUTHNZ_LDAP_LDADD="`$apu_config --ldap-libs`" \
|| MOD_AUTHNZ_LDAP_LDADD=""
meaning that when --ldap-libs is unrecognized, we surrender to the complete
libs exposed by apr-util-1.2.x. But this would only be a provisional change
for the lifespan of httpd-2.2.x until it absolutely requires 1.3.0.
This does create a hardship for ldap users of apr-util _ldap API, but with
consideration given to the many, many applications of apr-util which don't
utilize the _ldap API, this change seems very appropriate to me. The note
about --ldap-libs can be prominently listed as the first entry in CHANGES.
Comments, feedback or questions?
Bill
Re: Disconnecting -lldap from utilizing apr-util
Posted by "William A. Rowe, Jr." <wr...@rowe-clan.net>.
William A. Rowe, Jr. wrote:
> http://people.apache.org/~wrowe/ldap/apr-util-1.x-ldap.patch
>
> http://people.apache.org/~wrowe/ldap/httpd-2.x-ldap.patch illustrates how
> this translates for such a patched apr-util-1.
As an illustration, here are the results from this change on three example
binaries, htpasswd (built --enable-static-support), httpd and mod_ldap.so
[wrowe@s170 apache2x]$ ldd bin/htpasswd
linux-gate.so.1 => (0x00110000)
libm.so.6 => /lib/libm.so.6 (0x00dc6000)
libuuid.so.1 => /lib/libuuid.so.1 (0x0062d000)
librt.so.1 => /lib/librt.so.1 (0x001bf000)
libcrypt.so.1 => /lib/libcrypt.so.1 (0x04dd8000)
libpthread.so.0 => /lib/libpthread.so.0 (0x00111000)
libdl.so.2 => /lib/libdl.so.2 (0x00df1000)
libc.so.6 => /lib/libc.so.6 (0x00c6b000)
/lib/ld-linux.so.2 (0x001a0000)
[wrowe@s170 apache2x]$ ldd bin/httpd
linux-gate.so.1 => (0x00110000)
libm.so.6 => /lib/libm.so.6 (0x00dc6000)
libaprutil-1.so.0 => /usr/local/apr1.x/lib/libaprutil-1.so.0
(0x00111000)
libexpat.so.1 => /lib/libexpat.so.1 (0x0016e000)
libapr-1.so.0 => /usr/local/apr1.x/lib/libapr-1.so.0 (0x0012d000)
libuuid.so.1 => /lib/libuuid.so.1 (0x0062d000)
librt.so.1 => /lib/librt.so.1 (0x001bf000)
libcrypt.so.1 => /lib/libcrypt.so.1 (0x04dd8000)
libpthread.so.0 => /lib/libpthread.so.0 (0x00152000)
libdl.so.2 => /lib/libdl.so.2 (0x00df1000)
libc.so.6 => /lib/libc.so.6 (0x00c6b000)
/lib/ld-linux.so.2 (0x001a0000)
[wrowe@s170 apache2x]$ ldd modules/mod_ldap.so
linux-gate.so.1 => (0x00110000)
libldap-2.3.so.0 => /usr/lib/libldap-2.3.so.0 (0x0011e000)
liblber-2.3.so.0 => /usr/lib/liblber-2.3.so.0 (0x00158000)
libpthread.so.0 => /lib/libpthread.so.0 (0x00166000)
libc.so.6 => /lib/libc.so.6 (0x001bd000)
libresolv.so.2 => /lib/libresolv.so.2 (0x0017f000)
libsasl2.so.2 => /usr/lib/libsasl2.so.2 (0x00316000)
libssl.so.6 => /lib/libssl.so.6 (0x0032f000)
libcrypto.so.6 => /lib/libcrypto.so.6 (0x00374000)
/lib/ld-linux.so.2 (0x001a0000)
libdl.so.2 => /lib/libdl.so.2 (0x00193000)
libcrypt.so.1 => /lib/libcrypt.so.1 (0x004a7000)
libgssapi_krb5.so.2 => /usr/lib/libgssapi_krb5.so.2 (0x004d9000)
libkrb5.so.3 => /usr/lib/libkrb5.so.3 (0x00507000)
libcom_err.so.2 => /lib/libcom_err.so.2 (0x00198000)
libk5crypto.so.3 => /usr/lib/libk5crypto.so.3 (0x0059a000)
libz.so.1 => /lib/libz.so.1 (0x005c0000)
libkrb5support.so.0 => /usr/lib/libkrb5support.so.0 (0x005d3000)
libkeyutils.so.1 => /lib/libkeyutils.so.1 (0x0019b000)
libselinux.so.1 => /lib/libselinux.so.1 (0x005dc000)
Re: Disconnected ldap, abstracted dbd.
Posted by "William A. Rowe, Jr." <wr...@rowe-clan.net>.
Brad Nicholes wrote:
> There seems to be a mismatch in how the #define APU_DSO_BUILD is being used. APU_DSO_BUILD has been defined as a boolean yet much of the code is simply checking for the definition rather than the boolean state. Therefore setting APU_DSO_BUILD to 0 doesn't actually turn off the code compilation paths. Shouldn't all of the statements of #ifdef (or #ifndef) APU_DSO_BUILD be changed to #if APU_DSO_BUILD?
Quite right, and a caught a few others. 659591 committed and backported
to 1.3.
Bill
Re: Disconnected ldap, abstracted dbd.
Posted by Brad Nicholes <BN...@novell.com>.
There seems to be a mismatch in how the #define APU_DSO_BUILD is being used. APU_DSO_BUILD has been defined as a boolean yet much of the code is simply checking for the definition rather than the boolean state. Therefore setting APU_DSO_BUILD to 0 doesn't actually turn off the code compilation paths. Shouldn't all of the statements of #ifdef (or #ifndef) APU_DSO_BUILD be changed to #if APU_DSO_BUILD?
Brad
>>> On 5/22/2008 at 4:52 PM, in message <48...@rowe-clan.net>, "William
A. Rowe, Jr." <wr...@rowe-clan.net> wrote:
> William A. Rowe, Jr. wrote:
>> Eric Covener wrote:
>>>
>>> +1 on wrapping it all in 2.0, but I don't think the versioning
>>> restrictions allow apr-util to stop linking against LDAP in 1.3 -- for
>>> example applications that load ldap symbols privately ("not at all"
>>> shouldn't be an issue because they can't do anything useful with the
>>> apr-util LDAP API in that case).
>>
>> Be aware that there is a one line change to most applications who want
>> apr-1.3.x to provide that binding for them (so they can use the symbols
>> privately). Simply, this would just no longer be the default. I think
>> that's a healthy thing since this affects < 10% of applications that
>> consume apr-util, and of the others, they would be better off linking
>> ldap only for specific components, e.g. mod_ldap/mod_authnz_ldap.
>
> http://people.apache.org/~wrowe/ldap/apr-util-1.x-dso.patch has been
> committed and is ready for review. If I hear no objection I'll roll
> on in the morning to 1.3.0 for apr and apr-util. There should be no
> API change going into applying this concept for dbm, iconv or xml
> so those are not showstoppers.
>
> The dbd and ldap are both abstracted to use the apu_dso_load. There is
> an apu_dso_init but it will be implicitly called for apr 1.x; in apr 2.x
> we can dictate that the users must explicitly invoke a general apr-util
> initializer. The loaded dso's lifetime is the process lifetime.
>
> Netware has a real problem, it's apr_pool_parent_get stops short of the
> root pool. Two consumers with different pools will not share the loaded
> resources and can cause the 2nd consumer to segv when the 1st consumer
> frees it's highest level pool. Bad mojo, nothing I can do about it since
> I have no clue why this exception was made on Netware.
>
> The changes permit us to run make test without installing the loadable
> dso dbd and ldap providers, the make file is hacked to include these in
> the shared lib path.
>
> Note that we keep two hashes, new is private dsos hash of modules by their
> module name. While dbd is refactored to use the apu_dso schema, it's still
> most efficient to continue to maintain a hash by the driver's shortname.
>
> Please, review. Thanks.
>
> Bill
Re: Disconnected ldap, abstracted dbd.
Posted by Bojan Smojver <bo...@rexursive.com>.
On Thu, 2008-05-22 at 22:06 -0500, William A. Rowe, Jr. wrote:
> Can you recheck from the branch? should have been fixed in r659343
Check. All is well :-)
--
Bojan
Re: Disconnected ldap, abstracted dbd.
Posted by "William A. Rowe, Jr." <wr...@rowe-clan.net>.
Bojan Smojver wrote:
>
> Gettin' these:
> --------------------------------------------
> misc/apu_dso.c: In function 'apu_dso_init':
> misc/apu_dso.c:84: warning: suggest parentheses around assignment used
> as truth value
> misc/apu_dso.c:84: warning: statement with no effect
> misc/apu_dso.c: In function 'apu_dso_load':
> misc/apu_dso.c:129: warning: implicit declaration of function
> 'apr_env_get'
> misc/apu_dso.c:130: warning: implicit declaration of function
> 'apr_filepath_list_split'
Can you recheck from the branch? should have been fixed in r659343
Re: Disconnected ldap, abstracted dbd.
Posted by Bojan Smojver <bo...@rexursive.com>.
On Thu, 2008-05-22 at 17:52 -0500, William A. Rowe, Jr. wrote:
> Please, review. Thanks.
Gettin' these:
--------------------------------------------
misc/apu_dso.c: In function 'apu_dso_init':
misc/apu_dso.c:84: warning: suggest parentheses around assignment used
as truth value
misc/apu_dso.c:84: warning: statement with no effect
misc/apu_dso.c: In function 'apu_dso_load':
misc/apu_dso.c:129: warning: implicit declaration of function
'apr_env_get'
misc/apu_dso.c:130: warning: implicit declaration of function
'apr_filepath_list_split'
--------------------------------------------
--
Bojan
Re: Disconnected ldap, abstracted dbd.
Posted by "William A. Rowe, Jr." <wr...@rowe-clan.net>.
William A. Rowe, Jr. wrote:
> Eric Covener wrote:
>>
>> +1 on wrapping it all in 2.0, but I don't think the versioning
>> restrictions allow apr-util to stop linking against LDAP in 1.3 -- for
>> example applications that load ldap symbols privately ("not at all"
>> shouldn't be an issue because they can't do anything useful with the
>> apr-util LDAP API in that case).
>
> Be aware that there is a one line change to most applications who want
> apr-1.3.x to provide that binding for them (so they can use the symbols
> privately). Simply, this would just no longer be the default. I think
> that's a healthy thing since this affects < 10% of applications that
> consume apr-util, and of the others, they would be better off linking
> ldap only for specific components, e.g. mod_ldap/mod_authnz_ldap.
http://people.apache.org/~wrowe/ldap/apr-util-1.x-dso.patch has been
committed and is ready for review. If I hear no objection I'll roll
on in the morning to 1.3.0 for apr and apr-util. There should be no
API change going into applying this concept for dbm, iconv or xml
so those are not showstoppers.
The dbd and ldap are both abstracted to use the apu_dso_load. There is
an apu_dso_init but it will be implicitly called for apr 1.x; in apr 2.x
we can dictate that the users must explicitly invoke a general apr-util
initializer. The loaded dso's lifetime is the process lifetime.
Netware has a real problem, it's apr_pool_parent_get stops short of the
root pool. Two consumers with different pools will not share the loaded
resources and can cause the 2nd consumer to segv when the 1st consumer
frees it's highest level pool. Bad mojo, nothing I can do about it since
I have no clue why this exception was made on Netware.
The changes permit us to run make test without installing the loadable
dso dbd and ldap providers, the make file is hacked to include these in
the shared lib path.
Note that we keep two hashes, new is private dsos hash of modules by their
module name. While dbd is refactored to use the apu_dso schema, it's still
most efficient to continue to maintain a hash by the driver's shortname.
Please, review. Thanks.
Bill
Re: Disconnecting -lldap from utilizing apr-util
Posted by "William A. Rowe, Jr." <wr...@rowe-clan.net>.
Eric Covener wrote:
>
> +1 on wrapping it all in 2.0, but I don't think the versioning
> restrictions allow apr-util to stop linking against LDAP in 1.3 -- for
> example applications that load ldap symbols privately ("not at all"
> shouldn't be an issue because they can't do anything useful with the
> apr-util LDAP API in that case).
Be aware that there is a one line change to most applications who want
apr-1.3.x to provide that binding for them (so they can use the symbols
privately). Simply, this would just no longer be the default. I think
that's a healthy thing since this affects < 10% of applications that
consume apr-util, and of the others, they would be better off linking
ldap only for specific components, e.g. mod_ldap/mod_authnz_ldap.
Bill
Re: Disconnecting -lldap from utilizing apr-util
Posted by Eric Covener <co...@gmail.com>.
On Thu, May 15, 2008 at 6:39 PM, Graham Leggett <mi...@sharp.fm> wrote:
> Wrapping the LDAP library however does make things cleaner, and does empower
> us to create standard behaviour should a future LDAP library come along and
> act funny. So if you want to wrap all the LDAP functions for v2.0, I think
> it is a good idea. It also makes LDAP work like DBD, and as I said above,
> consistent is good.
An added benefit of wrapping all the calls, and not requiring
applications to link with LDAP, is that we won't see applications blow
up due to the use of another LDAP library in /etc/nsswitch.conf (see
crashes in initgroups() and other bizarre behavior when two SDKS end
up loaded at once)
I'm actually using a kludge in my build process where I statically
link my SDK into mod_ldap.so and provide the bits needed by
mod_authnz_ldap and APR as optional functions. The choice of mod_ldap
instead of APR is of course to keep all the bloat out when mod_ldap
isn't being used, which is resolved by the bigger proposal.
+1 on wrapping it all in 2.0, but I don't think the versioning
restrictions allow apr-util to stop linking against LDAP in 1.3 -- for
example applications that load ldap symbols privately ("not at all"
shouldn't be an issue because they can't do anything useful with the
apr-util LDAP API in that case).
--
Eric Covener
covener@gmail.com
Re: Disconnecting -lldap from utilizing apr-util
Posted by Graham Leggett <mi...@sharp.fm>.
William A. Rowe, Jr. wrote:
> What is interesting is that a patched apr-1.3.x remains binary compatible.
> Simply ./configuring --enable-dbd-dso (really should be --enable-dsos) in
> combination with --with-ldap will toggle the behavior.
Am I correct in understanding that this brings the LDAP binding in line
with the exact same behaviour as the DBD bindings?
A definitely +1 to behaviour that is consistent between DBD and LDAP.
> But this is not far enough, and only phase 1 of a patch. The complete
> patch should abstract the rest of the ldap provider such that they do not
> need to bind to -lldap in the first place. If this is not the goal, I'd
> argue that this ldap binding really wasn't appropriate in the first place.
To answer the question as to why the LDAP bindings are as they are:
The LDAP C API is a standard. All the toolkits out there, have the exact
same bindings for all the functions in the standard. The problem was,
the standard completely omitted SSL/TLS.
Every toolkit out there invented their own weird way of configuring
SSL/TLS. Every toolkit out there has it's own weird library
requirements, different per platform. And by every toolkit I mean
OpenLDAP, Netware, Mozilla, Microsoft, Solaris, and some others whose
names I don't remember, on Windows, Solaris, Linux, etc.
The first consumer of the LDAP binaries, httpd, didn't want to care what
heinous list of hoops you had to jump through to make LDAP magically
become available on your chosen platform. This fell to the "P" in APR.
APR removes a *serious* headache from LDAP users.
One thing httpd *didn't* need was APR wrappers for every single LDAP
function in the LDAP C API standard, simply because it was a standard.
As a result, only the incompatible stuff is implemented in the apr
namespace. The whole LDAP library wasn't wrapped, simply because it
wasn't necessary at the time.
Wrapping the LDAP library however does make things cleaner, and does
empower us to create standard behaviour should a future LDAP library
come along and act funny. So if you want to wrap all the LDAP functions
for v2.0, I think it is a good idea. It also makes LDAP work like DBD,
and as I said above, consistent is good.
Regards,
Graham
--
Re: Disconnecting -lldap from utilizing apr-util
Posted by "William A. Rowe, Jr." <wr...@rowe-clan.net>.
Amazing how consistently someone's vetoes only arrive after proposals
have been offered for over a week and developers had already invested
the time to author the solution they originally proposed.
Simply amazing.
Bill
On May 15, 2008 William A. Rowe, Jr. wrote:
>
> What is interesting is that a patched apr-1.3.x remains binary compatible.
> Simply ./configuring --enable-dbd-dso (really should be --enable-dsos) in
> combination with --with-ldap will toggle the behavior.
>
> What breaks in terms of ***build*** behavior is that it becomes the user's
> responsibility to add the result of `apu-1-config --ldap-libs` to the app's
> ld bindings. Otherwise, the non-ldap-consuming app won't have a direct
> binding to libldap. Which IMHO is how it always should be.
>
> The question is - is this a permitted change in 1.3.0? I believe it is, we
> should not have been in the business of providing libraries in the manner
> that happened for ldap. The other accessors, expat, iconv, etc don't carry
> this sort of communicative property for our util users. In short, it was
> a failed contract that should be harmonized without disrupting our users
> too terribly. The --ldap-libs query lets them resume.
>
> But this is not far enough, and only phase 1 of a patch. The complete
> patch should abstract the rest of the ldap provider such that they do not
> need to bind to -lldap in the first place. If this is not the goal, I'd
> argue that this ldap binding really wasn't appropriate in the first place.
> [Of course, that's an argument for 2.0 sans ldap and not a solution for 1.3
> next release ;-] But I didn't want to hold up discussing a patch just to
> add stubs to apr_ldap_err2string etc etc.
>
> I also didn't hold up to add the parsing of LD_LIBRARY_PATH etc to find the
> libraries (ldap, and dbd as well). That's something to discuss as well,
> and
> again not a reason to hold up this patch.
>
> Folks had hoped that httpd-2.2.9 would be compatible with either 1.2 or 1.3
> apr flavors. We can get closer by improving the config.m4 patches, e.g.
>
> MOD_AUTHNZ_LDAP_LDADD="`$apu_config --ldap-libs`"
>
> becomes
>
> MOD_AUTHNZ_LDAP_LDADD="`$apu_config --ldap-libs`" \
> || MOD_AUTHNZ_LDAP_LDADD=""
>
> meaning that when --ldap-libs is unrecognized, we surrender to the complete
> libs exposed by apr-util-1.2.x. But this would only be a provisional
> change
> for the lifespan of httpd-2.2.x until it absolutely requires 1.3.0.
>
> This does create a hardship for ldap users of apr-util _ldap API, but with
> consideration given to the many, many applications of apr-util which don't
> utilize the _ldap API, this change seems very appropriate to me. The note
> about --ldap-libs can be prominently listed as the first entry in CHANGES.
>
> Comments, feedback or questions?
>
> Bill
Re: Disconnecting -lldap from utilizing apr-util
Posted by Joe Orton <jo...@redhat.com>.
On Fri, May 23, 2008 at 04:27:59PM +0200, Graham Leggett wrote:
> Joe Orton wrote:
>
>> As expected, this breaks source compatibility, as defined by the APR
>> versioning guidelines. The fact that you need to patch httpd to make
>> httpd continue to build seems like ample demonstration that source
>> compatibility is broken ;) So it's 2.x material, and because it's 2.x
>> material the DSO stuff is unnecessary, the LDAP code can simply be moved
>> into a separate library.
>
> What purpose would be served by moving the LDAP code into a separate
> library now that we have DSO support?
It's functionally equivalent and much simpler. (you don't need all the
thunks via the DSO, essentially you revert all of Bill's patch other
than some build system changes; and you don't need to add new thunks for
new function calls into LDAP code, etc)
joe
Re: Disconnecting -lldap from utilizing apr-util
Posted by Graham Leggett <mi...@sharp.fm>.
Joe Orton wrote:
> As expected, this breaks source compatibility, as defined by the APR
> versioning guidelines. The fact that you need to patch httpd to make
> httpd continue to build seems like ample demonstration that source
> compatibility is broken ;) So it's 2.x material, and because it's 2.x
> material the DSO stuff is unnecessary, the LDAP code can simply be moved
> into a separate library.
What purpose would be served by moving the LDAP code into a separate
library now that we have DSO support?
Regards,
Graham
--
Re: [Vote] Disconnecting -lldap from utilizing apr-util
Posted by "William A. Rowe, Jr." <wr...@rowe-clan.net>.
> removing -lldap -llber from ./configure --with-ldap builds into a seperate
> recoverable ./apu-1-config --ldap-libs flag;
>
> [ ] Breaks our versioning contract
> [X] Does not break our versioning contract
Re: [Vote] Disconnecting -lldap from utilizing apr-util
Posted by Graham Leggett <mi...@sharp.fm>.
William A. Rowe, Jr. wrote:
> removing -lldap -llber from ./configure --with-ldap builds into a seperate
> recoverable ./apu-1-config --ldap-libs flag;
>
> [ ] Breaks our versioning contract
> [X] Does not break our versioning contract
The acid test is whether an APR consuming application binary (eg httpd)
would break, if an apr-util binary (which was before compiled with LDAP
enabled) was replaced with a new apr-util binary (which was also
compiled with LDAP enabled).
I understand from wrowe's explanation that this will not break the APR
consuming application binary, therefore as I see it doesn't break our
versioning contract.
Regards,
Graham
--
[Vote] Disconnecting -lldap from utilizing apr-util
Posted by "William A. Rowe, Jr." <wr...@rowe-clan.net>.
Joe Orton wrote:
> On Thu, May 15, 2008 at 05:17:00PM -0500, William Rowe wrote:
>> http://people.apache.org/~wrowe/ldap/apr-util-1.x-ldap.patch
>
> As expected, this breaks source compatibility, as defined by the APR
> versioning guidelines. The fact that you need to patch httpd to make
> httpd continue to build seems like ample demonstration that source
> compatibility is broken ;) So it's 2.x material, and because it's 2.x
> material the DSO stuff is unnecessary, the LDAP code can simply be moved
> into a separate library.
APR makes no contract to export ldap_*, only apr_ldap_*. The rest are all
assumptions and happy circumstance. Reasonable people can disagree with
our assertions, so let's vote...
removing -lldap -llber from ./configure --with-ldap builds into a seperate
recoverable ./apu-1-config --ldap-libs flag;
[ ] Breaks our versioning contract
[ ] Does not break our versioning contract
Re: Disconnecting -lldap from utilizing apr-util
Posted by Ruediger Pluem <rp...@apache.org>.
On 05/23/2008 10:05 PM, William A. Rowe, Jr. wrote:
> Joe Orton wrote:
>> More review:
>>
>> 1) the DSO-loading stuff is not thread-safe (dsos global not
>> mutex-protected) yet apr_ldap_init would/could/should be; it's not
>> documented to explicitly *not* be thread-safe, anyway.
>
> The caller must obtain the mutex first, because the caller is responsible
> for initialization before releasing the mutex (it wouldn't do to acquire
> an uninitialized resource.) I'll document, as you suggest.
Don't we have an issue if apu_dso_init is called by two threads in parallel
when dsos is still NULL?
IMHO this can happen as load_ldap could be called by two threads in parallel.
BTW: I guess it would be more effective to do the for loop for getting the global
pool after checking if dsos != NULL.
Regards
Rüdiger
Re: Disconnecting -lldap from utilizing apr-util
Posted by "William A. Rowe, Jr." <wr...@rowe-clan.net>.
Joe Orton wrote:
>
> Ah right, you mean adding an API to expose that. Yeah, makes sense I
> suppose, but this is kind of piling on more and more layers of
> workarounds. An alternative maybe is to just create another unparented
> global pool here using the new 1.3 pool_core_create_ex API.
Trouble is, we *WANT* this pool cleanly torn down to invoke all of the
appropriate destructors upon apr_terminate. That requires the global pool.
It's not piling on, it's mainstreaming.
> More review:
>
> 1) the DSO-loading stuff is not thread-safe (dsos global not
> mutex-protected) yet apr_ldap_init would/could/should be; it's not
> documented to explicitly *not* be thread-safe, anyway.
The caller must obtain the mutex first, because the caller is responsible
for initialization before releasing the mutex (it wouldn't do to acquire
an uninitialized resource.) I'll document, as you suggest.
I thought about a mutex within, but that would have required some flexible
init callout and it seemed much more trouble than it's worth.
> 2) on the DSO-failed-to-load error paths it looks like *result_err
> should be set but won't be.
Will research, but if you have a patch that would save me some effort.
Bill
Re: Disconnecting -lldap from utilizing apr-util
Posted by Joe Orton <jo...@redhat.com>.
On Fri, May 23, 2008 at 11:50:35AM -0500, William Rowe wrote:
> Joe Orton wrote:
>>> Then do we all decide that there is a global pool and an API to obtain it?
>>> I know how to do this, but I'm just asking if that's how to go.
>>
>> I'm not sure what you're asking here. Starting in 1.3.x there can be
>> multiple global pools each under control of separate consumers.
>
> There's only one global pool in the context of apr_[app_]initialize.
> That global initialization is refcounted.
Ah right, you mean adding an API to expose that. Yeah, makes sense I
suppose, but this is kind of piling on more and more layers of
workarounds. An alternative maybe is to just create another unparented
global pool here using the new 1.3 pool_core_create_ex API.
More review:
1) the DSO-loading stuff is not thread-safe (dsos global not
mutex-protected) yet apr_ldap_init would/could/should be; it's not
documented to explicitly *not* be thread-safe, anyway.
2) on the DSO-failed-to-load error paths it looks like *result_err
should be set but won't be.
joe
Re: Disconnecting -lldap from utilizing apr-util
Posted by "William A. Rowe, Jr." <wr...@rowe-clan.net>.
Joe Orton wrote:
>> Then do we all decide that there is a global pool and an API to obtain it?
>> I know how to do this, but I'm just asking if that's how to go.
>
> I'm not sure what you're asking here. Starting in 1.3.x there can be
> multiple global pools each under control of separate consumers.
There's only one global pool in the context of apr_[app_]initialize.
That global initialization is refcounted.
Right?
Bill
Re: Disconnecting -lldap from utilizing apr-util
Posted by Joe Orton <jo...@redhat.com>.
On Fri, May 23, 2008 at 09:23:43AM -0500, William Rowe wrote:
> There *IS* a build system schema change. I don't see where we wrote out
> the versioning rules for building APR. But I beg to differ with you, this
> change does not violate versioning guidelines.
I guess it's reasonable to argue that this isn't a source compat break
because it's not the default build configuration; the config option used
to enable it needs to make that pretty clear in that case. It still
sounds like a bit of headache. Do you think it's really so important to
do get this changed now in 1.x?
>> - the pool lifetime stuff is probably dodgy because in 1.3.x you can now
>> create unparented pools, i.e., there's no longer any guarantee that
>> following pool->parent gets you to the APR global pool which lasts for the
>> lifetime is the process. (who knows what else that new misfeature may
>> break)
>
> Then do we all decide that there is a global pool and an API to obtain it?
> I know how to do this, but I'm just asking if that's how to go.
I'm not sure what you're asking here. Starting in 1.3.x there can be
multiple global pools each under control of separate consumers.
> The reason for this of course is that people do want to consume APR in their
> libraries, which means consumers X and Y might not even be an "application",
> and may not start up and shut down at the same time.
Right, exactly. And in 1.3.x X and Y might be passing their own
independent global pool to apr_ldap_init() or whatever - snafu ensues.
joe
Re: Disconnecting -lldap from utilizing apr-util
Posted by "William A. Rowe, Jr." <wr...@rowe-clan.net>.
Joe Orton wrote:
> On Thu, May 15, 2008 at 05:17:00PM -0500, William Rowe wrote:
>> http://people.apache.org/~wrowe/ldap/apr-util-1.x-ldap.patch
>
> As expected, this breaks source compatibility, as defined by the APR
> versioning guidelines. The fact that you need to patch httpd to make
> httpd continue to build seems like ample demonstration that source
> compatibility is broken ;) So it's 2.x material, and because it's 2.x
> material the DSO stuff is unnecessary, the LDAP code can simply be moved
> into a separate library.
You know, I don't buy it. Not the bit of splitting into multiple libs
for 2.x, that's a possibility but possibly unnecessary if the dso approach
serves us well. (In fact, this might get us to the point of merging all
-util into the base apr-2. Something to ponder.)
No - I'm not buying that we've broken a rule...
There is no assurance that you have an apr-util built --with-ldap period.
If you have one, the source code is compatible. APR is not the author of
ldap_* namespace. Anyone who detected (the same) ldap as apr-util is not
affected.
The source code (.c) compiles identically against version 1.3.0, there is
no source code API change.
There *IS* a build system schema change. I don't see where we wrote out
the versioning rules for building APR. But I beg to differ with you, this
change does not violate versioning guidelines.
> Otherwise:
>
> - surprisingly simple patch, kudos
Thanks. Simple in retrospect, PITA to author ;-)
> - the pool lifetime stuff is probably dodgy because in 1.3.x you can now
> create unparented pools, i.e., there's no longer any guarantee that
> following pool->parent gets you to the APR global pool which lasts for
> the lifetime is the process. (who knows what else that new misfeature
> may break)
Then do we all decide that there is a global pool and an API to obtain it?
I know how to do this, but I'm just asking if that's how to go.
The reason for this of course is that people do want to consume APR in their
libraries, which means consumers X and Y might not even be an "application",
and may not start up and shut down at the same time.
> - mingling the autofoo in with the DBD autofoo is obviously ugly ;)
You mean DSOfoo with DBDfoo I presume. Yes, patch inc.
Re: Disconnecting -lldap from utilizing apr-util
Posted by Joe Orton <jo...@redhat.com>.
On Thu, May 15, 2008 at 05:17:00PM -0500, William Rowe wrote:
> http://people.apache.org/~wrowe/ldap/apr-util-1.x-ldap.patch
As expected, this breaks source compatibility, as defined by the APR
versioning guidelines. The fact that you need to patch httpd to make
httpd continue to build seems like ample demonstration that source
compatibility is broken ;) So it's 2.x material, and because it's 2.x
material the DSO stuff is unnecessary, the LDAP code can simply be moved
into a separate library.
Otherwise:
- surprisingly simple patch, kudos
- the pool lifetime stuff is probably dodgy because in 1.3.x you can now
create unparented pools, i.e., there's no longer any guarantee that
following pool->parent gets you to the APR global pool which lasts for
the lifetime is the process. (who knows what else that new misfeature
may break)
- mingling the autofoo in with the DBD autofoo is obviously ugly ;)
joe