need a solution with APR DSO

[Guenter Knauf <fu...@apache.org>]

All,
I have the this issue:
if I define APU_DSO_BUILD=1 then that means that we're forced to build
every driver as DSO. On NetWare though we dont want to do so: we have
always LDAP available and installed - there's no NetWare installation
without LDAP, while MySQL and all other DBs are optional - therefore we
want to always build LDAP statically linked into APR, while we _must_
make the DBD drivers dynamic and optional since they depend on the
database client libs. So what we need is this:
- define APU_DSO_BUILD=1 in order to build the DBD drivers as DSO
- define APU_DSO_LDAP_BUILD=0 in order to build LDAP support static
looking into apr_ldap_init.c this seems not possible:
http://svn.apache.org/viewvc/apr/apr-util/branches/1.3.x/ldap/apr_ldap_init.c?revision=661732
there we find at line 23ff:
#if APU_DSO_BUILD
#define APU_DSO_LDAP_BUILD
#endif

so this means for the NetWare build: either have set APU_DSO_BUILD=0 in
order to build LDAP static and then no DBD driver at all, or have set
APU_DSO_BUILD=1 and be forced to build LDAP as DSO.
If we have already an own define APU_DSO_LDAP_BUILD - why cant we just
move its setting into apu_config.h and each platform can set it as it fits?

please comment!

thanks, Gün.

Re: need a solution with APR DSO

[Guenter Knauf <fu...@apache.org>]

Bill,
William A. Rowe, Jr. schrieb:
> see APU_DSO_LDAP_BUILD before building out new things?
I dont care much how we finally name it - functionality counts for me;
APU_DSO_LDAP_BUILD is an uncompleted attempt perhaps, but does not work
at all for two reasons:
- as I wrote already previously we have this in apr_ldap_init.c:
http://svn.apache.org/viewvc/apr/apr-util/branches/1.3.x/ldap/apr_ldap_init.c?revision=661732
#if APU_DSO_BUILD
#define APU_DSO_LDAP_BUILD
#endif
which renders APU_DSO_LDAP_BUILD totally useless since it directly
depends unconditionally on APU_DSO_BUILD

- apr_ldap_stub.c does depend on APU_DSO_BUILD instead of
APU_DSO_LDAP_BUILD:
http://svn.apache.org/viewvc/apr/apr-util/branches/1.3.x/ldap/apr_ldap_stub.c?revision=720559

Gün.

Re: need a solution with APR DSO

["William A. Rowe, Jr." <wr...@rowe-clan.net>]

Guenter Knauf wrote:
> Bill,
> William A. Rowe, Jr. schrieb:
>> Guenter, I really missed nothing.  My point was that it consumes
>> process resources to load an ldap support library, which are not
>> needed by all apr consuming applications (in fact, needed only
>> by a tiny minority of applications.)
> I cant resist to think that you somehow dislike LDAP at all ....
> also I disagree with your point of view with 'tiny minority' and '10%
> only'; in fact almost every server platform has an LDAP server running
> (including W2K3), and makes use of it as user database and for access
> control, thus I would more tend to believe that 50% of all APR
> installations making use of LDAP support - at least when httpd is the
> APR consumer; and if I build my own LDAP-aware application upon APR then
> 100% of my users will use LDAP ...
> finally please keep in mind that I dont want to change any defaults - I
> expect 100% same settings and functionality after we have changed what I
> propose; but I want to be able to control DSO/static with a simple edit
> of a define, and without need for patching code at various places.

I have some dozen command line tools (common unix commands) that work more
precisely than their gnuwin tools/unxutils equivilants.  Convince me that
straightforward commands such as which or rm need ldap, or even httpd's
rotatelogs, logresolve or htcacheclean, or Tomcat's tcNative connector.

Loading all libraries for every command executed would be something like
trying to use libtool on Mac OS/X 10.3 PPC, with it's horrid fork() support.
Dog slow.  And unfunny.

"Server Applications" - sure maybe 10% of users try to configure LDAP
at some point, based on user list traffic.  But get it out of your head
that the *only* application of apr[util] is writing server applications :)
We are a bit more utilitarian and useful than that.

Re: need a solution with APR DSO

[Guenter Knauf <fu...@apache.org>]

Bill,
William A. Rowe, Jr. schrieb:
> Guenter, I really missed nothing.  My point was that it consumes
> process resources to load an ldap support library, which are not
> needed by all apr consuming applications (in fact, needed only
> by a tiny minority of applications.)
I cant resist to think that you somehow dislike LDAP at all ....
also I disagree with your point of view with 'tiny minority' and '10%
only'; in fact almost every server platform has an LDAP server running
(including W2K3), and makes use of it as user database and for access
control, thus I would more tend to believe that 50% of all APR
installations making use of LDAP support - at least when httpd is the
APR consumer; and if I build my own LDAP-aware application upon APR then
100% of my users will use LDAP ...
finally please keep in mind that I dont want to change any defaults - I
expect 100% same settings and functionality after we have changed what I
propose; but I want to be able to control DSO/static with a simple edit
of a define, and without need for patching code at various places.

> Thanks for reminding us of the Novell SDK as well :)
you're welcome!

Guen.

Re: need a solution with APR DSO

["William A. Rowe, Jr." <wr...@rowe-clan.net>]

Guenter, I really missed nothing.  My point was that it consumes
process resources to load an ldap support library, which are not
needed by all apr consuming applications (in fact, needed only
by a tiny minority of applications.)  Thanks for reminding us of
the Novell SDK as well :)

> ...
> #if APU_DSO_BUILD
> APR_HAS_LDAP_DSO 0

see APU_DSO_LDAP_BUILD before building out new things?

Re: need a solution with APR DSO

[Guenter Knauf <fu...@apache.org>]

Hi Bill,
William A. Rowe, Jr. schrieb:
> -1... yes it might be present, but it is a waste of process resource
> to have a fixed binding for the 90% of apr apps that don't make ldap
> queries.  The idea is to pull in only the bindings used and save
> a bunch of effort in the run time linker resolution table.  It gets
> worse and worse as apr offers more lib support.  And this is nothing
> more than what 'just in time' run time linking does, anyways, only
> we rolled our own.  Do we need 15 additional libraries loaded for
> 'hello world'?
> 
> Aside from which, there are three clients that can reasonably be
> expected to be plugged in on win32; openldap, netscape or microsoft.
four: + Novell CLDAP - works great :)
> As this is further abstracted, it becomes easy to substitute to
> work around interop quirks with different servers (although never
> 'all at once', more of a 'pick one' solution).
> 
> However Netware wants to handle it is fine, but we certainly don't
> want Windows to regress here.
sorry, but you completely missed the subject in first place here -
before we talk about what should be default or what not we need to agree
about a proper way how we handle to build one 'driver' as DSO and link
another 'driver' statically; once this is done everyone can simply
decide with a define change; currently this is impossible, and is like
we both said: only all or nothing.
So I'd suggest that we decouple the dynamic 'driver' builds from
APU_DSO_BUILD, and add separate defines for it, like:
APR_HAS_LDAP 1
APR_HAS_MYSQL 1
APR_HAS_PGSQL 1
...
#if APU_DSO_BUILD
APR_HAS_LDAP_DSO 0
APR_HAS_MYSQL_DSO 1
APR_HAS_PGSQL_DSO 1
...
#endif

Gün.

Re: need a solution with APR DSO

[Bojan Smojver <bo...@rexursive.com>]

On Tue, 2009-08-11 at 01:22 +0200, Guenter Knauf wrote:
> I'm perfectly fine, and even agree with your both ideas and thoughts
> as long as you dont force every developer to adopt them! Why dont you
> want to allow developers to choose what's best for their
> application/platform? Is that too much freedom??

I was just recalling a previous conversation on the list. No forcing
coming from here.

-- 
Bojan

Re: need a solution with APR DSO

[Guenter Knauf <fu...@apache.org>]

Hi,
Bojan Smojver schrieb:
> On Mon, 2009-08-10 at 10:15 -0500, William A. Rowe, Jr. wrote:
>> The idea is to pull in only the bindings used
> 
> Yeah, I thought so too. I kinda remember a discussion along those lines
> on the list...
I'm perfectly fine, and even agree with your both ideas and thoughts as
long as you dont force every developer to adopt them! Why dont you want
to allow developers to choose what's best for their
application/platform? Is that too much freedom??
We'are writing here mail stories about changing / adding few defines
with zero code changes - cant we just agree that it might be useful to
enable / disable DSO build per driver, and not global for all per
APU_DSO_BUILD? What's wrong with my suggestion? Where does it hurt any
other ones? I thought that we just agree about namings of the additional
defines, and then go ahead to change, and really didnt expect that
someone would object to it.

And here again my reasons but more detailed:
On NetWare we have 95% of all httpd/APR users using LDAP(S) - that might
be different on other platforms; but I know this for sure since I was
active as Novell developer sysop for 7+ years, and therefore have a
close view of it. With httpd 2.0.x its 100% of all installations since
httpd 2.0.x is the shipping administration server which is secured
through eDirectory via LDAP(S). In addition Novell provides iPrint,
iFolder, NetStorage httpd modules which all make use of eDirectory via
LDAP. We have a mod_edir which authenticates and fetches home directory
attributes via LDAP. I think these are enough good reasons why we want
to allways have LDAP statically linked.
Another reason which makes my suggestion valid is that we switched in
the middle of lifetime of httpd 2.2.x from APR 1.2.x to 1.3.x - if I
would now build LDAP as DSO this would require additional configuration
to load it, and thus break existing configs - and currently I see
nothing even mentioned in httpd mod_ldap or mod_authnz_ldap about how to
load the ldap 'driver' in case its build as DSO ....

that said, I wish we could just agree that decoupling driver DSO builds
from APU_DSO_BUILD setting is a valid feature request, and wish I could
get some some help with implementing, specially with the configure stuff
for apu_config.h so that I dont break things.

thanks, Gün.

Re: need a solution with APR DSO

[Bojan Smojver <bo...@rexursive.com>]

On Mon, 2009-08-10 at 10:15 -0500, William A. Rowe, Jr. wrote:
> The idea is to pull in only the bindings used

Yeah, I thought so too. I kinda remember a discussion along those lines
on the list...

-- 
Bojan

Re: need a solution with APR DSO

[Guenter Knauf <fu...@apache.org>]

William A. Rowe, Jr. schrieb:
> Guenter Knauf wrote:
>> Bill,
>> William A. Rowe, Jr. schrieb:
> Unix ./configure users have no trouble toggling features.  Win32 and Netware
> currently don't have a command line configuration tool.
maybe they can toggle APU_DSO_BUILD via configure which again results in
either all or none as DSO.

> I have no issue with adding flexibility.  But the current API won't really
> support having some loadable, some fixed dbd or dbm providers (well, except
> the special-case SDBM).  So I wouldn't want those to be sliced-and-diced
> into individual MSQL_USES_DSO - rather DBD_USES_DSO or something.
ok, fine with me. So unless you object further I would like to start with:
1) use APU_DSO_LDAP_BUILD for the whole ldap stuff instead of
APU_DSO_BUILD, and move APU_DSO_LDAP_BUILD define into apu_config.h
2) add APU_DSO_DBD_BUILD to control the DBD drivers


this would give me the flexibility I need, and I could start working on
it to see how that goes ...

Gün.

Re: need a solution with APR DSO

["William A. Rowe, Jr." <wr...@rowe-clan.net>]

Guenter Knauf wrote:
> Bill,
> William A. Rowe, Jr. schrieb:
>> Right, and there is a solution for that (although the win32 build system
>> isn't rich enough right now to offer a 'toggle' of this.)
> wrong - there's no solution, and no platform can just 'toggle' - see my
> other post.

Unix ./configure users have no trouble toggling features.  Win32 and Netware
currently don't have a command line configuration tool.

That was my entire point.

I have no issue with adding flexibility.  But the current API won't really
support having some loadable, some fixed dbd or dbm providers (well, except
the special-case SDBM).  So I wouldn't want those to be sliced-and-diced
into individual MSQL_USES_DSO - rather DBD_USES_DSO or something.

Re: need a solution with APR DSO

[Guenter Knauf <fu...@apache.org>]

Bill,
William A. Rowe, Jr. schrieb:
> Right, and there is a solution for that (although the win32 build system
> isn't rich enough right now to offer a 'toggle' of this.)
wrong - there's no solution, and no platform can just 'toggle' - see my
other post.

Gün.

Re: need a solution with APR DSO

["William A. Rowe, Jr." <wr...@rowe-clan.net>]

Mladen Turk wrote:
> On 10/08/09 17:15, William A. Rowe, Jr. wrote:
>> Mladen Turk wrote:
>>>
>>> IMHO every module should have an configure option to either
>>> build statically or as DSO, not as right now (all or none)
>>
>> -1... yes it might be present, but it is a waste of process resource
>> to have a fixed binding for the 90% of apr apps that don't make ldap
>> queries.  The idea is to pull in only the bindings used and save
>> a bunch of effort in the run time linker resolution table.  It gets
>> worse and worse as apr offers more lib support.  And this is nothing
>> more than what 'just in time' run time linking does, anyways, only
>> we rolled our own.  Do we need 15 additional libraries loaded for
>> 'hello world'?
>>
> 
> I simply hate when people start throwing -1's all over the places.
> This is what? Your 1000th -1?.
> 
> I didn't said it should be default linked in, but it should have
> an option to decide at compile time weather to build it statically
> or as a module. OTOH if application needs ldap then its a waste of
> time and resource to call the LoadLibrary (or do that 15 times for each
> additional module)

Right, and there is a solution for that (although the win32 build system
isn't rich enough right now to offer a 'toggle' of this.)

Re: need a solution with APR DSO

[Mladen Turk <mt...@apache.org>]

On 10/08/09 17:15, William A. Rowe, Jr. wrote:
> Mladen Turk wrote:
>>
>> IMHO every module should have an configure option to either
>> build statically or as DSO, not as right now (all or none)
>
> -1... yes it might be present, but it is a waste of process resource
> to have a fixed binding for the 90% of apr apps that don't make ldap
> queries.  The idea is to pull in only the bindings used and save
> a bunch of effort in the run time linker resolution table.  It gets
> worse and worse as apr offers more lib support.  And this is nothing
> more than what 'just in time' run time linking does, anyways, only
> we rolled our own.  Do we need 15 additional libraries loaded for
> 'hello world'?
>

I simply hate when people start throwing -1's all over the places.
This is what? Your 1000th -1?.

I didn't said it should be default linked in, but it should have
an option to decide at compile time weather to build it statically
or as a module. OTOH if application needs ldap then its a waste of
time and resource to call the LoadLibrary (or do that 15 times for each
additional module)


Regards
-- 
^TM

Re: need a solution with APR DSO

["William A. Rowe, Jr." <wr...@rowe-clan.net>]

Mladen Turk wrote:
> On 09/08/09 03:56, Guenter Knauf wrote:
>>
>> so this means for the NetWare build: either have set APU_DSO_BUILD=0 in
>> order to build LDAP static and then no DBD driver at all, or have set
>> APU_DSO_BUILD=1 and be forced to build LDAP as DSO.
>> If we have already an own define APU_DSO_LDAP_BUILD - why cant we just
>> move its setting into apu_config.h and each platform can set it as it
>> fits?
>>
>> please comment!
>>
> 
> Same situation on Windows where there is always winldap present.
> See no reason why would this need to be compiled as DSO.
> 
> IMHO every module should have an configure option to either
> build statically or as DSO, not as right now (all or none)

-1... yes it might be present, but it is a waste of process resource
to have a fixed binding for the 90% of apr apps that don't make ldap
queries.  The idea is to pull in only the bindings used and save
a bunch of effort in the run time linker resolution table.  It gets
worse and worse as apr offers more lib support.  And this is nothing
more than what 'just in time' run time linking does, anyways, only
we rolled our own.  Do we need 15 additional libraries loaded for
'hello world'?

Aside from which, there are three clients that can reasonably be
expected to be plugged in on win32; openldap, netscape or microsoft.
As this is further abstracted, it becomes easy to substitute to
work around interop quirks with different servers (although never
'all at once', more of a 'pick one' solution).

However Netware wants to handle it is fine, but we certainly don't
want Windows to regress here.

Re: need a solution with APR DSO

[Mladen Turk <mt...@apache.org>]

On 09/08/09 03:56, Guenter Knauf wrote:
>
> so this means for the NetWare build: either have set APU_DSO_BUILD=0 in
> order to build LDAP static and then no DBD driver at all, or have set
> APU_DSO_BUILD=1 and be forced to build LDAP as DSO.
> If we have already an own define APU_DSO_LDAP_BUILD - why cant we just
> move its setting into apu_config.h and each platform can set it as it fits?
>
> please comment!
>

Same situation on Windows where there is always winldap present.
See no reason why would this need to be compiled as DSO.

IMHO every module should have an configure option to either
build statically or as DSO, not as right now (all or none)


Regards
-- 
^TM