You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@apr.apache.org by Joe Orton <jo...@redhat.com> on 2009/03/24 15:39:22 UTC
Modular APR
On the topic of how to split up APR into multiple libraries, I had a
look through the current directories, and a first cut at how I'd propose
to split the code up would be:
(directory -> library-name [dependencies])
buckets -> libapr-buckets
crypto -> libapr-crypto
dbd -> libapr-db [libapr-buckets]
dbm -> libapr-db
ldap -> libapr-ldap (or, it disappears)
memcache -> libapr-memcache [libapr-buckets]
random -> libapr-crypto
xlate -> libapr-xlate
xml -> libapr-xml
util-misc/... ???
and everything else into the "core" libapr. The util-misc directory is
a bit random, I'm not sure about what to do with that code.
Regards, Joe
Re: Modular APR
Posted by Mladen Turk <mt...@apache.org>.
Ruediger Pluem wrote:
> On 24.03.2009 17:41, Mladen Turk wrote:
>> Joe Orton wrote:
>>> On the topic of how to split up APR into multiple libraries, I had a
>>> look through the current directories, and a first cut at how I'd
>>> propose to split the code up would be:
>>>
>>> (directory -> library-name [dependencies])
>>>
>>> buckets -> libapr-buckets
>>> crypto -> libapr-crypto
>>> dbd -> libapr-db [libapr-buckets]
>>> dbm -> libapr-db
>>> ldap -> libapr-ldap (or, it disappears)
>>> memcache -> libapr-memcache [libapr-buckets]
>>> random -> libapr-crypto
>>> xlate -> libapr-xlate
>>> xml -> libapr-xml
>>> util-misc/... ???
>>>
>> What's the technical reason for that?
>> With the merging of apr and apr-util I was hoping
>> we would have to care only on a single library
>> and now there will be dozen of them. Scary.
>
> The most important thing is that we move out those parts which introduce
> dependencies on 3rd party libraries (well this requires a definition of 3rd
> party libraries, but this is another story :-)).
> So e.g. buckets might stay in the core library whereas dbd stuff at least the
> drivers move out into separate so's like today.
> Other examples might be xml or memcache.
>
That's understood, and that's fine for any driver code
that depends on 3rd party lib.
Things like dbm should have libapr-dbm-berkeleldb
but not libapr-dbm. This should stay within the
core together with internal sdbm implementation.
Regards
--
^(TM)
Re: Modular APR
Posted by Ruediger Pluem <rp...@apache.org>.
On 24.03.2009 17:41, Mladen Turk wrote:
> Joe Orton wrote:
>> On the topic of how to split up APR into multiple libraries, I had a
>> look through the current directories, and a first cut at how I'd
>> propose to split the code up would be:
>>
>> (directory -> library-name [dependencies])
>>
>> buckets -> libapr-buckets
>> crypto -> libapr-crypto
>> dbd -> libapr-db [libapr-buckets]
>> dbm -> libapr-db
>> ldap -> libapr-ldap (or, it disappears)
>> memcache -> libapr-memcache [libapr-buckets]
>> random -> libapr-crypto
>> xlate -> libapr-xlate
>> xml -> libapr-xml
>> util-misc/... ???
>>
>
> What's the technical reason for that?
> With the merging of apr and apr-util I was hoping
> we would have to care only on a single library
> and now there will be dozen of them. Scary.
The most important thing is that we move out those parts which introduce
dependencies on 3rd party libraries (well this requires a definition of 3rd
party libraries, but this is another story :-)).
So e.g. buckets might stay in the core library whereas dbd stuff at least the
drivers move out into separate so's like today.
Other examples might be xml or memcache.
Regards
Rüdiger
Re: Modular APR
Posted by Branko Čibej <br...@xbc.nu>.
Joe Orton wrote:
> On Tue, Mar 24, 2009 at 05:43:52PM +0100, William Rowe wrote:
>
>> Mladen Turk wrote:
>>
>>> Joe Orton wrote:
>>>
>>>> On the topic of how to split up APR into multiple libraries, I had a
>>>> look through the current directories, and a first cut at how I'd
>>>> propose to split the code up would be:
>>>>
> ...
>
>>> What's the technical reason for that?
>>> With the merging of apr and apr-util I was hoping
>>> we would have to care only on a single library
>>> and now there will be dozen of them. Scary.
>>>
>> Yes - I think this is a little overboard, but must be balanced
>> by the size of the code.
>>
>
> Right - but the primary motivation is as Ruediger says, to isolate
> dependencies, not to split by size.
>
> w.r.t. size, I did this for the apr-trunk, which I think is a reasonable
> measure of code size across directories - for information...
>
> $ for f in *; do \
> find $f -name \*.o | xargs ar cru ${f}.a && strip -g ${f}.a; \
> done
> $ ls -1sk *.a | sort -n | tail -20
> 16 uri.a
> 20 hooks.a
> 28 xml.a
> 40 memory.a
> 44 random.a
> 44 tables.a
> 48 ldap.a
> 56 crypto.a
> 56 memcache.a
> 56 misc.a
> 56 poll.a
> 60 threadproc.a
> 64 dbd.a
> 68 locks.a
> 72 dbm.a
> 76 strings.a
> 80 network_io.a
> 104 buckets.a
> 104 util-misc.a
> 120 file_io.a
>
So ... are you proposing we modularize APR to a zillion libs so that one
can chose bits and pieces for, e.g., and embedded app? I really don't
think that'll work out that well.
I don't mind if we build these separate (static) libs as an intermediate
step -- that would enable some really picky user to take just the bits
she needs -- as long the real build result is one libapr.(a|so) plus a
bunch of DSOs with third-party dependencies and/or variant implementations.
-- Brane
Re: Modular APR
Posted by Joe Orton <jo...@redhat.com>.
On Tue, Mar 24, 2009 at 05:43:52PM +0100, William Rowe wrote:
> Mladen Turk wrote:
>> Joe Orton wrote:
>>> On the topic of how to split up APR into multiple libraries, I had a
>>> look through the current directories, and a first cut at how I'd
>>> propose to split the code up would be:
...
>> What's the technical reason for that?
>> With the merging of apr and apr-util I was hoping
>> we would have to care only on a single library
>> and now there will be dozen of them. Scary.
>
> Yes - I think this is a little overboard, but must be balanced
> by the size of the code.
Right - but the primary motivation is as Ruediger says, to isolate
dependencies, not to split by size.
w.r.t. size, I did this for the apr-trunk, which I think is a reasonable
measure of code size across directories - for information...
$ for f in *; do \
find $f -name \*.o | xargs ar cru ${f}.a && strip -g ${f}.a; \
done
$ ls -1sk *.a | sort -n | tail -20
16 uri.a
20 hooks.a
28 xml.a
40 memory.a
44 random.a
44 tables.a
48 ldap.a
56 crypto.a
56 memcache.a
56 misc.a
56 poll.a
60 threadproc.a
64 dbd.a
68 locks.a
72 dbm.a
76 strings.a
80 network_io.a
104 buckets.a
104 util-misc.a
120 file_io.a
Re: Modular APR
Posted by "William A. Rowe, Jr." <wr...@rowe-clan.net>.
Mladen Turk wrote:
> Joe Orton wrote:
>> On the topic of how to split up APR into multiple libraries, I had a
>> look through the current directories, and a first cut at how I'd
>> propose to split the code up would be:
>>
>> (directory -> library-name [dependencies])
>>
>> buckets -> libapr-buckets
>> crypto -> libapr-crypto
>> dbd -> libapr-db [libapr-buckets]
>> dbm -> libapr-db
>> ldap -> libapr-ldap (or, it disappears)
>> memcache -> libapr-memcache [libapr-buckets]
>> random -> libapr-crypto
>> xlate -> libapr-xlate
>> xml -> libapr-xml
>> util-misc/... ???
>>
>
> What's the technical reason for that?
> With the merging of apr and apr-util I was hoping
> we would have to care only on a single library
> and now there will be dozen of them. Scary.
Yes - I think this is a little overboard, but must be balanced
by the size of the code.
If we are talking about ~16kb of .text page, then really this is
insignificant. No OS has <64kb granularity, and soon most will
be much higher.
You can't measure LoC, you have to look at the address map and
actual bytes consumed by the load-linker.
Bill
Re: Modular APR
Posted by Mladen Turk <mt...@apache.org>.
Joe Orton wrote:
> On the topic of how to split up APR into multiple libraries, I had a
> look through the current directories, and a first cut at how I'd propose
> to split the code up would be:
>
> (directory -> library-name [dependencies])
>
> buckets -> libapr-buckets
> crypto -> libapr-crypto
> dbd -> libapr-db [libapr-buckets]
> dbm -> libapr-db
> ldap -> libapr-ldap (or, it disappears)
> memcache -> libapr-memcache [libapr-buckets]
> random -> libapr-crypto
> xlate -> libapr-xlate
> xml -> libapr-xml
> util-misc/... ???
>
What's the technical reason for that?
With the merging of apr and apr-util I was hoping
we would have to care only on a single library
and now there will be dozen of them. Scary.
Regards
--
^(TM)
Re: Modular APR
Posted by Joe Orton <jo...@redhat.com>.
On Tue, Mar 24, 2009 at 04:58:02PM +0200, Graham Leggett wrote:
> If I am understanding you correctly, for a library like dbd that
> contains sub-libraries, you would have libapr-db, which in turn loads
> dynamic modules called (say) libapr-db-pgsql (etc)?
apr_dbd.c would be linked into a library called libapr-db. Any consumer
of the DBD API will be adjusted to link in this library as well as
libapr, e.g. by using something like "apu-2-config --modules=db".
If you build the DBD backends as DSOs, they are built as DSOs and are
loaded at runtime exactly as currently. If you link them in, they get
linked into libapr-db and otherwise everything works exactly as
currently.
Regards, Joe
Re: Modular APR
Posted by Graham Leggett <mi...@sharp.fm>.
Joe Orton wrote:
> On the topic of how to split up APR into multiple libraries, I had a
> look through the current directories, and a first cut at how I'd propose
> to split the code up would be:
>
> (directory -> library-name [dependencies])
>
> buckets -> libapr-buckets
> crypto -> libapr-crypto
> dbd -> libapr-db [libapr-buckets]
If I am understanding you correctly, for a library like dbd that
contains sub-libraries, you would have libapr-db, which in turn loads
dynamic modules called (say) libapr-db-pgsql (etc)?
If so, +1.
Regards,
Graham
--
Re: Modular APR
Posted by Graham Leggett <mi...@sharp.fm>.
Joe Orton wrote:
> On Wed, Mar 25, 2009 at 01:46:22PM +0100, Branko Čibej wrote:
>> IMHO having to link 55 libraries instead of one is a big pain.
>
> The difference should be passing a single extra argument to
> apr-N-config. Why such a big pain?
APR is growing in size (as it is expected to do), keeping APR split into
small manageable pieces that do one thing, and do them well, is a good
thing.
It is unlikely that an application will ever need to bind to every
single one of 55 libraries. Even httpd won't, though it's modules might
if you had to load every single one (which you don't want to do anyway).
Regards,
Graham
--
Re: Modular APR
Posted by Branko Čibej <br...@xbc.nu>.
Joe Orton wrote:
> On Wed, Mar 25, 2009 at 01:46:22PM +0100, Branko Čibej wrote:
>
>> IMHO having to link 55 libraries instead of one is a big pain.
>>
>
> The difference should be passing a single extra argument to
> apr-N-config. Why such a big pain?
>
On Windows?
-- Brane
Re: Modular APR
Posted by Joe Orton <jo...@redhat.com>.
On Wed, Mar 25, 2009 at 01:46:22PM +0100, Branko Čibej wrote:
> IMHO having to link 55 libraries instead of one is a big pain.
The difference should be passing a single extra argument to
apr-N-config. Why such a big pain?
> I prefer just putting internal wrappers for 3rd-party stuff into
> external modules that are loaded dynamically (or, optionally, linked
> into the base lib if the packager prefers that). Telling the linker to
> "-lapr" should be enough, regardless of the features you're using.
So there are two distinct cases here:
1) hard dependencies: use of the APR API implies a direct dependency
between the consumer of the API and a *single* underlying 3rd-party
library.
-> xml (libexpat), crypto (whichever crypto library), xlate (iconv)
in all these cases, I want any consumer of the API to have a dependency
on the underlying library wihch is tracked via SONAMEs. None of these
APIs can isolate the user of the code from the library -- if you want to
use apr_xml.h, you need expat. If you want to use xlate, you need an
iconv implementation. It is desirable and correct that such deps are
expressed by library linkage and resultant SONAMEs/DT_NEEDED, so that:
a) the runtime linker checks they can be satisfied at app startup, and
b) your package management system of choice automatically translates
SONAME/DT_NEEDED into package dependencies, and can ensure everything
"just works"
2) soft dependencies: use of the particular APR API implies an indirect
dependency on one-of-many compatible implementations. It's fine for
this case to push the implementation out to a given DSO.
Trying to use DSOs for case 1 "hard" dependencies:
a) adds bloat and runtime overhead by requiring a stub entry point in
libapr for *every single API function* which thunks out to a DSO
b) adds unnecessary complexity by having error paths ("the DSO didn't
load") where none should be needed, c.f. all the poor sods in bugzilla
who found the LDAP DSO code doesn't work for them
c) increases cost of maintenance because you have to maintain the thunks
as well as the real API implementation, and make sure they are in sync.
d) increases costs for downstream distributors who have to start
manually tracking interdependencies between consumers-of-APR and
particular dependencies, or, more likely, just punt and ensure that any
consumer of APR pulls in all APR's dependencies. Which means all these
costs have exactly zero benefit.
Regards, Joe
Re: Modular APR
Posted by Branko Čibej <br...@xbc.nu>.
Joe Orton wrote:
> On the topic of how to split up APR into multiple libraries, I had a
> look through the current directories, and a first cut at how I'd propose
> to split the code up would be:
>
> (directory -> library-name [dependencies])
>
> buckets -> libapr-buckets
> crypto -> libapr-crypto
> dbd -> libapr-db [libapr-buckets]
> dbm -> libapr-db
> ldap -> libapr-ldap (or, it disappears)
> memcache -> libapr-memcache [libapr-buckets]
> random -> libapr-crypto
> xlate -> libapr-xlate
> xml -> libapr-xml
> util-misc/... ???
>
> and everything else into the "core" libapr. The util-misc directory is
> a bit random, I'm not sure about what to do with that code.
>
IMHO having to link 55 libraries instead of one is a big pain. I prefer
just putting internal wrappers for 3rd-party stuff into external modules
that are loaded dynamically (or, optionally, linked into the base lib if
the packager prefers that). Telling the linker to "-lapr" should be
enough, regardless of the features you're using.
-- Brane