You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by rb...@covalent.net on 2000/06/05 19:45:31 UTC
APR builds concept broken horribly
Okay, way back when, when APR was first written, it was possible to build
individual sections of APR independantly on Unix. This provided us with
small compact libraries that could be included or not based on the
program. This feature was removed a long time ago, and it needs to be
added back.
The problem was highlighted eariler today with the whold dso issue. If
htpasswd is an APR'ized program, and APR can only be built as one huge
library, then htpasswd all of a sudden needs DSO support. That is
ludicrous. The other option is to comfigure and compile APR multiple
times in one Apache build. Once for Apache itself and once for the
support programs.
If APR could be built as small libraries again, then htpasswd could just
link with the libs/file_io stuff and ignore the rest, which makes more
sense IMHO.
Thoughts?
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
Re: APR builds concept broken horribly
Posted by Brian Havard <br...@kheldar.apana.org.au>.
On Mon, 5 Jun 2000 10:45:31 -0700 (PDT), rbb@covalent.net wrote:
>Okay, way back when, when APR was first written, it was possible to build
>individual sections of APR independantly on Unix. This provided us with
>small compact libraries that could be included or not based on the
>program. This feature was removed a long time ago, and it needs to be
>added back.
>
>The problem was highlighted eariler today with the whold dso issue. If
>htpasswd is an APR'ized program, and APR can only be built as one huge
>library, then htpasswd all of a sudden needs DSO support. That is
>ludicrous. The other option is to comfigure and compile APR multiple
>times in one Apache build. Once for Apache itself and once for the
>support programs.
>
>If APR could be built as small libraries again, then htpasswd could just
>link with the libs/file_io stuff and ignore the rest, which makes more
>sense IMHO.
>
>Thoughts?
I think Greg pretty much covers what I think about this. If you link the
app against libapr.a and the app doesn't use ap_dso_* then dso.o won't get
linked so there'll be no need for -ldl.
If using libapr.so it should already contain everything it needs and any
half decent OS demand loads code pages so again, only the code that's
actually needed gets loaded.
Why break it up when the linker or OS can already pick out just the pieces
needed?
--
______________________________________________________________________________
| Brian Havard | "He is not the messiah! |
| brianh@kheldar.apana.org.au | He's a very naughty boy!" - Life of Brian |
------------------------------------------------------------------------------
Re: APR builds concept broken horribly
Posted by Sascha Schumann <sa...@schumann.cx>.
On Mon, 5 Jun 2000, Rodent of Unusual Size wrote:
> Sascha Schumann wrote:
> >
> > If .a refers to libapr.a which contains the DSO-related
> > functions, then the link editor will probably complain about
> > undefined references, unless you link against libdl.
>
> Why should it, if the library is only named in a conditional
> satisfy mode (i.e., "-lapr" rather than "libapr.a")?
Forget the comment, my brain assumed that one library = one
object file. In that case, the link editor will of course
complain, if you use symbols from that object file and not
all references are satisfied (no matter how you named the
library).
- Sascha
Re: APR builds concept broken horribly
Posted by Rodent of Unusual Size <Ke...@Golux.Com>.
Sascha Schumann wrote:
>
> If .a refers to libapr.a which contains the DSO-related
> functions, then the link editor will probably complain about
> undefined references, unless you link against libdl.
Why should it, if the library is only named in a conditional
satisfy mode (i.e., "-lapr" rather than "libapr.a")?
--
#ken P-)}
Ken Coar <http://Golux.Com/coar/>
Apache Software Foundation <http://www.apache.org/>
"Apache Server for Dummies" <http://Apache-Server.Com/>
"Apache Server Unleashed" <http://ApacheUnleashed.Com/>
Re: APR builds concept broken horribly
Posted by Sascha Schumann <sa...@schumann.cx>.
On Mon, 5 Jun 2000 rbb@covalent.net wrote:
> On Mon, 5 Jun 2000, Rodent of Unusual Size wrote:
>
> > rbb@covalent.net wrote:
> > >
> > > If the first, I don't know if I should be scared or impressed. Unix
> > > doesn't have that ability.
> >
> > Perhaps not.. but I don't see why we can't keep both the .so and
> > the .a files around, and link against whichever's appropriate.
> > Linking against the .a file will avoid htpasswd needing DSO. :-)
>
> Yep, I suggested this solution in some note somewhere today. :-)
If .a refers to libapr.a which contains the DSO-related
functions, then the link editor will probably complain about
undefined references, unless you link against libdl.
- Sascha
Re: APR builds concept broken horribly
Posted by rb...@covalent.net.
On Mon, 5 Jun 2000, Rodent of Unusual Size wrote:
> rbb@covalent.net wrote:
> >
> > If the first, I don't know if I should be scared or impressed. Unix
> > doesn't have that ability.
>
> Perhaps not.. but I don't see why we can't keep both the .so and
> the .a files around, and link against whichever's appropriate.
> Linking against the .a file will avoid htpasswd needing DSO. :-)
Yep, I suggested this solution in some note somewhere today. :-)
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
Re: APR builds concept broken horribly
Posted by Rodent of Unusual Size <Ke...@Golux.Com>.
Greg Stein wrote:
>
> Linking against the .so will avoid it, too. htpasswd isn't going
> to grow in side since you're linking against a .so. And if it
> doesn't use the DSO code, it isn't going to page the stuff in.
But having it linked against an .so makes it one hell of a lot less
portable. No more would you be able to copy *just* htpasswd
around; you'd need to copy the .so as well -- and ewhat end-user
is going to know that, or which file?
-1 on the support tools linking against .so files for this reason.
--
#ken P-)}
Ken Coar <http://Golux.Com/coar/>
Apache Software Foundation <http://www.apache.org/>
"Apache Server for Dummies" <http://Apache-Server.Com/>
"Apache Server Unleashed" <http://ApacheUnleashed.Com/>
Re: APR builds concept broken horribly
Posted by Greg Stein <gs...@lyra.org>.
On Mon, 5 Jun 2000, Rodent of Unusual Size wrote:
> rbb@covalent.net wrote:
> >
> > If the first, I don't know if I should be scared or impressed. Unix
> > doesn't have that ability.
>
> Perhaps not.. but I don't see why we can't keep both the .so and
> the .a files around, and link against whichever's appropriate.
> Linking against the .a file will avoid htpasswd needing DSO. :-)
Linking against the .so will avoid it, too. htpasswd isn't going to grow
in side since you're linking against a .so. And if it doesn't use the DSO
code, it isn't going to page the stuff in.
etc...
I see plenty of simplicity by keeping the library in a big unit, but not a
lot of rationale for divvying it into little pieces.
I don't see people complaining about libc-2.1.2.so being 4 meg. Why?
Because it doesn't negatively impact programs that use only a portion of
the library. But the simplicity? Quite a lot. I don't have to install
libc-threads, libc-io, libc-dso, libc-fork, libc-signals, libc-yourmom.
Cheers,
-g
--
Greg Stein, http://www.lyra.org/
Re: APR builds concept broken horribly
Posted by Rodent of Unusual Size <Ke...@Golux.Com>.
rbb@covalent.net wrote:
>
> If the first, I don't know if I should be scared or impressed. Unix
> doesn't have that ability.
Perhaps not.. but I don't see why we can't keep both the .so and
the .a files around, and link against whichever's appropriate.
Linking against the .a file will avoid htpasswd needing DSO. :-)
--
#ken P-)}
Ken Coar <http://Golux.Com/coar/>
Apache Software Foundation <http://www.apache.org/>
"Apache Server for Dummies" <http://Apache-Server.Com/>
"Apache Server Unleashed" <http://ApacheUnleashed.Com/>
RE: APR builds concept broken horribly
Posted by "William A. Rowe, Jr." <wr...@lnd.com>.
> From: rbb@covalent.net [mailto:rbb@covalent.net]
> Sent: Monday, June 05, 2000 1:26 PM
>
> > Yes, Win32 'suffered' the same problem.
> >
> > The short term solution has been to build APR into a monolithic library.
> > Any object can be linked out of that library for standalone apps, such
> > as htpasswd. Few objects are used in these cases, so the size of the
> > executable changes insignificantly.
>
> I can't really parse this. Does this mean that if I have a large DLL and
> a program that is only using a small part of the DLL then my small program
> just statically links to that small part? Or, is it just that we end of
> dynamically linking to the whole thing and only the parts needed are used?
NO...
Phase 1> Build apr.lib of all the source .obj files.
Phase 2> Link apr.lib objects into htdigest and other stand-alones.
These are -not- dynamically linked :-)
Phase 3> Build aprlib.dll from apr.lib objects.
Phase 4> Link aprlib.dll into more exhaustive apr clients, i.e. Apache.
This is a dynamic linkage that loads monolithic apr.
Phase 2 and 3 are interchangable, but I took them out of order to illustrate
the concept.
> If the first, I don't know if I should be scared or impressed. Unix
> doesn't have that ability. If the second, the same is done on Unix, so
> the executable doesn't get too large, but we end up having to dynamically
> link to things it doesn't need, which slows things down a bit.
I don't know the build environment... but rethink in terms of the steps above.
RE: APR builds concept broken horribly
Posted by Sascha Schumann <sa...@schumann.cx>.
On Mon, 5 Jun 2000 rbb@covalent.net wrote:
>
> > > If the second, the same is done on Unix, so
> > > the executable doesn't get too large, but we end up having to dynamically
> > > link to things it doesn't need, which slows things down a bit.
> >
> > Are you still advocating enabling Pthreads in APR? That slows
> > down things more than a bit, if Pthreads are not needed.
>
> APR only includes pthreads if it is found/asked for. This is two separate
> issues. One is building APR, the other is building APR with
> Apache. Either way, APR should be configured the same. APR has
> everything possible turned on, unless it is specifically told to turn
> something off. If you are building a prefork Apache, then you should be
> manually telling APR to build without threads. But that is not this
> discussion. :-)
That is a fundamental disagreement, unfortunately.
> dependancy: file_io time
> package
> file_io x
> threadroc x
> locks x
> misc x (for other child could be made its own library.)
>
> Those are all of the dependancies I can think of off the top of my head.
Ok, so we would need a tool which spits out a list of
libraries which a program needs to be linked against, if it
needs one or more APIs. I.e.
$ apr-config file_io
-Lpath/unix/file_io -lapr_file_io -Lpath/unix/time -lapr_time ...
- Sascha
RE: APR builds concept broken horribly
Posted by rb...@covalent.net.
> > If the second, the same is done on Unix, so
> > the executable doesn't get too large, but we end up having to dynamically
> > link to things it doesn't need, which slows things down a bit.
>
> Are you still advocating enabling Pthreads in APR? That slows
> down things more than a bit, if Pthreads are not needed.
APR only includes pthreads if it is found/asked for. This is two separate
issues. One is building APR, the other is building APR with
Apache. Either way, APR should be configured the same. APR has
everything possible turned on, unless it is specifically told to turn
something off. If you are building a prefork Apache, then you should be
manually telling APR to build without threads. But that is not this
discussion. :-)
> > The solution I would advocate would be to build each individual part of
> > APR as a small library, and to build the huge monolithic library. Then,
> > htpasswd just links against the small static libraries that it needs, and
> > Apache uses the big one either shared or static (I have no opinion about
> > that today).
>
> If APR is a collection of independent APIs, then the
> mini-library approach works. But if any of these libraries
> depend on another one, we would need to introduce further
> dependencies between the mini-libraries.
>
> On the other side, the huge-library approach might waste some
> time on program startup. But that is probably negligible.
For the most part, APR is a collection of independant API's. There are a
few that are required, like pools and time. But more of them are not
dependant on each other. Basically:
(everything is dependant on libs because it provides pools so I am leaving
that out.)
dependancy: file_io time
package
file_io x
threadroc x
locks x
misc x (for other child could be made its own library.)
Those are all of the dependancies I can think of off the top of my head.
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
RE: APR builds concept broken horribly
Posted by Sascha Schumann <sa...@schumann.cx>.
On Mon, 5 Jun 2000 rbb@covalent.net wrote:
> If the second, the same is done on Unix, so
> the executable doesn't get too large, but we end up having to dynamically
> link to things it doesn't need, which slows things down a bit.
Are you still advocating enabling Pthreads in APR? That slows
down things more than a bit, if Pthreads are not needed.
> > That monolithic library is then linked into a windows dso (.dll) and
> > can be bound to any 'bigger' program that uses a more complete subset
> > of the APR.
> >
> > I don't know if an equivilant build logic exists for unix. But I don't
> > know that you want to haul in 10 seperate 'smaller' dso libraries to
> > load the Apache server, either.
>
> The solution I would advocate would be to build each individual part of
> APR as a small library, and to build the huge monolithic library. Then,
> htpasswd just links against the small static libraries that it needs, and
> Apache uses the big one either shared or static (I have no opinion about
> that today).
If APR is a collection of independent APIs, then the
mini-library approach works. But if any of these libraries
depend on another one, we would need to introduce further
dependencies between the mini-libraries.
On the other side, the huge-library approach might waste some
time on program startup. But that is probably negligible.
- Sascha
RE: APR builds concept broken horribly
Posted by rb...@covalent.net.
> Yes, Win32 'suffered' the same problem.
>
> The short term solution has been to build APR into a monolithic library.
> Any object can be linked out of that library for standalone apps, such
> as htpasswd. Few objects are used in these cases, so the size of the
> executable changes insignificantly.
I can't really parse this. Does this mean that if I have a large DLL and
a program that is only using a small part of the DLL then my small program
just statically links to that small part? Or, is it just that we end of
dynamically linking to the whole thing and only the parts needed are used?
If the first, I don't know if I should be scared or impressed. Unix
doesn't have that ability. If the second, the same is done on Unix, so
the executable doesn't get too large, but we end up having to dynamically
link to things it doesn't need, which slows things down a bit.
> That monolithic library is then linked into a windows dso (.dll) and
> can be bound to any 'bigger' program that uses a more complete subset
> of the APR.
>
> I don't know if an equivilant build logic exists for unix. But I don't
> know that you want to haul in 10 seperate 'smaller' dso libraries to
> load the Apache server, either.
The solution I would advocate would be to build each individual part of
APR as a small library, and to build the huge monolithic library. Then,
htpasswd just links against the small static libraries that it needs, and
Apache uses the big one either shared or static (I have no opinion about
that today).
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
RE: APR builds concept broken horribly
Posted by "William A. Rowe, Jr." <wr...@lnd.com>.
> From: rbb@covalent.net [mailto:rbb@covalent.net]
> Sent: Monday, June 05, 2000 12:46 PM
>
> Okay, way back when, when APR was first written, it was possible to build
> individual sections of APR independantly on Unix. This provided us with
> small compact libraries that could be included or not based on the
> program. This feature was removed a long time ago, and it needs to be
> added back.
>
> The problem was highlighted eariler today with the whold dso issue. If
> htpasswd is an APR'ized program, and APR can only be built as one huge
> library, then htpasswd all of a sudden needs DSO support. That is
> ludicrous. The other option is to comfigure and compile APR multiple
> times in one Apache build. Once for Apache itself and once for the
> support programs.
>
> If APR could be built as small libraries again, then htpasswd could just
> link with the libs/file_io stuff and ignore the rest, which makes more
> sense IMHO.
>
> Thoughts?
Yes, Win32 'suffered' the same problem.
The short term solution has been to build APR into a monolithic library.
Any object can be linked out of that library for standalone apps, such
as htpasswd. Few objects are used in these cases, so the size of the
executable changes insignificantly.
That monolithic library is then linked into a windows dso (.dll) and
can be bound to any 'bigger' program that uses a more complete subset
of the APR.
I don't know if an equivilant build logic exists for unix. But I don't
know that you want to haul in 10 seperate 'smaller' dso libraries to
load the Apache server, either.
Bill