You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by Paul Sutton <pa...@awe.com> on 1997/12/07 16:05:21 UTC
Shared modules on Unix
I've been playing with mod_so on Unix, and it looks pretty good. It would
very nice to allow third party modules to be loaded at run time, or even
normal Apache modules. We could distribute, for example, a binary
containing just the default modules, with extra modules available as
shared libraries (e.g. mod_rewrite, mod_proxy) for loading only _if_ the
end use wants them.
Anyway to make compiling modules for shared libraries easier, the patch
below updates Configure. Now as well as AddModule you can also use
SharedModule, like this:
SharedModule modules/standard/mod_rewrite.so
(you have to change the extension from .o to .so). When ./Configure runs
it will create module makefiles that build the SharedModule modules to .so
files. To do this Configure has to know about the compiler and linker
options required: I've put some default ones in for FreeBSD and Linux.
This can also be set in the Configuration file (if set here, it will
override the defaults contained within the Configure script). Oh, and if
there are no SharedModule lines none of these extra options will be used
anywhere in the build process.
The Configure variables and Configuration options are
CFLAGS_SHLIB Options when building .o files ready for sharing
(e.g. -fpic)
LDFLAGS_SHLIB Options when linking .o files to .so (e.g. -Bshareable,
-export, -assert pure-text)
LDFLAGS_SHLIB_EXPORT
Options required when linking httpd so that it exports
its symbols for linking at runtime (e.g. -Bdynamic,
-rdynamic, -export-dynamic)
The options used in Configure could be placed in Configuration like this:
CFLAGS_SHLIB=-fpic
LDFLAGS_SHLIB=-Bshareable
LDFLAGS_SHLIB_EXPORT=-rdynamic
Other systems may need to alter these.
I've tried this out by converting _all_ the AddModules to SharedModule
lines, and adding AddModule modules/extra/mod_so.o and it seems to work.
You can get a minimal httpd (just http_core.o and mod_so.o) and
dynamically load every other module. I've only been testing on Linux.
//pcs
Index: Configure
===================================================================
RCS file: /export/home/cvs/apachen/src/Configure,v
retrieving revision 1.173
diff -u -r1.173 Configure
--- Configure 1997/11/30 19:17:22 1.173
+++ Configure 1997/12/07 14:15:57
@@ -69,9 +69,12 @@
sed 's/^Rule[ ]*/##Rule:/' | \
sed 's/^[ ]*AddModule/AddModule/' | \
sed 's/^[ ]*%AddModule/%AddModule/' | \
+ sed 's/^[ ]*SharedModule/SharedModule/' | \
sed 's/^[ ]*Module/Module/' | \
sed 's/^[ ]*%Module/%Module/' > $tmpfile
+using_shlib=`grep -q ^SharedModule $tmpfile && echo 1`
+
#
# Only "assignment" ("=") statements and Module lines
# should be left at this point. If there is other stuff
@@ -79,6 +82,7 @@
#
if egrep -v '^%?Module[ ]+[A-Za-z0-9_]+[ ]+[^ ]+$' $tmpfile \
| egrep -v '^%?AddModule[ ]+[^ ]+$' \
+ | egrep -v '^SharedModule[ ]+[^ ]+$' \
| grep -v = > /dev/null
then
echo "Syntax error --- The configuration file is used only to"
@@ -86,6 +90,7 @@
echo "options or Configure rules, and I don't see that at all:"
egrep -v '^%?Module[ ]+[A-Za-z0-9_]+[ ]+[^ ]+$' $tmpfile \
| egrep -v '^%?AddModule[ ]+[^ ]+$' \
+ | egrep -v '^%?SharedModule[ ]+[^ ]+$' \
| grep -v =
exitcode=1
exit 1
@@ -312,11 +317,15 @@
OS='Linux'
CFLAGS="$CFLAGS -DLINUX=2"
LIBS="$LIBS -lm"
+ CFLAGS_SHLIB="-fpic"
+ LDFLAGS_SHLIB="-Bshareable"
;;
*-linux1)
DEF_WANTHSREGEX=yes
OS='Linux'
CFLAGS="$CFLAGS -DLINUX=1"
+ CFLAGS_SHLIB="-fpic"
+ LDFLAGS_SHLIB="-Bshareable"
;;
*-lynx-lynxos)
OS='LynxOS 2.x'
@@ -348,6 +357,8 @@
LIBS="$LIBS -lcrypt"
DBM_LIB=""
DB_LIB=""
+ CFLAGS_SHLIB="-fpic"
+ LDFLAGS_SHLIB="-Bshareable"
;;
*-openbsd*)
OS='OpenBSD'
@@ -624,6 +635,8 @@
# Look for OPTIM and save for later
#
TOPTIM=`egrep '^OPTIM=' Makefile.config | tail -1 | awk -F= '{print $2}'`
+TLDFLAGS_SHLIB=`egrep '^LDFLAGS_SHLIB=' Makefile.config | tail -1 | awk -F= '{print $2}'`
+TCFLAGS_SHLIB=`egrep '^CFLAGS_SHLIB=' Makefile.config | tail -1 | awk -F= '{print $2}'`
####################################################################
# Set the value of CC if need be
@@ -632,6 +645,26 @@
echo "CC=$CC" >> Makefile.config
fi
+if [ "x$using_shlib" = "x1" ] ; then
+ #
+ # Set the value of the shared libary flags, if they aren't explicitly
+ # set in the configuration file
+ #
+ if [ "x$TCFLAGS_SHLIB" = "x" ]; then
+ echo "CFLAGS_SHLIB=$CFLAGS_SHLIB" >> Makefile.config
+ fi
+ if [ "x$TLDFLAGS_SHLIB" = "x" ]; then
+ echo "LDFLAGS_SHLIB=$LDFLAGS_SHLIB" >> Makefile.config
+ fi
+ if [ "x$TLDFLAGS_SHLIB_EXPORT" = "x" ]; then
+ if [ "x$TCC" = "xgcc" ] ||
+ [ "x$TCC" = "x" -a "x$CC" = "xgcc" ] ; then
+ LDFLAGS_SHLIB_EXPORT=-rdynamic
+ fi
+ echo "LDFLAGS_SHLIB_EXPORT=$LDFLAGS_SHLIB_EXPORT" >> Makefile.config
+ fi
+fi # end of $using_shlib section
+
####################################################################
# Now we do some OS specific adjustments... for some OSs, we need
# to adjust CFLAGS and/or OPTIM depending on which compiler we
@@ -782,7 +815,7 @@
SEEN[pp[2]] = 1
}
}
- ($1 == "AddModule" && $2 ~ /^modules\//) {
+ (($1 == "AddModule" || $1 == "SharedModule") && $2 ~ /^modules\//) {
split ($2, pp, "/")
if (! SEEN[pp[2]]) {
printf "%s ", pp[2]
@@ -1078,10 +1111,35 @@
printf "\n"
}'
+ cat << 'EOF' >> $moddir/Makefile
+
+all: lib shlib
+
+EOF
echo "LIB=lib$basedir.a" >> $moddir/Makefile
+ awk >> $moddir/Makefile < $tmpfile '
+ ($1 == "SharedModule" && $2 ~ /^modules\/'$basedir'\//) {
+ split($2, pp, "/")
+ shlibs=shlibs " " pp[3]
+ so=pp[3]
+ split(pp[3], parts, ".")
+ base=parts[1]
+ shlibsobj=shlibsobj " " base "-so.o"
+ comp=comp base ".so: " base "-so.o\n"
+ comp=comp " $(LD) $(LDFLAGS) $(LDFLAGS_SHLIB) -o " base ".so $<\n"
+ comp=comp base "-so.o: " base ".c\n"
+ comp=comp " $(CC) $(CFLAGS) $(INCLUDES) $(CFLAGS_SHLIB) -c -o " base "-so.o $<\n"
+ }
+ END {
+ printf "SHLIBS=%s\n", shlibs;
+ printf "SHLIBS_OBJ=%s\n", shlibsobj;
+ print "\n" comp "\n" }'
+
cat << 'EOF' >> $moddir/Makefile
-all: $(LIB)
+lib: $(LIB)
+
+shlib: $(SHLIBS)
$(LIB): $(OBJS)
rm -f $@
@@ -1092,9 +1150,9 @@
$(CC) -c $(INCLUDES) $(CFLAGS) $(SPACER) $<
clean:
- rm -f *.o $(LIB)
+ rm -f *.o $(LIB) $(SHLIB)
-$(OBJS): Makefile
+$(OBJS) $(SHLIBS) $(SHLIBS_OBJ): Makefile
EOF
fi
Index: Makefile.tmpl
===================================================================
RCS file: /export/home/cvs/apachen/src/Makefile.tmpl,v
retrieving revision 1.73
diff -u -r1.73 Makefile.tmpl
--- Makefile.tmpl 1997/11/30 19:17:23 1.73
+++ Makefile.tmpl 1997/12/07 14:15:57
@@ -29,7 +29,7 @@
$(TARGET): subdirs modules.o
$(CC) -c $(CFLAGS) buildmark.c
- $(CC) $(CFLAGS) $(LDFLAGS) -o $(TARGET) buildmark.o $(OBJS) $(REGLIB) $(LIBS)
+ $(CC) $(CFLAGS) $(LDFLAGS) $(LDFLAGS_SHLIB_EXPORT) -o $(TARGET) buildmark.o $(OBJS) $(REGLIB) $(LIBS)
subdirs:
for i in $(SUBDIRS); do \
Re: Shared modules on Unix
Posted by Paul Sutton <pa...@eu.c2.net>.
On Mon, 8 Dec 1997, Martin Kraemer wrote:
> > +using_shlib=`grep -q ^SharedModule $tmpfile && echo 1`
> ^^^^^^^
> This is not portable enough; older greps don't have -q (yet);
> besides, plain vanilla bourne shells interpret '^' identical to '|',
> i.e., as a pipe sign. Therefore,
>
> using_shlib=`grep '^SharedModule' $tmpfile >/dev/null && echo 1`
>
> would be a more defensive approach.
Yep, sounds good. I knew I shouldn't have used -q since I'd never heard of
it before I read man grep recently.
> Other than that, I like the patch *very much* -- I just didn't find out
> the option for my system yet to make the linker export the apache symbols
> for use by the .so objects :-( will keep trying, though!
Umm, I'm still getting some little problems... like modules can't access
add_common_vars(), but they call add_cgi_vars() without problem, and both
are defined identically as far as I can tell. Umm.
//pcs
Re: Shared modules on Unix
Posted by Martin Kraemer <Ma...@mch.sni.de>.
On Sun, Dec 07, 1997 at 03:05:21PM +0000, Paul Sutton wrote:
> Anyway to make compiling modules for shared libraries easier, the patch
> below updates Configure. Now as well as AddModule you can also use
> SharedModule, like this:
>
> SharedModule modules/standard/mod_rewrite.so
>
> +using_shlib=`grep -q ^SharedModule $tmpfile && echo 1`
^^^^^^^
This is not portable enough; older greps don't have -q (yet);
besides, plain vanilla bourne shells interpret '^' identical to '|',
i.e., as a pipe sign. Therefore,
using_shlib=`grep '^SharedModule' $tmpfile >/dev/null && echo 1`
would be a more defensive approach.
Other than that, I like the patch *very much* -- I just didn't find out
the option for my system yet to make the linker export the apache symbols
for use by the .so objects :-( will keep trying, though!
Martin
--
| S I E M E N S | <Ma...@mch.sni.de> | Siemens Nixdorf
| ------------- | Voice: +49-89-636-46021 | Informationssysteme AG
| N I X D O R F | FAX: +49-89-636-44994 | 81730 Munich, Germany
~~~~~~~~~~~~~~~~My opinions only, of course; pgp key available on request
Re: Shared modules on Unix
Posted by Ben Laurie <be...@algroup.co.uk>.
Alexei Kosut wrote:
> Generally, courses will last for one semester or one quarter, no longer.
> Material that takes longer to cover is broken up into more than one class.
> So every class has a final when it is over. Which, for me, is right about
> now.
>
> Hopefully that's clear. How does it work on the other side of the pond?
That depends on the university and the course. Broadly speaking,
however, you do your finals at the end of the course (i.e. just before
you get your degree). There are usually other exams you may or may not
have to pass, which may or may not contribute towards your final mark.
To give an example (the one I'm most familiar with, since it is what I
did :-), to get a degree at Cambridge, you do what is known as a tripos.
In keeping with the wacky numbering schemes universities seem to favour,
a tripos is actually in two parts[1], known, appropriately enough as
Part I and Part II. You have to pass both parts to get an honours degree
(although it is technically possible to get an ordinary degree, they'd
sooner fling you out than allow it to happen). The two parts are
semi-independent, apart from some arcane rules. The arcane rules for
Maths (which is what I did) are that you must pass Part I to do Part
II[2], and that if you do a Part II in Computer Science, you must do the
length of Part II (did I mention that Parts I and II can each be 1 or 2
years, and you have to do at least 3 years?) that makes three years
total from your last qualifying exam (this means that if you switch from
Maths to Computer Science after Part I you must do a 2 year Part II in 1
year, and appears to be the sole reason for the rule's existence).
[1] Actually it isn't as crazed as it seems. The tripos is probably
named after a geezer named "Mr. Tripos" who gave a speech from a
three-legged stool.
[2] In fact, Maths Part I is split into two parts, knows as Ia and Ib.
Ia, strangely, is the qualifying exam for Part I (i.e. it counts towards
the tripos), but you have to pass Ib to do Part II Maths (but no other
Part II). Only Maths is like this.
Is that clear? :-)
I think most other unis are considerably less arcane. Isn't tradition
wonderful?
Cheers,
Ben.
--
Ben Laurie |Phone: +44 (181) 735 0686|Apache Group member
Freelance Consultant |Fax: +44 (181) 735 0689|http://www.apache.org
and Technical Director|Email: ben@algroup.co.uk |Apache-SSL author
A.L. Digital Ltd, |http://www.algroup.co.uk/Apache-SSL
London, England. |"Apache: TDG" http://www.ora.com/catalog/apache
Re: Shared modules on Unix
Posted by Alexei Kosut <ak...@leland.Stanford.EDU>.
On Sun, 7 Dec 1997, Ben Laurie wrote:
> Alexei Kosut wrote:
> > I think this is something I'll get more into when I think more closely
> > about the Apache 2.0 API. But right now I have finals, so I'll stop
> > thinking now (no, wait, that didn't come out right...)
>
> Finals? Isn't a bit soon for those? Or did I just fall asleep for 3
> years? Or is a final something different over there?
Probably a bit of all three. This reminds me of a thread that was on
rec.arts.sf.tv.babylon5.moderated a month or so ago that involved US and
UK residents explaining their education systems to each other. I noticed
that a good chunk of the US people didn't quite understand their own
system (some things - like what SAT stands for - have changed recently, so
I forgive them).
At any rate, US undergraduate schools are generally run on either the
semester or the quarter system. There are two semesters in an academic
year, one from September through January (or August
through December), and the other from February (or January) through May.
An academic year can also consist of three quarters (summer is the forth),
generally spanning the months of (late) September through December,
January through March, and April through mid-June. Stanford is on the
quarter system, so right now is the end of the Fall quarter for me.
Generally, courses will last for one semester or one quarter, no longer.
Material that takes longer to cover is broken up into more than one class.
So every class has a final when it is over. Which, for me, is right about
now.
Hopefully that's clear. How does it work on the other side of the pond?
-- Alexei Kosut <ak...@stanford.edu> <http://www.stanford.edu/~akosut/>
Stanford University, Class of 2001 * Apache <http://www.apache.org> *
Re: Shared modules on Unix
Posted by Ben Laurie <be...@algroup.co.uk>.
Alexei Kosut wrote:
> I think this is something I'll get more into when I think more closely
> about the Apache 2.0 API. But right now I have finals, so I'll stop
> thinking now (no, wait, that didn't come out right...)
Finals? Isn't a bit soon for those? Or did I just fall asleep for 3
years? Or is a final something different over there?
Cheers,
Ben.
--
Ben Laurie |Phone: +44 (181) 735 0686|Apache Group member
Freelance Consultant |Fax: +44 (181) 735 0689|http://www.apache.org
and Technical Director|Email: ben@algroup.co.uk |Apache-SSL author
A.L. Digital Ltd, |http://www.algroup.co.uk/Apache-SSL
London, England. |"Apache: TDG" http://www.ora.com/catalog/apache
Re: Shared modules on Unix
Posted by Alexei Kosut <ak...@leland.Stanford.EDU>.
On Sun, 7 Dec 1997, Paul Sutton wrote:
> Anyway to make compiling modules for shared libraries easier, the patch
> below updates Configure. Now as well as AddModule you can also use
> SharedModule, like this:
>
> SharedModule modules/standard/mod_rewrite.so
Interesting. Haven't tried it yet, but it's a good idea, although I'm not
sure if we should by default on Unix make any modules as shared libaries
(IMHO, it's still too inconsistent).
One thing we should think about is including a -DSHARED_MODULE or
something in the compiler line when a module is being compiled that way,
so it knows and can maybe do things differently (what, I don't know).
I think this is something I'll get more into when I think more closely
about the Apache 2.0 API. But right now I have finals, so I'll stop
thinking now (no, wait, that didn't come out right...)
-- Alexei Kosut <ak...@stanford.edu> <http://www.stanford.edu/~akosut/>
Stanford University, Class of 2001 * Apache <http://www.apache.org> *