You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@apr.apache.org by Chris Darroch <ch...@pearsoncmg.com> on 2005/11/15 21:00:38 UTC
[Fwd: Re: apr-util config, etc.]
Hi --
I've encountered some issues with the apr-util build/configure
system that I thought I might pass on for wiser minds than mine
to review.
The crux of the problem seems to be twofold. First, it's
unclear to me what provision the apr-util build system has for
libraries (such as commercial binary-only libraries) that lack
.la libtool archive files. My current solution to that problem
is to add -R/path/to/lib alongside the -L/path/to/lib in the
apr-util buildconf process. (Bear with me if it seems dumb to
be fiddling with the buildconf process.)
Second, because the httpd build process depends on the output
from apu-1-config, and specifically the output of
apu-1-config --ldflags and --libs, certain types of library
dependency problems get passed along to it, and I'm not sure
that's really necessary.
My ultimate question, for the impatient, is whether adding
-R to APRUTIL_LDFLAGS is an appropriate solution when using
libraries that one expects not to have accompanying .la files.
Here's a second level of detail. So long as the libraries
that are detected by the autoconf process have .la libtool
archive files as well as the .so files, all is well.
If one has a library that lacks a .la file, though, trouble
creeps in. (If you're thinking, "Don't do that!", do bear with me.)
You can simulate this problem by, for example, hiding your
normal iconv libraries, installing new ones in a private location,
and then removing the .la files.
The libaprutil-1.so library still builds and installs "OK",
but contains a problem that crops up when you run "make check":
it has unresolved paths to its dependent libraries. Probing with
ldd will show, for example:
libiconv.so.2 => not found
If you then compile httpd with this libaprutil-1.so, it'll
do so but fail to run with the obvious error, "libiconv.so.2:
cannot open shared object file: No such file or directory".
So that's the first problem, in essence -- lack of .la files
causes the libaprutil-1.so library to have undefined paths to
its dependent libraries, even though it compiled "OK".
The next problem -- if indeed it is one -- turned up as I
began to investigate ways to solve the first one. My initial
attempt was to hack into apr-util's build/rules.mk file and
add something like:
NOTEST_LDFLAGS=-R/path/to/lib
The purpose here is to ensure that the -R option gets added
to the libtool command link created by $LINK in rules.mk, and
specifically somewhere after $COMPILE, i.e., like this:
.../libtool ... --mode=link ... gcc ... \
-version-info 2:1:2 ... -R/path/to/lib ...
This corresponds to the advice I could find in the libtool
info pages; see, for example, libtool --mode=link --help.
Now I didn't have a working solution yet, because this was
just a post-buildconf, post-configure hack. But, OTOH, the
libaprutil-1.so was OK, with no undefined dependencies listed
by ldd.
OK, so now I went over and compiled and installed my httpd.
But it didn't run, with the same old error: "libiconv.so.2:
cannot open shared object file...." Checking the httpd
binary with ldd, I see that it has, weirdly:
libiconv.so.2 => not found
...
libiconv.so.2 => /path/to/lib/libiconv.so.2
Huh? This appears to be the consequence of building httpd
with the output of apu-1-config --libs and --ldflags. The final
step of the httpd make process is, roughly:
.../libtool --mode=link ... $EXTRA_LDFLAGS ... $EXTRA_LIBS ...
where EXTRA_LDFLAGS contains the output of apu-1-config --ldflags
and EXTRA_LIBS contains the output of apu-1-config --libs.
And EXTRA_LDFLAGS doesn't contain my -R/path/to/lib hack because
I slipped it into apr-util's build/rules.mk by hand, after
running configure, so it's not in apr-1-config.
So the second problem -- if it is one -- is that because
httpd's configure process imports the list of -L and -l (and -I)
options from apr-util (and apr too, of course), it re-links
the httpd binary against libraries that perhaps it does not
need to. In the contrived instance above, I'm not sure if there's
any reason for httpd to link against -liconv: after all, that's
a dependency of libaprutil-1.so, not httpd.
My understanding from the libtool documentation is that certain
older Unix-like systems, such as perhaps AIX, don't allow inter-
library dependencies. So that may be the reason for importing
the whole list of -L and -l options from apr and apr-util, even
on a system like Linux or Solaris, where inter-library dependencies
are resolved by ld.
Still, I thought I'd point it out, because it only turned up
as a result of an unusual hack, but then it did occur to me to
wonder why httpd needed to link against libraries that apr-util
abstracts and "hides".
So that's a summary, now some rationale. The use of libiconv
in the examples above was just for demonstration (but it is
reproducible, if you remove the .la file). My real problem is
with certain commercial libraries that are distributed as
binaries, and without .la files, specifically Oracle's
libclntsh.so. I've been working with Nick Kew to complete
a first pass at the code for an apr_dbd_oracle.c driver and
accompanying autoconf magic. So far, my best solution to the
problems described above is what I initially stated: add -R to
APRUTIL_LDFLAGS alongside the -L for Oracle's library.
This -R solves the first problem by ensuring that
libaprutil-1.so has no dependency problems itself.
Next, because it gets output by apu-1-config --ldflags, it's
used during httpd's compilation as well, so the fact that
httpd re-links against Oracle's library (because of the -l output
by apu-1-config --libs) doesn't cause dependency problems
either; probing the resultant httpd with ldd reveals that
it lists libclntsh.so twice, both with full paths and no
"not found" problems.
Still, as I said, it seems odd for httpd to need to link
against Oracle's library when it's fully abstracted behind
apr-util, at least on an OS that supports inter-library
dependencies.
Thus, back to my original question: is -R appropriate?
Is there a better way of linking against non-libtool-generated
libraries?
I should say, this is all on Linux, specifically RedHat;
libtool 1.5.6, autoconf 2.59, ld 2.15.92.0.2, gcc 3.4.3,
apr 1.2.1, apr-util 1.2.1, and httpd 2.1.8-beta.
Chris.
--
GPG Key ID: 366A375B
GPG Key Fingerprint: 485E 5041 17E1 E2BB C263 E4DE C8E3 FA36 366A 375B
Re: apr-util config, etc.
Posted by Nick Kew <ni...@webthing.com>.
On Tuesday 15 November 2005 20:44, Justin Erenkrantz wrote:
> --On November 15, 2005 3:36:39 PM -0500 Chris Darroch
(For the record, Chris and I have developed an apr_dbd_oracle
driver which we'll soon be contributing to APR)
> > At any rate, I've still got a problem on my hands, because
> > to deal with Oracle I need to get something into $LINK after
> > $COMPILE, which means one of LDFLAGS, EXTRA_LDFLAGS, or
> > NOTEST_LDFLAGS in build/rules.mk. No amount of fiddling
> > with prior-to-configure env vars has allowed me to do this;
> > although things turn up in config.nice, they don't make it
> > into the actual make process. Any help appreciated! Thanks!
>
> I'm not sure what your problem is here.
Oracle requires not just its own paths for headers and client lib,
but also its own environment variables. For runtime, putting it
in httpd's envvars by hand works fine, but it doesn't automate
the build.
--
Nick Kew
Re: apr-util config, etc.
Posted by Chris Darroch <ch...@pearsoncmg.com>.
Hi --
> I see that apr-util's configure script simply copies its
> build/rules.mk from apr, at line 168 of configure.in:
>
> cp $APR_BUILD_DIR/apr_rules.mk $abs_builddir/build/rules.mk
>
> So that's perhaps the key to my trouble. I was expecting that
> setting LDFLAGS prior to running apr-util's configure would
> work similarly to apr's, but it appears only to affect the test
> programs; once the make process begins, the values from
> build/rules.mk (aka apr's apr_rules.mk) take over, and whatever I
> put in my env vars is lost.
Sorry to inquire twice, but I thought it might clarify this
for myself ... is this the expected behaviour of the apr-util
configure?
Chris.
--
GPG Key ID: 366A375B
GPG Key Fingerprint: 485E 5041 17E1 E2BB C263 E4DE C8E3 FA36 366A 375B
Re: apr-util config, etc.
Posted by Chris Darroch <ch...@pearsoncmg.com>.
Justin:
>> I do the same to get Berkeley DB to work with apr-util, with
>> the -Wl,-R option. That seems to be sufficient to ensure that
>> configure's test programs compile and execute. (For some reason,
>> just having -L doesn't do it, and configure wrongly decides that
>> BDB can't be found because one of the test programs fails.)
>
> Yes, the -L flag needs to be added by configure. Otherwise, it won't know
> what to do. =)
Right ... I misspoke there slightly. What I mean is that,
for me at least, doing just this:
./configure --with-dbm=db43 \
--with-berkeley-db=/path/to/bdb/include:/path/to/bdb/lib \
...
isn't sufficient, because although configure internally adds
-L/path/to/bdb/lib when compiling its test programs, they
still fail to execute without an additional -R option. So,
doing this:
LDFLAGS=-Wl,-R/path/to/bdb/lib \
./configure --with-dbm=db43 \
--with-berkeley-db=/path/to/bdb/include:/path/to/bdb/lib \
...
works nicely. The -Wl,-R is passed to gcc when compiling the
test programs during configure, the test programs execute,
and configure accepts that Berkeley has been found. But that's
all just a tangent. :-)
>> However, if I read the situation correctly, that LDFLAGS setting
>> doesn't actually make it into make process itself; certainly it
>> doesn't turn up outside of config.{log,nice,status} or in make's
>> output. And in both the "fake" libiconv and "real" Oracle instances
>
> Well, it should be preserved in the GNU libtool interfaces. That means
> that if you perform a libtool link against libaprutil-1.la, the run-time
> linker flags should be preserved.
>
> There are some version of GNU libtool that are broken in this respect
> though. Please ensure you are running the latest version of GNU libtool
> (1.5.20 should work fine). IIRC, all 1.5.x version of GNU libtool prior to
> 1.5.10 are broken on most platforms in this regard.
I'm running 1.5.14, because that's what is packaged in
apr 1.2.1. I'm sort of inclinded not to fool with that, since
it's what other folks will be using too, I assume. (Note: I
had initially said 1.5.6, but that's because I was testing the
system's libtool, not apr's.)
Looking at libaprutil-1.la, nothing that I put in LDFLAGS or
NOTEST_LDFLAGS prior to running configure shows up. I see my LDFLAGS
values in config.nice, but nowhere else. Note that I'm running
only apr-util's configure script here.
I see that apr-util's configure script simply copies its
build/rules.mk from apr, at line 168 of configure.in:
cp $APR_BUILD_DIR/apr_rules.mk $abs_builddir/build/rules.mk
So that's perhaps the key to my trouble. I was expecting that
setting LDFLAGS prior to running apr-util's configure would
work similarly to apr's, but it appears only to affect the test
programs; once the make process begins, the values from
build/rules.mk (aka apr's apr_rules.mk) take over, and whatever I
put in my env vars is lost.
> I'm not sure what your problem is here.
>
> Can you provide some more details?
Well, working backwards, my problem is this: I've got a
commerical library that lacks a .la file. That seems to
require a -R flag to libtool and ld; otherwise, libaprutil-1.so
is built with an unresolved dependency (the path to Oracle's
library is "not found" when listed by ldd).
Now, if putting -R into the buildconf/configure process
isn't advisable, I still need some way to get it into the final
libtool --mode=link step. That step is generated by the $LINK
variable in build/rules.mk. The -R has to appear after the $COMPILE
value (i.e., after the "gcc" LINK-COMMAND to libtool). That means it
needs to get into $ALL_LDFLAGS, which means it needs to come from
$LDFLAGS, $EXTRA_LDFLAGS, or $NOTEST_LDFLAGS in build/rules.mk.
But, that whole file seems to come out of apr, not apr-util.
So what do I advise people who might want to compile apr-util
with apr_dbd_oracle.c to do? I guess I have to tell them to go
back to apr, re-build that with LDFLAGS=-R/path/to/oracle/lib,
re-install, and then do apr-util.
That seems wrong to me, though -- if adding -R options
automatically is verboten in the configure process, then I'd
hope for a way to do so through LDFLAGS or a similar env var
prior to running apr-util's configure (i.e., after you've
read that ./configure --help :-)
Thanks for all the help and insightful queries,
Chris.
--
GPG Key ID: 366A375B
GPG Key Fingerprint: 485E 5041 17E1 E2BB C263 E4DE C8E3 FA36 366A 375B
Re: apr-util config, etc.
Posted by Justin Erenkrantz <ju...@erenkrantz.com>.
--On November 15, 2005 3:36:39 PM -0500 Chris Darroch
<ch...@pearsoncmg.com> wrote:
> OK, understood, and sorry to recapitulate an old discussion.
> (Also apologies for the strange subject line -- too much proofreading
> of the body!)
No need to apologize. I just wanted to indicate that we've thought about
this problem in the past, and the best resolution we came up with was to
agree to disagree about the merits of having APR-util be aware of the -R
flags.
>> Setting LDFLAGS before running configure should work, like so:
>>
>> LDFLAGS="-R/path/to/lib" \
>> ./configure
>>
>> is an acceptable workaround that should work. It's what I use.
>
> I do the same to get Berkeley DB to work with apr-util, with
> the -Wl,-R option. That seems to be sufficient to ensure that
> configure's test programs compile and execute. (For some reason,
> just having -L doesn't do it, and configure wrongly decides that
> BDB can't be found because one of the test programs fails.)
Yes, the -L flag needs to be added by configure. Otherwise, it won't know
what to do. =)
> However, if I read the situation correctly, that LDFLAGS setting
> doesn't actually make it into make process itself; certainly it
> doesn't turn up outside of config.{log,nice,status} or in make's
> output. And in both the "fake" libiconv and "real" Oracle instances
Well, it should be preserved in the GNU libtool interfaces. That means
that if you perform a libtool link against libaprutil-1.la, the run-time
linker flags should be preserved.
There are some version of GNU libtool that are broken in this respect
though. Please ensure you are running the latest version of GNU libtool
(1.5.20 should work fine). IIRC, all 1.5.x version of GNU libtool prior to
1.5.10 are broken on most platforms in this regard.
> At any rate, I've still got a problem on my hands, because
> to deal with Oracle I need to get something into $LINK after
> $COMPILE, which means one of LDFLAGS, EXTRA_LDFLAGS, or
> NOTEST_LDFLAGS in build/rules.mk. No amount of fiddling
> with prior-to-configure env vars has allowed me to do this;
> although things turn up in config.nice, they don't make it
> into the actual make process. Any help appreciated! Thanks!
I'm not sure what your problem is here.
Can you provide some more details? -- justin
Re: apr-util config, etc.
Posted by Chris Darroch <ch...@pearsoncmg.com>.
Justin:
>> My ultimate question, for the impatient, is whether adding
>> -R to APRUTIL_LDFLAGS is an appropriate solution when using
>> libraries that one expects not to have accompanying .la files.
>
> We've had this discussion several times. There's a group of developers who
> believe we should automatically add the -R flags and an equally vocal
> developers who believe that the user must hand-edit ldconfig on their
> platform.
>
> At the moment, the folks who advocate not having the -R flag represent our
> state of affairs.
OK, understood, and sorry to recapitulate an old discussion.
(Also apologies for the strange subject line -- too much proofreading
of the body!)
> Setting LDFLAGS before running configure should work, like so:
>
> LDFLAGS="-R/path/to/lib" \
> ./configure
>
> is an acceptable workaround that should work. It's what I use.
I do the same to get Berkeley DB to work with apr-util, with
the -Wl,-R option. That seems to be sufficient to ensure that
configure's test programs compile and execute. (For some reason,
just having -L doesn't do it, and configure wrongly decides that
BDB can't be found because one of the test programs fails.)
However, if I read the situation correctly, that LDFLAGS setting
doesn't actually make it into make process itself; certainly it
doesn't turn up outside of config.{log,nice,status} or in make's
output. And in both the "fake" libiconv and "real" Oracle instances
I've tried, adding either -R or -Wl,-R to LDFLAGS or NOTEST_LDFLAGS,
etc., before running configure has no effect; libaprutil-1.so
is still built with an unresolved "not found" dependency, because
of the missing .la file and the fact that no -R option makes it
from my env vars into the make process.
Now, maybe that's some other kind of problem -- should one be
able to do "NOTEST_LDFLAGS=... ./configure" and have those
values appear in, perhaps, build/rules.mk? I'm not sure if
that's the intention or not from the comments in rules.mk.
At any rate, I've still got a problem on my hands, because
to deal with Oracle I need to get something into $LINK after
$COMPILE, which means one of LDFLAGS, EXTRA_LDFLAGS, or
NOTEST_LDFLAGS in build/rules.mk. No amount of fiddling
with prior-to-configure env vars has allowed me to do this;
although things turn up in config.nice, they don't make it
into the actual make process. Any help appreciated! Thanks!
Chris.
--
GPG Key ID: 366A375B
GPG Key Fingerprint: 485E 5041 17E1 E2BB C263 E4DE C8E3 FA36 366A 375B
Re: [Fwd: Re: apr-util config, etc.]
Posted by Justin Erenkrantz <ju...@erenkrantz.com>.
--On November 15, 2005 3:00:38 PM -0500 Chris Darroch
<ch...@pearsoncmg.com> wrote:
> My ultimate question, for the impatient, is whether adding
> -R to APRUTIL_LDFLAGS is an appropriate solution when using
> libraries that one expects not to have accompanying .la files.
We've had this discussion several times. There's a group of developers who
believe we should automatically add the -R flags and an equally vocal
developers who believe that the user must hand-edit ldconfig on their
platform.
At the moment, the folks who advocate not having the -R flag represent our
state of affairs.
> Now I didn't have a working solution yet, because this was
> just a post-buildconf, post-configure hack. But, OTOH, the
> libaprutil-1.so was OK, with no undefined dependencies listed
> by ldd.
Setting LDFLAGS before running configure should work, like so:
LDFLAGS="-R/path/to/lib" \
./configure
is an acceptable workaround that should work. It's what I use.
HTH. -- justin