You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by "William A. Rowe Jr." <wr...@rowe-clan.net> on 2010/09/23 22:02:03 UTC
Re: svn commit: r1000593 - in /httpd/httpd/trunk: CHANGES server/util_script.c
On 9/23/2010 2:50 PM, wrowe@apache.org wrote:
> Author: wrowe
> Date: Thu Sep 23 19:50:14 2010
> New Revision: 1000593
>
> URL: http://svn.apache.org/viewvc?rev=1000593&view=rev
> Log:
> Because PATH and the library path are closely interrelated, and the cause
> of most confusion over cgi or fcgid failures, or even starting rotatelogs,
> etc, when the server binaries have been relocated, pass the library path
> as paired with the system PATH.
>
> Of course, PATH and platform-specific library path(s) may be modified as
> needed with mod_env, so there is no loss of functionality with this change.
>
> The days of monolithic binaries are long gone, even on 1970's architectures,
> and PATH should not be decoupled from the library path.
Feedback welcome, I'd appreciate some affirmative +/- opinions on this change
before the .9 tag.
Re: svn commit: r1000593 - in /httpd/httpd/trunk: CHANGES server/util_script.c
Posted by Jeff Trawick <tr...@gmail.com>.
On Thu, Sep 23, 2010 at 9:56 PM, William A. Rowe Jr. <wr...@rowe-clan.net>wrote:
> On 9/23/2010 6:58 PM, Jeff Trawick wrote:
> >
> > These two are somewhat different in practice.
> >
> > When the path to the binary is omitted on the invocation/load, the
> shell/loader/whatever finds
> >
> > * executables only because of the PATH envvar
>
> Correct
>
> > * shared libraries usually via the system search path or in the
> executable/other-library's
> > rpath
>
> Typically, yes
>
> > PATH always, LD_LIBRARY_PATH in exceptional situations
>
> And always permitted at the shell prior to execution. But once httpd has
> started,
> dropping LD_LIBRARY_PATH deprives the kernel of resolving such libraries,
> due to
> our arbitrary choice to propagate PATH, but not propagate
> LD_LIBRARY_PATH... leading
> potentially to broken process invocations. Does this make sense?
>
> E.g. the choice to propagate PATH, but not LD_LIBRARY_PATH, seems foolish.
> For those
> who wish to argue the 'unsafety' of relocatable/dynamic path resolutions,
> is there
> really any difference between propagating PATH but not LD_LIBRARY_PATH?
>
A couple of slight exaggerations:
. almost nothing works without propagating PATH
. almost everything works without propagating LD_LIBRARY_PATH
That's the distinction. They are used differently. IOW, we propagate PATH
because of the gnashing of teeth which would occur otherwise. I'm aware of
no such gnashing of teeth due to the lack of LD_LIBRARY_PATH propagation.
For those things that need LD_LIBRARY_PATH, it isn't clear to me that the
LD_LIBRARY_PATH in the shell starting httpd or in httpd's bin/envvars is the
one needed by the CGI/FastCGI. And when somebody needs to code the
LD_LIBRARY_PATH somewhere for their CGI/FastCGI, I'd suggest that httpd.conf
is a better place than bin/envvars or somewhere in the shell used to start
httpd.
Perhaps an assumption of mine that setting LD_LIBRARY_PATH or .profile is a
Bad Thing is coming into play here. I guess if lots of tools on the system
are unusable without setting LD_LIBRARY_PATH (busted tool build or install),
then LD_LIBRARY_PATH is about as important as PATH. That's a very bad
example IMO.
I guess I'm still at my original comment ;) I'd like to understand more
about the CGI/FastCGI use case -- what is a real world example and why is
the shell which starts httpd or bin/envvars the natural place to set
LD_LIBRARY_PATH appropriately. I'm not trying to scream that propagating
LD_LIBRARY_PATH is terrible; I am just hunting for the problem that this is
the best solution for. (I will save my screaming for the notion that we
propagate both or neither, if it is put into code :) )
Re: svn commit: r1000593 - in /httpd/httpd/trunk: CHANGES server/util_script.c
Posted by Jeff Trawick <tr...@gmail.com>.
On Fri, Sep 24, 2010 at 6:30 PM, William A. Rowe Jr. <wr...@rowe-clan.net>wrote:
> On 9/24/2010 12:09 PM, Jeff Trawick wrote:
> > On Fri, Sep 24, 2010 at 12:50 PM, William A. Rowe Jr. <
> wrowe@rowe-clan.net
> > <ma...@rowe-clan.net>> wrote:
> >
> > On 9/24/2010 11:12 AM, Rainer Jung wrote:
> > >
> > > I guess I'm also missing the bigger picture, but:
> > >
> > > - at first sight it sounds very reasonable to provide the
> LD_LIBRARY_PATH as well
> > >
> > > - it might result in problems, if the binary to start uses system
> libs, which exist in
> > > LD_LIBRARY_PATH and in system default locations in incompatible
> versions. IMHO this is
> > > more a theoretical problem. E.g. the apr libs are versioned via the
> soname such that the
> > > runtime linker won't link against a wrong version.
> >
> > But these same problems would be true at the shell, prior to invoking
> httpd, no?
> >
> >
> > httpd could pick up library paths from the shell, but it could also pick
> up paths from
> > bin/envvars; the latter is specifically tailored for dependencies
> bundled+built with httpd
> > (and same ABI/debug-ability/etc.); for the slim cases where those defs
> are not irrelevant
> > for the normal CGI/FastCGI, I dunno whether it would be good or bad more
> often
>
> AIUI - apachectl triggers bin/envvars, which injects the appropriate
> LD_LIBRARY_PATH,
> yet this would still not be passed to child processes without my suggested
> commit, right?
>
bin/envvars puts it in the shell used to start httpd, so it works just like
PassEnv
>
> > This could be discussed purely in terms of features we already have:
> >
> > Do admins find 'SetEnv LD_LIBRARY_PATH foo' or 'PassEnv LD_LIBRARY_PATH'
> to solve their
> > problem most often?
> > How often is it problematic to use 'PassEnv LD_LIBRARY_PATH'?
>
> Frequently enough to be a FAQ/documented? (And sometimes not well.)
>
> http://httpd.apache.org/docs/2.2/mod/mod_env.html#passenv
>
I saw that too, but disrespected it as a real world example. (There's also
the example "UnsetEnv LD_LIBRARY_PATH", which may become more useful now ;)
)
> The original mod_fcgid config docs (cannot find them anymore via google,
> even)
>
> http://httpd.apache.org/docs/2.2/mod/mod_cgi.html (whoops!)
> http://httpd.apache.org/mod_fcgid/mod/mod_fcgid.html (whoops!)
>
I guess "whoops" means we don't document it
> http://bugs.php.net/bug.php?id=49173&edit=1
>
looks like poor advice to me
>
> or as encountered 300k+ times in google;
> http://s.apache.org/cOb
>
>
This one yields 181,000 results: >httpd ld_library_path setenv<
This one yields 3,850: >httpd ld_library_path passenv<
So many of these are unhelpful though. E.g., some of those references to
PassEnv say to first modify the script used to start Apache to include
LD_LIBRARY_PATH then code PassEnv in httpd.conf, which seems clumsy.
httpd.conf is the place to consolidate both.
--/--
I guess I'm at the YMMV stage; I think the PassEnv semantics for
LD_LIBRARY_PATH still means that the user ends up configuring the paths in
the shell script used to start httpd or bin/envvars, which seems worse than
just putting SetEnv in httpd.conf. And automatically passing through the
value required for httpd to run could have occasional negative
consequences. (But it is unclear to me that it either helps or hurts with
any interesting frequency.)
At any rate, I suggest that you edit the CHANGES to remove the reference to
piped loggers and fix the suggestion that PassEnv is still applicable. One
possibility:
core: For script invocation, pass the system library path (LD_LIBRARY_PATH
or other
platform-specific variable) by default; PassEnv is no longer necessary for
that. As before,
use SetEnv to configure a specific value different from that required by
httpd.
Re: svn commit: r1000593 - in /httpd/httpd/trunk: CHANGES server/util_script.c
Posted by "William A. Rowe Jr." <wr...@rowe-clan.net>.
On 9/24/2010 12:09 PM, Jeff Trawick wrote:
> On Fri, Sep 24, 2010 at 12:50 PM, William A. Rowe Jr. <wrowe@rowe-clan.net
> <ma...@rowe-clan.net>> wrote:
>
> On 9/24/2010 11:12 AM, Rainer Jung wrote:
> >
> > I guess I'm also missing the bigger picture, but:
> >
> > - at first sight it sounds very reasonable to provide the LD_LIBRARY_PATH as well
> >
> > - it might result in problems, if the binary to start uses system libs, which exist in
> > LD_LIBRARY_PATH and in system default locations in incompatible versions. IMHO this is
> > more a theoretical problem. E.g. the apr libs are versioned via the soname such that the
> > runtime linker won't link against a wrong version.
>
> But these same problems would be true at the shell, prior to invoking httpd, no?
>
>
> httpd could pick up library paths from the shell, but it could also pick up paths from
> bin/envvars; the latter is specifically tailored for dependencies bundled+built with httpd
> (and same ABI/debug-ability/etc.); for the slim cases where those defs are not irrelevant
> for the normal CGI/FastCGI, I dunno whether it would be good or bad more often
AIUI - apachectl triggers bin/envvars, which injects the appropriate LD_LIBRARY_PATH,
yet this would still not be passed to child processes without my suggested commit, right?
> This could be discussed purely in terms of features we already have:
>
> Do admins find 'SetEnv LD_LIBRARY_PATH foo' or 'PassEnv LD_LIBRARY_PATH' to solve their
> problem most often?
> How often is it problematic to use 'PassEnv LD_LIBRARY_PATH'?
Frequently enough to be a FAQ/documented? (And sometimes not well.)
http://httpd.apache.org/docs/2.2/mod/mod_env.html#passenv
The original mod_fcgid config docs (cannot find them anymore via google, even)
http://httpd.apache.org/docs/2.2/mod/mod_cgi.html (whoops!)
http://httpd.apache.org/mod_fcgid/mod/mod_fcgid.html (whoops!)
http://bugs.php.net/bug.php?id=49173&edit=1
or as encountered 300k+ times in google;
http://s.apache.org/cOb
Re: svn commit: r1000593 - in /httpd/httpd/trunk: CHANGES server/util_script.c
Posted by Jeff Trawick <tr...@gmail.com>.
On Fri, Sep 24, 2010 at 12:50 PM, William A. Rowe Jr.
<wr...@rowe-clan.net>wrote:
> On 9/24/2010 11:12 AM, Rainer Jung wrote:
> >
> > I guess I'm also missing the bigger picture, but:
> >
> > - at first sight it sounds very reasonable to provide the LD_LIBRARY_PATH
> as well
> >
> > - it might result in problems, if the binary to start uses system libs,
> which exist in
> > LD_LIBRARY_PATH and in system default locations in incompatible versions.
> IMHO this is
> > more a theoretical problem. E.g. the apr libs are versioned via the
> soname such that the
> > runtime linker won't link against a wrong version.
>
> But these same problems would be true at the shell, prior to invoking
> httpd, no?
>
httpd could pick up library paths from the shell, but it could also pick up
paths from bin/envvars; the latter is specifically tailored for dependencies
bundled+built with httpd (and same ABI/debug-ability/etc.); for the slim
cases where those defs are not irrelevant for the normal CGI/FastCGI, I
dunno whether it would be good or bad more often
This could be discussed purely in terms of features we already have:
Do admins find 'SetEnv LD_LIBRARY_PATH foo' or 'PassEnv LD_LIBRARY_PATH' to
solve their problem most often?
How often is it problematic to use 'PassEnv LD_LIBRARY_PATH'?
Re: svn commit: r1000593 - in /httpd/httpd/trunk: CHANGES server/util_script.c
Posted by "William A. Rowe Jr." <wr...@rowe-clan.net>.
On 9/24/2010 11:12 AM, Rainer Jung wrote:
>
> I guess I'm also missing the bigger picture, but:
>
> - at first sight it sounds very reasonable to provide the LD_LIBRARY_PATH as well
>
> - it might result in problems, if the binary to start uses system libs, which exist in
> LD_LIBRARY_PATH and in system default locations in incompatible versions. IMHO this is
> more a theoretical problem. E.g. the apr libs are versioned via the soname such that the
> runtime linker won't link against a wrong version.
But these same problems would be true at the shell, prior to invoking httpd, no?
Re: svn commit: r1000593 - in /httpd/httpd/trunk: CHANGES server/util_script.c
Posted by Rainer Jung <ra...@kippdata.de>.
On 24.09.2010 03:56, William A. Rowe Jr. wrote:
> On 9/23/2010 6:58 PM, Jeff Trawick wrote:
>>
>> These two are somewhat different in practice.
>>
>> When the path to the binary is omitted on the invocation/load, the shell/loader/whatever finds
>>
>> * executables only because of the PATH envvar
>
> Correct
>
>> * shared libraries usually via the system search path or in the executable/other-library's
>> rpath
>
> Typically, yes
>
>> PATH always, LD_LIBRARY_PATH in exceptional situations
>
> And always permitted at the shell prior to execution. But once httpd has started,
> dropping LD_LIBRARY_PATH deprives the kernel of resolving such libraries, due to
> our arbitrary choice to propagate PATH, but not propagate LD_LIBRARY_PATH... leading
> potentially to broken process invocations. Does this make sense?
>
> E.g. the choice to propagate PATH, but not LD_LIBRARY_PATH, seems foolish. For those
> who wish to argue the 'unsafety' of relocatable/dynamic path resolutions, is there
> really any difference between propagating PATH but not LD_LIBRARY_PATH?
I guess I'm also missing the bigger picture, but:
- at first sight it sounds very reasonable to provide the
LD_LIBRARY_PATH as well
- it might result in problems, if the binary to start uses system libs,
which exist in LD_LIBRARY_PATH and in system default locations in
incompatible versions. IMHO this is more a theoretical problem. E.g. the
apr libs are versioned via the soname such that the runtime linker won't
link against a wrong version.
Regards,
Rainer
Re: svn commit: r1000593 - in /httpd/httpd/trunk: CHANGES server/util_script.c
Posted by "William A. Rowe Jr." <wr...@rowe-clan.net>.
On 9/23/2010 6:58 PM, Jeff Trawick wrote:
>
> These two are somewhat different in practice.
>
> When the path to the binary is omitted on the invocation/load, the shell/loader/whatever finds
>
> * executables only because of the PATH envvar
Correct
> * shared libraries usually via the system search path or in the executable/other-library's
> rpath
Typically, yes
> PATH always, LD_LIBRARY_PATH in exceptional situations
And always permitted at the shell prior to execution. But once httpd has started,
dropping LD_LIBRARY_PATH deprives the kernel of resolving such libraries, due to
our arbitrary choice to propagate PATH, but not propagate LD_LIBRARY_PATH... leading
potentially to broken process invocations. Does this make sense?
E.g. the choice to propagate PATH, but not LD_LIBRARY_PATH, seems foolish. For those
who wish to argue the 'unsafety' of relocatable/dynamic path resolutions, is there
really any difference between propagating PATH but not LD_LIBRARY_PATH?
Re: svn commit: r1000593 - in /httpd/httpd/trunk: CHANGES server/util_script.c
Posted by Jeff Trawick <tr...@gmail.com>.
On Thu, Sep 23, 2010 at 6:39 PM, William A. Rowe Jr. <wr...@rowe-clan.net>wrote:
> On 9/23/2010 3:59 PM, Jeff Trawick wrote:
> > On Thu, Sep 23, 2010 at 4:02 PM, William A. Rowe Jr. <
> wrowe@rowe-clan.net
> > <ma...@rowe-clan.net>> wrote:
> >
> > On 9/23/2010 2:50 PM, wrowe@apache.org <ma...@apache.org>
> wrote:
> > > Author: wrowe
> > > Date: Thu Sep 23 19:50:14 2010
> > > New Revision: 1000593
> > >
> > > URL: http://svn.apache.org/viewvc?rev=1000593&view=rev
> > <http://svn.apache.org/viewvc?rev=1000593&view=rev>
> > > Log:
> > > Because PATH and the library path are closely interrelated, and the
> cause
> > > of most confusion over cgi or fcgid failures, or even starting
> rotatelogs,
> > > etc, when the server binaries have been relocated, pass the library
> path
> > > as paired with the system PATH.
> >
> >
> > this doesn't change anything for rotatelogs; it isn't affected by
> subprocess_env, and
> > should just need the same settings that allowed httpd to start
> successfully
>
> Ahhh, yes. There was a reason I never committed this in the first place.
> I let Ruediger's comments propel me to finishing it, but with the caveat
> that I had forgotten the missing bit.
>
> I definitely encountered piped log failures due to the environment table.
> The non-static relocated rotatelogs binary would not start. But I need
> to investigate the why's again. My thought, where I abandoned this patch,
> is that this logic might be generic to invoking all ap_ processes, not just
> cgi/request level processes.
>
> So perhaps the next step is factoring this into per-request v.s. generic
> process invocation. I need to review my notes on this next week and offer
> the second half of the solution.
>
> > we don't ship CGI or FastCGI binaries, so relocating httpd and adjusting
> our
> > LD_LIBRARY_PATH in bin/envvars won't be the adjustment needed for a CGI
> or FastCGI, so
> > passing on our setting won't help, will it?
>
> That is exactly the point. The mod_fcgid configuration on windows used to
> be
> horrid, due to missing env vars. The double-whammy is how stderr is
> mistreated
> by mod_fcgid, and error information will not hit the log. Until you try to
> run in a console, and let win32 vomit popup exception windows, you won't
> have
> a clue why things don't work. Today, this is already improved, but the
> patch
> I committed attempts to be neutral and generic across platforms.
>
> > I guess the case it helps is where the CGI or FastCGI relies on a library
> in the same odd
> > place as httpd?
>
> No, it has more to do with PATH'ed applications relying on
> LD_LIBRARY_PATH'ed
> dependencies.
>
> > or people want to rely on an envvar in the the shell used to start httpd
> to get their
> > CGIs/FastCGIs working, which may result in failures when httpd is started
> slightly differently
> >
> > I guess I'm missing the big picture :)
>
> My essential argument: Pass LD_LIBRARY_PATH + PATH, or let's drop both.
> From an environmental/operational perspective they are two peas in a pod.
>
>
These two are somewhat different in practice.
When the path to the binary is omitted on the invocation/load, the
shell/loader/whatever finds
* executables only because of the PATH envvar
* shared libraries usually via the system search path or in the
executable/other-library's rpath
PATH always, LD_LIBRARY_PATH in exceptional situations
Re: svn commit: r1000593 - in /httpd/httpd/trunk: CHANGES server/util_script.c
Posted by "William A. Rowe Jr." <wr...@rowe-clan.net>.
On 9/23/2010 3:59 PM, Jeff Trawick wrote:
> On Thu, Sep 23, 2010 at 4:02 PM, William A. Rowe Jr. <wrowe@rowe-clan.net
> <ma...@rowe-clan.net>> wrote:
>
> On 9/23/2010 2:50 PM, wrowe@apache.org <ma...@apache.org> wrote:
> > Author: wrowe
> > Date: Thu Sep 23 19:50:14 2010
> > New Revision: 1000593
> >
> > URL: http://svn.apache.org/viewvc?rev=1000593&view=rev
> <http://svn.apache.org/viewvc?rev=1000593&view=rev>
> > Log:
> > Because PATH and the library path are closely interrelated, and the cause
> > of most confusion over cgi or fcgid failures, or even starting rotatelogs,
> > etc, when the server binaries have been relocated, pass the library path
> > as paired with the system PATH.
>
>
> this doesn't change anything for rotatelogs; it isn't affected by subprocess_env, and
> should just need the same settings that allowed httpd to start successfully
Ahhh, yes. There was a reason I never committed this in the first place.
I let Ruediger's comments propel me to finishing it, but with the caveat
that I had forgotten the missing bit.
I definitely encountered piped log failures due to the environment table.
The non-static relocated rotatelogs binary would not start. But I need
to investigate the why's again. My thought, where I abandoned this patch,
is that this logic might be generic to invoking all ap_ processes, not just
cgi/request level processes.
So perhaps the next step is factoring this into per-request v.s. generic
process invocation. I need to review my notes on this next week and offer
the second half of the solution.
> we don't ship CGI or FastCGI binaries, so relocating httpd and adjusting our
> LD_LIBRARY_PATH in bin/envvars won't be the adjustment needed for a CGI or FastCGI, so
> passing on our setting won't help, will it?
That is exactly the point. The mod_fcgid configuration on windows used to be
horrid, due to missing env vars. The double-whammy is how stderr is mistreated
by mod_fcgid, and error information will not hit the log. Until you try to
run in a console, and let win32 vomit popup exception windows, you won't have
a clue why things don't work. Today, this is already improved, but the patch
I committed attempts to be neutral and generic across platforms.
> I guess the case it helps is where the CGI or FastCGI relies on a library in the same odd
> place as httpd?
No, it has more to do with PATH'ed applications relying on LD_LIBRARY_PATH'ed
dependencies.
> or people want to rely on an envvar in the the shell used to start httpd to get their
> CGIs/FastCGIs working, which may result in failures when httpd is started slightly differently
>
> I guess I'm missing the big picture :)
My essential argument: Pass LD_LIBRARY_PATH + PATH, or let's drop both.
>From an environmental/operational perspective they are two peas in a pod.
Re: svn commit: r1000593 - in /httpd/httpd/trunk: CHANGES server/util_script.c
Posted by Jeff Trawick <tr...@gmail.com>.
On Thu, Sep 23, 2010 at 4:02 PM, William A. Rowe Jr. <wr...@rowe-clan.net>wrote:
> On 9/23/2010 2:50 PM, wrowe@apache.org wrote:
> > Author: wrowe
> > Date: Thu Sep 23 19:50:14 2010
> > New Revision: 1000593
> >
> > URL: http://svn.apache.org/viewvc?rev=1000593&view=rev
> > Log:
> > Because PATH and the library path are closely interrelated, and the cause
> > of most confusion over cgi or fcgid failures, or even starting
> rotatelogs,
> > etc, when the server binaries have been relocated, pass the library path
> > as paired with the system PATH.
>
this doesn't change anything for rotatelogs; it isn't affected by
subprocess_env, and should just need the same settings that allowed httpd to
start successfully
we don't ship CGI or FastCGI binaries, so relocating httpd and adjusting our
LD_LIBRARY_PATH in bin/envvars won't be the adjustment needed for a CGI or
FastCGI, so passing on our setting won't help, will it?
I guess the case it helps is where the CGI or FastCGI relies on a library in
the same odd place as httpd?
or people want to rely on an envvar in the the shell used to start httpd to
get their CGIs/FastCGIs working, which may result in failures when httpd is
started slightly differently
I guess I'm missing the big picture :)
>
> > Of course, PATH and platform-specific library path(s) may be modified as
> > needed with mod_env, so there is no loss of functionality with this
> change.
> >
> > The days of monolithic binaries are long gone, even on 1970's
> architectures,
> > and PATH should not be decoupled from the library path.
>
> Feedback welcome, I'd appreciate some affirmative +/- opinions on this
> change
> before the .9 tag.
>
>
--
Born in Roswell... married an alien...