You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@apr.apache.org by "William A. Rowe, Jr." <wr...@rowe-clan.net> on 2002/04/13 20:20:25 UTC
Unix missing fd 0..2, Win32 service missing stdin/out/err
handles
A suggestion to implement in apr_app_initialize, but first some background.
Without harming apr_initialize(), which should continue to be useful
for initializing apr in a non-apr application (e.g. a mod_jk connector built
with apr, that is written for Apache 1.3), I had started an apr_app_initialize
a while back.
One paragraph of background; I created apr_app_initialize primarily to allow
us to use the apr signals API... if we detect the app is running in a console,
we can set up the Windows console API in the signal schema. If we detect
we are running 'as a service', we can set up the Windows SCM context
immediately, and when apr_daemonize is invoked, we can report 'started'.
We also use apr_app_initialize on Win32 to transform our Unicode input
(command line args and the environment table) into utf-8 internal encoding.
So far, so simple [yet I haven't had three days to put the signals together.]
Back to the purpose of this post;
Because third party libraries have a nasty habit of dropping messages out
to stderr or stdout... and will sometimes even poll stdin (think passphrases
or other bits in encryption libraries, etc) ... it is rather dangerous for
-our-
applications to ever use fd 0..2. Sure, you can chalk it up to a bug in the
caller, but imagine if 'by chance' we open up an sdbm as fd 2. Another
library prints something to stderr and bang ... database is corrupted.
On the Win32 side, same goes for FILE *'s stdin, stderr and stdout, for
the low level 'fd's 0..2 (not really fd's as unix knows them, but the clib's
table of Win32 handles), and the Win32 standard handles. Win32 services
have -no- STD handles, even when they are command line apps.
So, on the unix side, within apr_app_initialize, I suggest calling
fopen("/dev/null", ) until it returns an fd >2. On the Win32 side, I suggest
calling GetStandardHandle() and filling in any missing stdhandles with
the appropriate FILE*'s fd's handle after opening 'NUL' in the clib, so all
three bases are covered. If we end up with an fd >2, then we immediately
close that last /dev/null file and go on.
Does this make good sense?
Bill
Re: Unix missing fd 0..2, Win32 service missing stdin/out/err
handles
Posted by "William A. Rowe, Jr." <wr...@rowe-clan.net>.
At 08:18 PM 4/13/2002, Jon Travis wrote:
>On Sat, Apr 13, 2002 at 01:20:25PM -0500, William A. Rowe, Jr. wrote:
> > Because third party libraries have a nasty habit of dropping messages out
> > to stderr or stdout... and will sometimes even poll stdin (think
> passphrases
> > or other bits in encryption libraries, etc) ... it is rather dangerous for
> > -our- applications to ever use fd 0..2. Sure, you can chalk it up to a
> bug in
> > the caller, but imagine if 'by chance' we open up an sdbm as fd 2. Another
> > library prints something to stderr and bang ... database is corrupted.
> > So, on the unix side, within apr_app_initialize, I suggest calling
> > fopen("/dev/null", ) until it returns an fd >2. On the Win32 side, I
> suggest
> > calling GetStandardHandle() and filling in any missing stdhandles with
> > the appropriate FILE*'s fd's handle after opening 'NUL' in the clib, so all
> > three bases are covered. If we end up with an fd >2, then we immediately
> > close that last /dev/null file and go on.
> >
> > Does this make good sense?
>
>Not particularly.
>
>The operating system pre-allocates those fd's (0..2 for Unix) -- why would
>opening an SDBM ever return any of those file descriptors? The only way
>would be if the consumer closed those handles beforehand. If the user
>does something like that, their program is broken -- we shouldn't try
>to work around that.
That's the sort of feedback I was looking for. Thanks. Yes - if it succeeds
in opening fd 0..2 then the caller fooed up. And if we don't care to protect
against that case, I can go along with that. However...
> > On the Win32 side, same goes for FILE *'s stdin, stderr and stdout, for
> > the low level 'fd's 0..2 (not really fd's as unix knows them, but the
> clib's
> > table of Win32 handles), and the Win32 standard handles. Win32 services
> > have -no- STD handles, even when they are command line apps.
This is the operating system's responsibility - and it won't happen on Win32
services. So in the Win32 case, I see this as required.
Bill
Re: Unix missing fd 0..2, Win32 service missing stdin/out/err handles
Posted by Jon Travis <jt...@covalent.net>.
On Sun, Apr 14, 2002 at 10:45:40PM +0200, Sascha Schumann wrote:
> > If a user knows that a third-party library (Oracle, whatever) can
> > provide messages to stderr, and they knowingly close that file
> > descriptor without dup()ing it elsewhere, then that is a bug in
> > their software.
>
> Note that the author of the non-apr application does not
> necessarily must submit to that wisdom. While you are
> theoritecally correct, it matters little practically.
The P in APR stands for portable, not 'obscures bugs in your code.'
This seems to come up every now and then -- APR (poorly) trying to protect
or obscure bugs from programmers -- remember the insanity that occurred
when all that socket inheritance stuff went in?
> > Having APR open a bunch of file-descriptors so
> > that this fixes a potential problem (IMO bug in their software)
> > is just wasting file descriptors.
>
> Not really. (error handling excluded)
>
> do {
> fd = open("/dev/null", O_RDWR);
> } while (fd < 3);
> close(fd);
>
> No fd wasted.
I as a daemon, startup, close all my file descriptors, then do an
apr_file_open("/etc/hosts") or some such -- that gives me file
descriptor 3. That means that I now have 3 fewer file descriptors
to use in my super-duper-file-descriptor-usage application. Sounds
like wasted descriptors to me.
-- Jon
RE: Unix missing fd 0..2, Win32 service missing stdin/out/err handles
Posted by Ryan Bloom <rb...@covalent.net>.
I tend to agree with Jon. On Windows, this is required behavior, and we
should enforce it. On Unix, the OS is going to open fd's 0-2
automatically. In order for them to no longer be opened when the
program is run, the programmer must have done something to close them.
On Windows, the function should exist and it should open the fds. On
Unix, the function should exist, but it should be a no-op.
Ryan
----------------------------------------------
Ryan Bloom rbb@covalent.net
645 Howard St. rbb@apache.org
San Francisco, CA
> -----Original Message-----
> From: William A. Rowe, Jr. [mailto:wrowe@rowe-clan.net]
> Sent: Sunday, April 14, 2002 2:44 PM
> To: Sander Striker
> Cc: Sascha Schumann; Jon Travis; dev@apr.apache.org
> Subject: RE: Unix missing fd 0..2, Win32 service missing stdin/out/err
> handles
>
> At 04:39 PM 4/14/2002, Sander Striker wrote:
> > > From: Sascha Schumann [mailto:sascha@schumann.cx]
> > > Sent: 14 April 2002 22:46
> >
> > > do {
> > > fd = open("/dev/null", O_RDWR);
> > > } while (fd < 3);
> > > close(fd);
> >
> >And this is what most daemons (mostly) do. We could create a
> >convenience function for this. I don't think it should be default
> >apr behaviour though. APR is a library. Libraries shouldn't
> >take options away from users.
>
> Notice -where- I suggested we put this code.
>
> apr_initialize() performs library initialization. It doesn't care
what
> program
> is using the APR features.
>
> apr_app_initialize is -required- on win32 to compensate for Unicode,
> windows services, and the console 'signals' API.
>
> An application -entirely- APR-centric Will use apr_app_initialize to
> compensate for whatever platform it was built-for/running-on. If this
is
> platform compensation, Sascha and I are only suggesting that a typical
> app (daemon or not) will -prefer- this behavior at -invocation-.
>
> If that app then closes fd 0..2, we don't care. I'm absolutely -not-
> suggesting that we would ever want to test that apr_file_open()
doesn't
> return an fd 0..2. Only for startup of an app that we don't know
-who-
> has invoked us, or what a broken daemon-manager has overlooked.
>
> Bill
RE: Unix missing fd 0..2, Win32 service missing stdin/out/err
handles
Posted by "William A. Rowe, Jr." <wr...@rowe-clan.net>.
At 04:39 PM 4/14/2002, Sander Striker wrote:
> > From: Sascha Schumann [mailto:sascha@schumann.cx]
> > Sent: 14 April 2002 22:46
>
> > do {
> > fd = open("/dev/null", O_RDWR);
> > } while (fd < 3);
> > close(fd);
>
>And this is what most daemons (mostly) do. We could create a
>convenience function for this. I don't think it should be default
>apr behaviour though. APR is a library. Libraries shouldn't
>take options away from users.
Notice -where- I suggested we put this code.
apr_initialize() performs library initialization. It doesn't care what program
is using the APR features.
apr_app_initialize is -required- on win32 to compensate for Unicode,
windows services, and the console 'signals' API.
An application -entirely- APR-centric Will use apr_app_initialize to
compensate for whatever platform it was built-for/running-on. If this is
platform compensation, Sascha and I are only suggesting that a typical
app (daemon or not) will -prefer- this behavior at -invocation-.
If that app then closes fd 0..2, we don't care. I'm absolutely -not-
suggesting that we would ever want to test that apr_file_open() doesn't
return an fd 0..2. Only for startup of an app that we don't know -who-
has invoked us, or what a broken daemon-manager has overlooked.
Bill
RE: Unix missing fd 0..2, Win32 service missing stdin/out/err handles
Posted by Sander Striker <st...@apache.org>.
> From: Sascha Schumann [mailto:sascha@schumann.cx]
> Sent: 14 April 2002 22:46
> > If a user knows that a third-party library (Oracle, whatever) can
> > provide messages to stderr, and they knowingly close that file
> > descriptor without dup()ing it elsewhere, then that is a bug in
> > their software.
>
> Note that the author of the non-apr application does not
> necessarily must submit to that wisdom. While you are
> theoritecally correct, it matters little practically.
>
> > Having APR open a bunch of file-descriptors so
> > that this fixes a potential problem (IMO bug in their software)
> > is just wasting file descriptors.
>
> Not really. (error handling excluded)
>
> do {
> fd = open("/dev/null", O_RDWR);
> } while (fd < 3);
> close(fd);
And this is what most daemons (mostly) do. We could create a
convenience function for this. I don't think it should be default
apr behaviour though. APR is a library. Libraries shouldn't
take options away from users.
Sander
Re: Unix missing fd 0..2, Win32 service missing stdin/out/err handles
Posted by Sascha Schumann <sa...@schumann.cx>.
> If a user knows that a third-party library (Oracle, whatever) can
> provide messages to stderr, and they knowingly close that file
> descriptor without dup()ing it elsewhere, then that is a bug in
> their software.
Note that the author of the non-apr application does not
necessarily must submit to that wisdom. While you are
theoritecally correct, it matters little practically.
> Having APR open a bunch of file-descriptors so
> that this fixes a potential problem (IMO bug in their software)
> is just wasting file descriptors.
Not really. (error handling excluded)
do {
fd = open("/dev/null", O_RDWR);
} while (fd < 3);
close(fd);
No fd wasted.
- Sascha Experience IRCG
http://schumann.cx/ http://schumann.cx/ircg
Re: Unix missing fd 0..2, Win32 service missing stdin/out/err handles
Posted by Jon Travis <jt...@covalent.net>.
On Sun, Apr 14, 2002 at 01:38:25PM +0200, Sascha Schumann wrote:
> > As I described, it would be possible for a user to close those
> > descriptors before doing something like say, opening an SDBM. If
> > they then try to _write_ to those (via something like write() to
> > stderr), then that is a programming error which we should not be
> > trying to solve.
>
> Not necessarily. The daemon might use a third-party library
> which writes diagnostic messages to stderr (something like
> Oracle's libs). Or even worse, the user might be running the
> application on FreeBSD where malloc(3) writes messages to
> stderr, if configured to do so. There are probably more
> cases where this can happen which the app's author cannot
> influence. Supplying module authors with a function which
> addresses this issue makes sense from that perspective.
If a user knows that a third-party library (Oracle, whatever) can
provide messages to stderr, and they knowingly close that file
descriptor without dup()ing it elsewhere, then that is a bug in
their software. Having APR open a bunch of file-descriptors so
that this fixes a potential problem (IMO bug in their software)
is just wasting file descriptors.
-- Jon
Re: Unix missing fd 0..2, Win32 service missing stdin/out/err handles
Posted by Sascha Schumann <sa...@schumann.cx>.
> As I described, it would be possible for a user to close those
> descriptors before doing something like say, opening an SDBM. If
> they then try to _write_ to those (via something like write() to
> stderr), then that is a programming error which we should not be
> trying to solve.
Not necessarily. The daemon might use a third-party library
which writes diagnostic messages to stderr (something like
Oracle's libs). Or even worse, the user might be running the
application on FreeBSD where malloc(3) writes messages to
stderr, if configured to do so. There are probably more
cases where this can happen which the app's author cannot
influence. Supplying module authors with a function which
addresses this issue makes sense from that perspective.
- Sascha Experience IRCG
http://schumann.cx/ http://schumann.cx/ircg
Re: Unix missing fd 0..2, Win32 service missing stdin/out/err handles
Posted by Jon Travis <jt...@covalent.net>.
On Sun, Apr 14, 2002 at 10:18:57AM +0200, Sascha Schumann wrote:
> > The operating system pre-allocates those fd's (0..2 for Unix) -- why would
> > opening an SDBM ever return any of those file descriptors? The only way
> > would be if the consumer closed those handles beforehand. If the user
> > does something like that, their program is broken -- we shouldn't try
> > to work around that.
>
> There are various daemons which close all fds after forking
> and detaching from the tty, because they have no need to keep
> those fds open. open(), socket() or accept() can then return
> an fd in the range 0..2. If APR would be used in such a
> context, the scenario Bill laid out would be quite possible.
As I described, it would be possible for a user to close those
descriptors before doing something like say, opening an SDBM. If
they then try to _write_ to those (via something like write() to
stderr), then that is a programming error which we should not be
trying to solve.
-- Jon
Re: Unix missing fd 0..2, Win32 service missing stdin/out/err handles
Posted by Sascha Schumann <sa...@schumann.cx>.
> The operating system pre-allocates those fd's (0..2 for Unix) -- why would
> opening an SDBM ever return any of those file descriptors? The only way
> would be if the consumer closed those handles beforehand. If the user
> does something like that, their program is broken -- we shouldn't try
> to work around that.
There are various daemons which close all fds after forking
and detaching from the tty, because they have no need to keep
those fds open. open(), socket() or accept() can then return
an fd in the range 0..2. If APR would be used in such a
context, the scenario Bill laid out would be quite possible.
- Sascha Experience IRCG
http://schumann.cx/ http://schumann.cx/ircg
Re: Unix missing fd 0..2, Win32 service missing stdin/out/err handles
Posted by Jon Travis <jt...@covalent.net>.
On Sat, Apr 13, 2002 at 01:20:25PM -0500, William A. Rowe, Jr. wrote:
> A suggestion to implement in apr_app_initialize, but first some background.
>
> Without harming apr_initialize(), which should continue to be useful
> for initializing apr in a non-apr application (e.g. a mod_jk connector built
> with apr, that is written for Apache 1.3), I had started an apr_app_initialize
> a while back.
>
> One paragraph of background; I created apr_app_initialize primarily to allow
> us to use the apr signals API... if we detect the app is running in a console,
> we can set up the Windows console API in the signal schema. If we detect
> we are running 'as a service', we can set up the Windows SCM context
> immediately, and when apr_daemonize is invoked, we can report 'started'.
> We also use apr_app_initialize on Win32 to transform our Unicode input
> (command line args and the environment table) into utf-8 internal encoding.
> So far, so simple [yet I haven't had three days to put the signals together.]
>
> Back to the purpose of this post;
>
> Because third party libraries have a nasty habit of dropping messages out
> to stderr or stdout... and will sometimes even poll stdin (think passphrases
> or other bits in encryption libraries, etc) ... it is rather dangerous for
> -our-
> applications to ever use fd 0..2. Sure, you can chalk it up to a bug in the
> caller, but imagine if 'by chance' we open up an sdbm as fd 2. Another
> library prints something to stderr and bang ... database is corrupted.
>
> On the Win32 side, same goes for FILE *'s stdin, stderr and stdout, for
> the low level 'fd's 0..2 (not really fd's as unix knows them, but the clib's
> table of Win32 handles), and the Win32 standard handles. Win32 services
> have -no- STD handles, even when they are command line apps.
>
> So, on the unix side, within apr_app_initialize, I suggest calling
> fopen("/dev/null", ) until it returns an fd >2. On the Win32 side, I suggest
> calling GetStandardHandle() and filling in any missing stdhandles with
> the appropriate FILE*'s fd's handle after opening 'NUL' in the clib, so all
> three bases are covered. If we end up with an fd >2, then we immediately
> close that last /dev/null file and go on.
>
> Does this make good sense?
Not particularly.
The operating system pre-allocates those fd's (0..2 for Unix) -- why would
opening an SDBM ever return any of those file descriptors? The only way
would be if the consumer closed those handles beforehand. If the user
does something like that, their program is broken -- we shouldn't try
to work around that.
-- Jon