You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by David Hill <dd...@zk3.dec.com> on 2002/03/28 19:27:26 UTC
connect to listener warning message
Hi all,
When I run 2.0.32 on Compaq Tru64 and give it a bit of a load (ok, I whack it good :-), I get warnings similar to the following looping into the error log file. I am getting one per second, even well after the load is removed.I do not get any of these until after the load has been applied for a 10 seconds or so.
[Thu Mar 28 13:11:43 2002] [warn] (49)Can't assign requested address: connect to listener
I am guessing that this is coming from dummy connection in server/mpm_common.c but don't understand enough about what else is going on to know where to look further.
The server does seem to continue to respond to requests.
I tried bumping up the thread count but that did not seem to make any difference.
I have not tried the nightly yet to see if that makes any difference.
Any suggestions ?
thanks,
Dave Hill
Re: connect to listener warning message
Posted by Thom May <th...@planetarytramp.net>.
* David Hill (ddhill@zk3.dec.com) wrote :
>
> ----- Original Message -----
> From: "Thom May" <th...@planetarytramp.net>
> To: <de...@httpd.apache.org>
> Sent: Monday, April 15, 2002 3:46 PM
> Subject: Re: connect to listener warning message
>
>
> > * David Hill (ddhill@zk3.dec.com) wrote :
> > >
> > >
> > >
> > >
> > > > "David Hill" <dd...@zk3.dec.com> writes:
> > > >
> > > > > Jeff,
> > > > >
> > > > > > of course :) (I also remember your other problem where I suggested
> > > > > > running buildconf to pick up local libtool... what happened with
> > > > > > that?)
> > > > >
> > > > > Just got around to trying that (long queue, what with a brand new
> Apache
> > > 2
> > > > > to play with :-). Yes, the latest libtool installed 1.4.2 and the
> > > tarball
> > > > > builds without a hitch. I saw a thread in dev@ talking about trying
> to
> > > > > resync with the public lib tool. Hopefully the libtool sync will
> make it
> > > > > into the next spin of the tarball.
> > >
> > > Found another oddity.... not sure where this one comes from, but doing
> the
> > > ./buildconf results in a srclib/apr/Makefile with top_builddir set to
> > > nothing, which causes an error during install. Part of what I don't
> > > understand is that without running ./buildconf, the build results in
> > > top_builddir matching top_srcdir which means the the failing line in the
> > > install phase is going to copy the header files that the previous line
> just
> > > copied........
> >
> > Sander and I were looking at this earlier today - it's yet another
> autoconf
> > 2.53 "feature". Sander posted a diff to IRC that appears to be a
> workaround
> > for the problem, but the real fix appears to be along the lines of what
> you
> > suggest - we should always use the builddir unless it is unset. It should
> > eliminate a lot of redundancy, and be a bit more fail proof too.
> > Thoughts?
>
> If you are suggesting adding a simple check to the Makefile to test if
> top_builddir is set before doing the copy, then I am fine with that. If you
> are suggesting dropping the top_srcdir copy and only doing hte top_builddir
> copy, then there may be an issue.... as that is currently where I am getting
> at least one copy of the include files installed into the output tree
>
> Dave
Just the check, yes.
-Thom
--
Thom May -> thom@planetarytramp.net
"We shall not make Britain's mistakes. Too wise to try to rule the world, we shall merely own it".
--Ludwell Denny
Re: connect to listener warning message
Posted by David Hill <dd...@zk3.dec.com>.
----- Original Message -----
From: "Thom May" <th...@planetarytramp.net>
To: <de...@httpd.apache.org>
Sent: Monday, April 15, 2002 3:46 PM
Subject: Re: connect to listener warning message
> * David Hill (ddhill@zk3.dec.com) wrote :
> >
> >
> >
> >
> > > "David Hill" <dd...@zk3.dec.com> writes:
> > >
> > > > Jeff,
> > > >
> > > > > of course :) (I also remember your other problem where I suggested
> > > > > running buildconf to pick up local libtool... what happened with
> > > > > that?)
> > > >
> > > > Just got around to trying that (long queue, what with a brand new
Apache
> > 2
> > > > to play with :-). Yes, the latest libtool installed 1.4.2 and the
> > tarball
> > > > builds without a hitch. I saw a thread in dev@ talking about trying
to
> > > > resync with the public lib tool. Hopefully the libtool sync will
make it
> > > > into the next spin of the tarball.
> >
> > Found another oddity.... not sure where this one comes from, but doing
the
> > ./buildconf results in a srclib/apr/Makefile with top_builddir set to
> > nothing, which causes an error during install. Part of what I don't
> > understand is that without running ./buildconf, the build results in
> > top_builddir matching top_srcdir which means the the failing line in the
> > install phase is going to copy the header files that the previous line
just
> > copied........
>
> Sander and I were looking at this earlier today - it's yet another
autoconf
> 2.53 "feature". Sander posted a diff to IRC that appears to be a
workaround
> for the problem, but the real fix appears to be along the lines of what
you
> suggest - we should always use the builddir unless it is unset. It should
> eliminate a lot of redundancy, and be a bit more fail proof too.
> Thoughts?
If you are suggesting adding a simple check to the Makefile to test if
top_builddir is set before doing the copy, then I am fine with that. If you
are suggesting dropping the top_srcdir copy and only doing hte top_builddir
copy, then there may be an issue.... as that is currently where I am getting
at least one copy of the include files installed into the output tree
Dave
Re: connect to listener warning message
Posted by Thom May <th...@planetarytramp.net>.
* David Hill (ddhill@zk3.dec.com) wrote :
>
>
>
>
> > "David Hill" <dd...@zk3.dec.com> writes:
> >
> > > Jeff,
> > >
> > > > of course :) (I also remember your other problem where I suggested
> > > > running buildconf to pick up local libtool... what happened with
> > > > that?)
> > >
> > > Just got around to trying that (long queue, what with a brand new Apache
> 2
> > > to play with :-). Yes, the latest libtool installed 1.4.2 and the
> tarball
> > > builds without a hitch. I saw a thread in dev@ talking about trying to
> > > resync with the public lib tool. Hopefully the libtool sync will make it
> > > into the next spin of the tarball.
>
> Found another oddity.... not sure where this one comes from, but doing the
> ./buildconf results in a srclib/apr/Makefile with top_builddir set to
> nothing, which causes an error during install. Part of what I don't
> understand is that without running ./buildconf, the build results in
> top_builddir matching top_srcdir which means the the failing line in the
> install phase is going to copy the header files that the previous line just
> copied........
Sander and I were looking at this earlier today - it's yet another autoconf
2.53 "feature". Sander posted a diff to IRC that appears to be a workaround
for the problem, but the real fix appears to be along the lines of what you
suggest - we should always use the builddir unless it is unset. It should
eliminate a lot of redundancy, and be a bit more fail proof too.
Thoughts?
-Thom
--
Thom May -> thom@planetarytramp.net
The memory management on PowerPC can be used to frighten small children.
-- Linus Torvalds
Re: connect to listener warning message
Posted by David Hill <dd...@zk3.dec.com>.
> "David Hill" <dd...@zk3.dec.com> writes:
>
> > Jeff,
> >
> > > of course :) (I also remember your other problem where I suggested
> > > running buildconf to pick up local libtool... what happened with
> > > that?)
> >
> > Just got around to trying that (long queue, what with a brand new Apache
2
> > to play with :-). Yes, the latest libtool installed 1.4.2 and the
tarball
> > builds without a hitch. I saw a thread in dev@ talking about trying to
> > resync with the public lib tool. Hopefully the libtool sync will make it
> > into the next spin of the tarball.
Found another oddity.... not sure where this one comes from, but doing the
./buildconf results in a srclib/apr/Makefile with top_builddir set to
nothing, which causes an error during install. Part of what I don't
understand is that without running ./buildconf, the build results in
top_builddir matching top_srcdir which means the the failing line in the
install phase is going to copy the header files that the previous line just
copied........
dave hill
Re: connect to listener warning message
Posted by David Hill <dd...@zk3.dec.com>.
I spoke too soon on the libtool thingie.... the box I tried it on had some
other stuff on it. Also, on this particular box, for some reason it did not
find threads (specifically pthread.h according to apr/config.log).
On a cleaner box, I found I had to install GNU m4, then GNU autoconf as well
as libtool before buildconf would work.
So on this box, I unpacked the tarball, ran ./buildconf and then did a:
CC='cc -std1'
./configure --prefix='/usr/internet/httpd2' --enable-debug --enable-maintain
er-mode
and then did a build. The build completed without error. But... no threads.
apr/config.log shows that it barfed on finding pthreads.h because of
no -pthread on the cc line.
I made it CC='cc -std1 -pthread' ./configure and things got better. The
build ran to completion, with prefork as the default (Thought it was
supposed to be perchild?). Is this expected that I need to specify -pthread
?
>kernel lets us get IPv6 socket
>kernel lets us bind to IN6ADDR_ANY (what is your Listen statement?)
>kernel won't let us connect to IN6ADDR_ANY (::)
That would match what I am seeing... I will send you my two test programs so
you can see what I did. I pirated most of the code from the examples that
are published with our OS. BTW, my network guys don't dispute this (yet),
but want more time to look/think.
> Here is something else to try with jist on the weird machine to
> verify:
>
> socket af_inet6
> assign s1 lastfd
> bind s1 :: 5555
> listen s1
> socket af_inet6
> assign s2 lastfd
> connect s2 :: 5555
>
> I suspect that the bind() will work but the connect() will fail.
two runs....
# done on a ipv6 enabled machine
# jist
Enter a command (or "help"):
socket af_inet6
Defaulting to type 1...
Defaulting to protocol 0...
The descriptor is 3.
Enter a command (or "help"):
assign s1 lastfd
Enter a command (or "help"):
bind s1 :: 5555
Enter a command (or "help"):
listen s1
Enter a command (or "help"):
socket af_inet6
Defaulting to type 1...
Defaulting to protocol 0...
The descriptor is 4.
Enter a command (or "help"):
assign s2 lastfd
Enter a command (or "help"):
connect s2 :: 5555
connected...
Enter a command (or "help"):
# machine with no ipv6 enabled
# jist
Enter a command (or "help"):
socket af_inet6
Defaulting to type 1...
Defaulting to protocol 0...
The descriptor is 3.
Enter a command (or "help"):
assign s1 lastfd
Enter a command (or "help"):
bind s1 :: 5555
Enter a command (or "help"):
listen s1
Enter a command (or "help"):
socket af_inet6
Defaulting to type 1...
Defaulting to protocol 0...
The descriptor is 4.
Enter a command (or "help"):
assign s2 lastfd
Enter a command (or "help"):
connect s2 :: 5555
connect: Can't assign requested address
errno 49
The request failed.
Enter a command (or "help"):
> Then either really disable IPv6 (such that you can't bind to ::) or
> really enable IPv6 (make sure loopback has an IPv6 address) and Apache
> should work fine. (lots of assumptions here :) )
That is easier said than done on the two boxes I am using. Will try to do
that later after I set up another box.
> I wonder if this machine would be considered "properly configured." I
> wonder how many machines are configured such that this would happen.
> I wonder if Apache, when determining at startup if it should try to do
> IPv6, should make sure something like the jist scenario above will
> actually work (with an ephemeral port, of course) before deciding to
> use IPv6.
The ipv4 machine is a machine with a fairly normal default configuration. I
am under the impression that the networks guys try to do as much fallback to
v4 under the hood as possible. I would have thought that :: would fall back
to the equivilent of 0.0.0.0 if v6 is not enabled.
BTW, I am out Friday, back Monday.
later,
Dave Hill
Re: connect to listener warning message
Posted by Jeff Trawick <tr...@attglobal.net>.
"David Hill" <dd...@zk3.dec.com> writes:
> > A lookup of :: is supposed to get you in6addr_any :)... I'll have to
> > look at this further later... Thanks for showing the jist results...
>
> Sounds like what I was telling them :-) And I added that in6addr_any should
> work regardless of v4 or v6 otherwise us poor programmers would have to
> special case our code. They are sympathetic and are looking into the
> plumbing to see if they can figure out why it does not seem to work. I will
> let you know what they say when they get back to me.
I don't have 100% confidence that I know what is going on. Here is
what I suspect:
kernel lets us get IPv6 socket
kernel lets us bind to IN6ADDR_ANY (what is your Listen statement?)
kernel won't let us connect to IN6ADDR_ANY (::)
It seems like a Solaris box I was using recently was somehow
configured with IPv6 support except that there were no interfaces with
an IPv6 address, and the same thing happened.
Here is something else to try with jist on the weird machine to
verify:
socket af_inet6
assign s1 lastfd
bind s1 :: 5555
listen s1
socket af_inet6
assign s2 lastfd
connect s2 :: 5555
I suspect that the bind() will work but the connect() will fail.
Then either really disable IPv6 (such that you can't bind to ::) or
really enable IPv6 (make sure loopback has an IPv6 address) and Apache
should work fine. (lots of assumptions here :) )
I wonder if this machine would be considered "properly configured." I
wonder how many machines are configured such that this would happen.
I wonder if Apache, when determining at startup if it should try to do
IPv6, should make sure something like the jist scenario above will
actually work (with an ephemeral port, of course) before deciding to
use IPv6.
--
Jeff Trawick | trawick@attglobal.net
Born in Roswell... married an alien...
Re: connect to listener warning message
Posted by David Hill <dd...@zk3.dec.com>.
> I just added a Tru64 note at
>
> http://www.apache.org/dist/httpd/
>
> (click on the "Check here to see" link right under the links to the
> tarballs)
Cool, thanks.
> A lookup of :: is supposed to get you in6addr_any :)... I'll have to
> look at this further later... Thanks for showing the jist results...
Sounds like what I was telling them :-) And I added that in6addr_any should
work regardless of v4 or v6 otherwise us poor programmers would have to
special case our code. They are sympathetic and are looking into the
plumbing to see if they can figure out why it does not seem to work. I will
let you know what they say when they get back to me.
thanks again,
Dave Hill
Re: connect to listener warning message
Posted by Jeff Trawick <tr...@attglobal.net>.
"David Hill" <dd...@zk3.dec.com> writes:
> Jeff,
>
> > of course :) (I also remember your other problem where I suggested
> > running buildconf to pick up local libtool... what happened with
> > that?)
>
> Just got around to trying that (long queue, what with a brand new Apache 2
> to play with :-). Yes, the latest libtool installed 1.4.2 and the tarball
> builds without a hitch. I saw a thread in dev@ talking about trying to
> resync with the public lib tool. Hopefully the libtool sync will make it
> into the next spin of the tarball.
I don't expect any miracles myself. There is no one libtool that
works on all the platforms we support. If there ever is, then great!
I just added a Tru64 note at
http://www.apache.org/dist/httpd/
(click on the "Check here to see" link right under the links to the
tarballs)
hudson has 1.3.5 and I just verified that using the libtool in the
Apache tarball there results in the -pthread problem but using 1.3.5
is fine.
> A bug somewhere in our network code/implementation ? I wrote a small program
> to setup a simple listener using the v6 api and another small program to
> connect to it also using the v6 api and was able to reproduce the fact that
> I could not connect to :: on the machine without ipv6 enabled. But... on the
> server side I did use in6addr_any as opposed to doing a lookup of :: and
> then using that addr.
A lookup of :: is supposed to get you in6addr_any :)... I'll have to
look at this further later... Thanks for showing the jist results...
--
Jeff Trawick | trawick@attglobal.net
Born in Roswell... married an alien...
Re: connect to listener warning message
Posted by David Hill <dd...@zk3.dec.com>.
Jeff,
> of course :) (I also remember your other problem where I suggested
> running buildconf to pick up local libtool... what happened with
> that?)
Just got around to trying that (long queue, what with a brand new Apache 2
to play with :-). Yes, the latest libtool installed 1.4.2 and the tarball
builds without a hitch. I saw a thread in dev@ talking about trying to
resync with the public lib tool. Hopefully the libtool sync will make it
into the next spin of the tarball.
> > When I build, I get (and want) IPv6 support, but when I run the image on
a
> > IPv4 only machine, everything seems to work but the POD. Turns out that
this
> > fails because getaddrinfo("::"...) is not providing a usefull value on
an
> > IPv4 only machine, at least as far as the POD connection goes.
>
> Weird. We only chose "::" because we were able to get an IPv6 socket
> (we tried at run-time in case the build machine and the run machine
> had different capabilities).
A bug somewhere in our network code/implementation ? I wrote a small program
to setup a simple listener using the v6 api and another small program to
connect to it also using the v6 api and was able to reproduce the fact that
I could not connect to :: on the machine without ipv6 enabled. But... on the
server side I did use in6addr_any as opposed to doing a lookup of :: and
then using that addr.
>
> > When I hacked
> > listen.c changing "::" to "0.0.0.0" then everything seems to work fine.
I am
> > talking with my network guys about correcting this, but in the mean time
the
> > listen.c hack is simple for me and works.
>
> Before your network guys fix this broken environment, if you have time
> I'd love a little double-checking. If you could grab the
> /home/trawick/ra/jist executable file from hudson, stick it on that
> machine, and do this:
>
> start ./jist then run these jist commands:
>
> socket af_inet6
> getaddrinfo af_unspec ::
> getaddrinfo af_inet6 ::
> (generate EOF to get out of jist)
>
> and send the output to me
Got same results on 2 machines, one without ipv6 configured. results follow.
Anything else you would like me to try?
thanks,
Dave
# ./jist
Enter a command (or "help"):
socket af_inet6
Defaulting to type 1...
Defaulting to protocol 0...
The descriptor is 3.
Enter a command (or "help"):
getaddrinfo af_unspec ::
Getaddrinfo results:
IP Addr: ::
Family: 26
Flags: 0
Port: 0
Canonname: "::"
Enter a command (or "help"):
getaddrinfo af_inet6 ::
Getaddrinfo results:
IP Addr: ::
Family: 26
Flags: 0
Port: 0
Canonname: "::"
Enter a command (or "help"):
Re: connect to listener warning message
Posted by Jeff Trawick <tr...@attglobal.net>.
"David Hill" <dd...@zk3.dec.com> writes:
> Jeff,
> remember this one ?
of course :) (I also remember your other problem where I suggested
running buildconf to pick up local libtool... what happened with
that?)
> When I build, I get (and want) IPv6 support, but when I run the image on a
> IPv4 only machine, everything seems to work but the POD. Turns out that this
> fails because getaddrinfo("::"...) is not providing a usefull value on an
> IPv4 only machine, at least as far as the POD connection goes.
Weird. We only chose "::" because we were able to get an IPv6 socket
(we tried at run-time in case the build machine and the run machine
had different capabilities).
> When I hacked
> listen.c changing "::" to "0.0.0.0" then everything seems to work fine. I am
> talking with my network guys about correcting this, but in the mean time the
> listen.c hack is simple for me and works.
Before your network guys fix this broken environment, if you have time
I'd love a little double-checking. If you could grab the
/home/trawick/ra/jist executable file from hudson, stick it on that
machine, and do this:
start ./jist then run these jist commands:
socket af_inet6
getaddrinfo af_unspec ::
getaddrinfo af_inet6 ::
(generate EOF to get out of jist)
and send the output to me
Hmmm...
--
Jeff Trawick | trawick@attglobal.net
Born in Roswell... married an alien...
Re: connect to listener warning message
Posted by David Hill <dd...@zk3.dec.com>.
Jeff,
remember this one ?
Found my answer to what is going on, have a workaround but not a fix.
Thought you might be interested in what I found.
When I build, I get (and want) IPv6 support, but when I run the image on a
IPv4 only machine, everything seems to work but the POD. Turns out that this
fails because getaddrinfo("::"...) is not providing a usefull value on an
IPv4 only machine, at least as far as the POD connection goes. When I hacked
listen.c changing "::" to "0.0.0.0" then everything seems to work fine. I am
talking with my network guys about correcting this, but in the mean time the
listen.c hack is simple for me and works.
Dave Hill
----- Original Message -----
From: "Jeff Trawick" <tr...@attglobal.net>
To: <de...@httpd.apache.org>
Sent: Saturday, March 30, 2002 8:51 PM
Subject: Re: connect to listener warning message
> "David Hill" <dd...@zk3.dec.com> writes:
>
> > Hi all,
> > When I run 2.0.32 on Compaq Tru64 and give it a bit of a load (ok, I
whack it good :-), I get warnings similar to the following looping into the
error log file. I am getting one per second, even well after the load is
removed.I do not get any of these until after the load has been applied for
a 10 seconds or so.
> >
> > [Thu Mar 28 13:11:43 2002] [warn] (49)Can't assign requested address:
connect to listener
>
> "one per second" sounds like idle server maintenance
>
> "can't assign requested address" on a connect sounds like your machine
> is out of ephemeral ports (you were whacking it over loopback I'll
> bet) to assign to the source socket doing the dummy connection
>
> > Any suggestions ?
>
> tune the os to have more ephemeral ports??
>
> --
> Jeff Trawick | trawick@attglobal.net
> Born in Roswell... married an alien...
>
>
Re: connect to listener warning message
Posted by David Hill <dd...@zk3.dec.com>.
> > [Thu Mar 28 13:11:43 2002] [warn] (49)Can't assign requested address:
connect to listener
>
> "one per second" sounds like idle server maintenance
That does seem to be the Apache code where the message is coming from based
on the comments around it.
>
> "can't assign requested address" on a connect sounds like your machine
> is out of ephemeral ports (you were whacking it over loopback I'll
> bet) to assign to the source socket doing the dummy connection
Actually no - I am wacking it from a remote machine.
I think that something else is going on, but what I don't know at the
moment. The problem starts when a load is put on the machine but continues
well after the load is removed (as in hours). The machine is tuned well
enough that it handles the load without a sweat with 1.3
I can reproduce the problem with ./ab -n 400 -c 20
http://hostname/staticfile which should hardly be enough load to warm up the
cpu.
I am going to try turning off ipv6 to see if that makes any difference, and
if that does not put some debugging code in to see if I can figure out what
is going on.
Any other suggestions as to what to look for would be appreciated.
Dave Hill
Re: connect to listener warning message
Posted by Jeff Trawick <tr...@attglobal.net>.
"David Hill" <dd...@zk3.dec.com> writes:
> Hi all,
> When I run 2.0.32 on Compaq Tru64 and give it a bit of a load (ok, I whack it good :-), I get warnings similar to the following looping into the error log file. I am getting one per second, even well after the load is removed.I do not get any of these until after the load has been applied for a 10 seconds or so.
>
> [Thu Mar 28 13:11:43 2002] [warn] (49)Can't assign requested address: connect to listener
"one per second" sounds like idle server maintenance
"can't assign requested address" on a connect sounds like your machine
is out of ephemeral ports (you were whacking it over loopback I'll
bet) to assign to the source socket doing the dummy connection
> Any suggestions ?
tune the os to have more ephemeral ports??
--
Jeff Trawick | trawick@attglobal.net
Born in Roswell... married an alien...