You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by Randy Terbush <ra...@zyzzyva.com> on 1997/07/05 19:20:18 UTC
www.jpl.nasa.gov
Anyone know someone at the JPL? This site is surely being hammered
right now and is in my opinion not handling it too well. Quite a
few performance enhancements since b7. It would be nice to help
them out if possible.
=> telnet www.jpl.nasa.gov 80
Trying 137.78.99.23...
Connected to www.jpl.nasa.gov.
Escape character is '^]'.
HEAD / HTTP/1.0
HTTP/1.1 200 OK
Date: Sat, 05 Jul 1997 17:08:00 GMT
Server: Apache/1.2b7
Connection: close
Content-Type: text/html
Last-Modified: Fri, 04 Jul 1997 17:55:54 GMT
ETag: "512-e01-33bd392a"
Content-Length: 3585
Accept-Ranges: bytes
Re: apache and threading (was Re: www.jpl.nasa.gov)
Posted by Marc Slemko <ma...@worldgate.com>.
On Sun, 6 Jul 1997, Marc Slemko wrote:
> Sigh. I'm trying to think of a way for FreeBSD, but all I have come up
> with is using the kernel debugger, which isn't exactly the greatest thing.
Hmm. procfs gives me:
0x1000 0x36000 45 0 r-x COW vnode
0x36000 0x38000 2 2 rwx COW vnode
0x38000 0x93000 45 7 rwx COW swap
0x93000 0x98000 0 0 rwx COW swap
0x98000 0xa0000 5 5 rwx default
0x8036000 0x8044000 10 0 r-x COW vnode
0x8044000 0x8046000 2 0 rwx COW vnode
0x8046000 0x804f000 5 2 rwx COW swap
0x8050000 0x8053000 3 0 r-x COW vnode
0x8053000 0x8054000 1 0 rwx COW vnode
0x8054000 0x8065000 0 0 rwx none
0x8065000 0x80c6000 68 0 r-x COW vnode
0x80c6000 0x80ca000 4 3 rwx COW vnode
0x80ca000 0x80e1000 9 8 rwx COW swap
0x80e1000 0x80ed000 12 0 rwx default
0xefbde000 0xefbfe000 5 4 rwx COW swap
(on a httpd process, with a moderate set of modules compiled in)
When I add that up, I get 151552 bytes non-COW. Everything still COW
is presumably just a reference to a shared physical page, so I don't
care about it.
I do wonder about the map entry with the type of "none" though;
I'm not sure what the vm system does with that. It isn't standard
BSD any more, I think dg did a bunch of changes there. That map
entry may be bogus, in which case it could drop to 81920 bytes,
plus process overhead of course.
Interesting. I will have to look at my numbers again. This does not
agree with my experience.
Re: apache and threading (was Re: www.jpl.nasa.gov)
Posted by Marc Slemko <ma...@worldgate.com>.
On Sun, 6 Jul 1997, Dean Gaudet wrote:
> I took another look, in case I was hallucinating last time I checked
> this server. It's averaging 100k unshared per child right now (based
> on ps -max which displays the number of shared pages). Task gunk takes
> about 12k on linux, and so accounting 112k per task I get about 30Mb.
> The system reports 38Mb of RAM in use (non-disk page). So it checks out.
>
> Incidentally this server and its twin run hourly restarts and the parents
> have been going for over 2.7 million seconds, almost a month!
>
> Anyone know how to get the same data from a solaris box? I haven't seen
> a method of determining shared pages.
Sigh. I'm trying to think of a way for FreeBSD, but all I have come up
with is using the kernel debugger, which isn't exactly the greatest thing.
USTLing and checking the daemon book right now...
Oh yea, does anyone have a threaded Unix implementation that will run on
FreeBSD for comparison? Shucks, didn't think so. Sigh.
>
> I guess I need to rethink my attitude towards threads if linux is the only
> one that somehow manages to be this aggressive. Or perhaps we're doing
> something silly that causes unsharing on the other systems.
>
> Dean
>
> On Sun, 6 Jul 1997, Marc Slemko wrote:
>
> > On Sun, 6 Jul 1997, Dean Gaudet wrote:
> >
> > > Apache as it is will not benefit as much from threading as people are
> > > hoping.
> > >
> > > Consider memory usage. After a fork(), what actually has to be
> > > copied/unshared? Pretty much nothing. We don't dink with the pconf pool
> > > used to hold the config info, we dink with a few globals. We create
> > > new pools for each request. So pretty much everything up to fork time
> > > is shared between all children. I see this now, on a busy linux server
> > > I see children running with 80k unshared data, and 770k shared.
> >
> > I don't see that under FreeBSD.
> >
> > On a 64 meg server, I can only hit a couple of hundred children before
> > running out of memory. Using your numbers, 500 shouldn't be a problem. I
> > didn't think Linux was that much better than FreeBSD at sharing...
> >
> >
>
Re: apache and threading (was Re: www.jpl.nasa.gov)
Posted by Dean Gaudet <dg...@arctic.org>.
I took another look, in case I was hallucinating last time I checked
this server. It's averaging 100k unshared per child right now (based
on ps -max which displays the number of shared pages). Task gunk takes
about 12k on linux, and so accounting 112k per task I get about 30Mb.
The system reports 38Mb of RAM in use (non-disk page). So it checks out.
Incidentally this server and its twin run hourly restarts and the parents
have been going for over 2.7 million seconds, almost a month!
Anyone know how to get the same data from a solaris box? I haven't seen
a method of determining shared pages.
I guess I need to rethink my attitude towards threads if linux is the only
one that somehow manages to be this aggressive. Or perhaps we're doing
something silly that causes unsharing on the other systems.
Dean
On Sun, 6 Jul 1997, Marc Slemko wrote:
> On Sun, 6 Jul 1997, Dean Gaudet wrote:
>
> > Apache as it is will not benefit as much from threading as people are
> > hoping.
> >
> > Consider memory usage. After a fork(), what actually has to be
> > copied/unshared? Pretty much nothing. We don't dink with the pconf pool
> > used to hold the config info, we dink with a few globals. We create
> > new pools for each request. So pretty much everything up to fork time
> > is shared between all children. I see this now, on a busy linux server
> > I see children running with 80k unshared data, and 770k shared.
>
> I don't see that under FreeBSD.
>
> On a 64 meg server, I can only hit a couple of hundred children before
> running out of memory. Using your numbers, 500 shouldn't be a problem. I
> didn't think Linux was that much better than FreeBSD at sharing...
>
>
Re: apache and threading (was Re: www.jpl.nasa.gov)
Posted by Marc Slemko <ma...@worldgate.com>.
On Sun, 6 Jul 1997, Dean Gaudet wrote:
> Apache as it is will not benefit as much from threading as people are
> hoping.
>
> Consider memory usage. After a fork(), what actually has to be
> copied/unshared? Pretty much nothing. We don't dink with the pconf pool
> used to hold the config info, we dink with a few globals. We create
> new pools for each request. So pretty much everything up to fork time
> is shared between all children. I see this now, on a busy linux server
> I see children running with 80k unshared data, and 770k shared.
I don't see that under FreeBSD.
On a 64 meg server, I can only hit a couple of hundred children before
running out of memory. Using your numbers, 500 shouldn't be a problem. I
didn't think Linux was that much better than FreeBSD at sharing...
apache and threading (was Re: www.jpl.nasa.gov)
Posted by Dean Gaudet <dg...@arctic.org>.
Apache as it is will not benefit as much from threading as people are
hoping.
Consider memory usage. After a fork(), what actually has to be
copied/unshared? Pretty much nothing. We don't dink with the pconf pool
used to hold the config info, we dink with a few globals. We create
new pools for each request. So pretty much everything up to fork time
is shared between all children. I see this now, on a busy linux server
I see children running with 80k unshared data, and 770k shared.
The threading model we are considering is to replace the fork() with a
thr_create() (or whatever). At that point the thread is running a request
in its own pool. The only shared data is all the config data, and stuff
that isn't ever "unshared" in the current model. We get a bit finer
granularity, we probably save 16k or 20k per process. But that's not huge
(and we could improve the current forking code with a little more
judicious allocation of globals, page aligning and crud).
Ok, so what else do we gain from threading as currently proposed? Well we
gain the opportunity of process-wide caches for various things like mmap()d
open()d files. We could do this without threads, but we'd probably get a
lower hit-ratio, making them a lot less useful. So this is a Good Thing.
But is it good enough? I dunno, we'll see though.
For an example of how threading can be made to do things faster take a look
at JAWS on NT. See
<http://www.cs.wustl.edu/~jxh/research/research.html#papers> for an
interesting paper on server performance over ATM. Apache sucks, and it
will still suck if it's just naively threaded. (Although they're using
slightly old code, and I've asked them to try again with 1.2 release.)
The essential idea is that you have enough threads to take advantage
of parallelism available in the hardware, in the operating system, and
in asynch i/o. To do that you need to either run a user-level threads
abstraction over top of the OS threads, or you need to completely revamp
the code to do event-style processing. Squid, and I'm guessing Zeus,
fall in that latter category. A thread picks up a job, and works it up
to an i/o point then goes back to the "i/o done" queue to pick up another
job whose i/o is done and processes it up to an i/o point ... and so on.
(Squid is only single threaded at the moment though, but has multithreaded
asynch i/o in recent development versions.)
Apache isn't written as an event processing loop. But it does fit well
with the user-threads over top of OS threads model. But we'd need to find
a portable threads library which implements this async i/o abstraction
in a way that we'd be able to use it.
Dean
P.S. Despite my dislike for NT, I like those extra primitives they supply
for speeding up very common program operations... like "Open File and Read
First Block", and "Copy descriptor to descriptor".
Re: www.jpl.nasa.gov
Posted by Marc Slemko <ma...@worldgate.com>.
On Sun, 6 Jul 1997, Dean Gaudet wrote:
> This would have made an excellent test case for the optimizations I put
> into 1.3 last monday :)
Would make a better test cas efor threading. Sigh.
>
> Dean
>
> On Sat, 5 Jul 1997, Randy Terbush wrote:
>
> > Anyone know someone at the JPL? This site is surely being hammered
> > right now and is in my opinion not handling it too well. Quite a
> > few performance enhancements since b7. It would be nice to help
> > them out if possible.
> >
> >
> >
> > => telnet www.jpl.nasa.gov 80
> > Trying 137.78.99.23...
> > Connected to www.jpl.nasa.gov.
> > Escape character is '^]'.
> > HEAD / HTTP/1.0
> >
> > HTTP/1.1 200 OK
> > Date: Sat, 05 Jul 1997 17:08:00 GMT
> > Server: Apache/1.2b7
> > Connection: close
> > Content-Type: text/html
> > Last-Modified: Fri, 04 Jul 1997 17:55:54 GMT
> > ETag: "512-e01-33bd392a"
> > Content-Length: 3585
> > Accept-Ranges: bytes
> >
> >
> >
> >
>
Re: www.jpl.nasa.gov
Posted by Dean Gaudet <dg...@arctic.org>.
This would have made an excellent test case for the optimizations I put
into 1.3 last monday :)
Dean
On Sat, 5 Jul 1997, Randy Terbush wrote:
> Anyone know someone at the JPL? This site is surely being hammered
> right now and is in my opinion not handling it too well. Quite a
> few performance enhancements since b7. It would be nice to help
> them out if possible.
>
>
>
> => telnet www.jpl.nasa.gov 80
> Trying 137.78.99.23...
> Connected to www.jpl.nasa.gov.
> Escape character is '^]'.
> HEAD / HTTP/1.0
>
> HTTP/1.1 200 OK
> Date: Sat, 05 Jul 1997 17:08:00 GMT
> Server: Apache/1.2b7
> Connection: close
> Content-Type: text/html
> Last-Modified: Fri, 04 Jul 1997 17:55:54 GMT
> ETag: "512-e01-33bd392a"
> Content-Length: 3585
> Accept-Ranges: bytes
>
>
>
>