You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by Ben Laurie <be...@algroup.co.uk> on 1998/09/06 15:06:36 UTC
Apache 2.0/NSPR
As some of you will have noticed, I was playing with Apache 2.0 (nice
work, Dean). So now I'm wondering what the status of NSPR is - I know
Dean was after some changes, and the one that is bugging me is the
working directory of subprocesses - I know Dean was talking to the NSPR
team, but what's the state of play? Anyone know? Or do I wait for Dean?
Cheers,
Ben.
--
Ben Laurie |Phone: +44 (181) 735 0686| Apache Group member
Freelance Consultant |Fax: +44 (181) 735 0689|http://www.apache.org/
and Technical Director|Email: ben@algroup.co.uk |
A.L. Digital Ltd, |Apache-SSL author http://www.apache-ssl.org/
London, England. |"Apache: TDG" http://www.ora.com/catalog/apache/
WE'RE RECRUITING! http://www.aldigital.co.uk/
Re: Apache 2.0/NSPR
Posted by Ben Laurie <be...@algroup.co.uk>.
Dean Gaudet wrote:
>
> On Wed, 9 Sep 1998, Ben Laurie wrote:
>
> > Hmm. You made some cryptic comments, particularly with reference to
> > flushing. A bit of clarity in those comments would be nice
> > (alternatively a licence to ignore them will do).
>
> Oh, the layering in NSPR doesn't provide a distinction between "flush all
> the way down the stack" and "flush just this one layer". Both operations
> are important when buffering is involved. Or at least I think they're
> important. hmm. Maybe not, maybe I was deluded. I can't figure out why
> "flush one layer" is useful now.
I share your doubt. Clearly flushing is good. Single layer flushing
seems less obviously useful.
Cheers,
Ben.
--
Ben Laurie |Phone: +44 (181) 735 0686| Apache Group member
Freelance Consultant |Fax: +44 (181) 735 0689|http://www.apache.org/
and Technical Director|Email: ben@algroup.co.uk |
A.L. Digital Ltd, |Apache-SSL author http://www.apache-ssl.org/
London, England. |"Apache: TDG" http://www.ora.com/catalog/apache/
WE'RE RECRUITING! http://www.aldigital.co.uk/
Re: Apache 2.0/NSPR
Posted by Alexei Kosut <ak...@leland.Stanford.EDU>.
On Thu, 10 Sep 1998, Ben Laurie wrote:
> > A year ago, when we were discussing our layered BUFF stuff, Ed and I
> > discussed this a bunch, and the only time we could figure when a "flush
> > one layer" would be useful is when removing that layer from the stack. But
> > we figured that the pop operation would do that when it removed the layer.
> > Assuming NSPR provides a callback for when a layer is removed, that should
> > work.
> >
> > (I don't know if it does)
>
> If a layer doesn't know its being removed, surely you are going to have
> problems? And that isn't just flushing - it may have to shut down
> protocols and stuff.
Right. This is why I assume that NSPR provides a remove-layer callback.
Someone should check on that :)
Hmm... a quick glance at nsprpub/pr/include/prio.h reveals a dtor function
pointer in PRFileDesc, described as "a destructor function for layer."
Presuming this is called when a layer is removed, it is sufficient. We
just need to write in the Apache 2.0 API Programming Guide (hehe) that
NSPR layers used in Apache should make sure they write all their data to
the next layer when destructed, but that they shouldn't call flush.
(flushing is bad - consider pipelined requests)
I think that's sufficient.
-- Alexei Kosut <ak...@stanford.edu> <http://www.stanford.edu/~akosut/>
Stanford University, Class of 2001 * Apache <http://www.apache.org> *
Re: Apache 2.0/NSPR
Posted by Ben Laurie <be...@algroup.co.uk>.
Alexei Kosut wrote:
>
> On Wed, 9 Sep 1998, Dean Gaudet wrote:
>
> >
> >
> > On Wed, 9 Sep 1998, Ben Laurie wrote:
> >
> > > Hmm. You made some cryptic comments, particularly with reference to
> > > flushing. A bit of clarity in those comments would be nice
> > > (alternatively a licence to ignore them will do).
> >
> > Oh, the layering in NSPR doesn't provide a distinction between "flush all
> > the way down the stack" and "flush just this one layer". Both operations
> > are important when buffering is involved. Or at least I think they're
> > important. hmm. Maybe not, maybe I was deluded. I can't figure out why
> > "flush one layer" is useful now.
>
> A year ago, when we were discussing our layered BUFF stuff, Ed and I
> discussed this a bunch, and the only time we could figure when a "flush
> one layer" would be useful is when removing that layer from the stack. But
> we figured that the pop operation would do that when it removed the layer.
> Assuming NSPR provides a callback for when a layer is removed, that should
> work.
>
> (I don't know if it does)
If a layer doesn't know its being removed, surely you are going to have
problems? And that isn't just flushing - it may have to shut down
protocols and stuff.
Cheers,
Ben.
--
Ben Laurie |Phone: +44 (181) 735 0686| Apache Group member
Freelance Consultant |Fax: +44 (181) 735 0689|http://www.apache.org/
and Technical Director|Email: ben@algroup.co.uk |
A.L. Digital Ltd, |Apache-SSL author http://www.apache-ssl.org/
London, England. |"Apache: TDG" http://www.ora.com/catalog/apache/
WE'RE RECRUITING! http://www.aldigital.co.uk/
Re: Apache 2.0/NSPR
Posted by Alexei Kosut <ak...@leland.Stanford.EDU>.
On Wed, 9 Sep 1998, Dean Gaudet wrote:
>
>
> On Wed, 9 Sep 1998, Ben Laurie wrote:
>
> > Hmm. You made some cryptic comments, particularly with reference to
> > flushing. A bit of clarity in those comments would be nice
> > (alternatively a licence to ignore them will do).
>
> Oh, the layering in NSPR doesn't provide a distinction between "flush all
> the way down the stack" and "flush just this one layer". Both operations
> are important when buffering is involved. Or at least I think they're
> important. hmm. Maybe not, maybe I was deluded. I can't figure out why
> "flush one layer" is useful now.
A year ago, when we were discussing our layered BUFF stuff, Ed and I
discussed this a bunch, and the only time we could figure when a "flush
one layer" would be useful is when removing that layer from the stack. But
we figured that the pop operation would do that when it removed the layer.
Assuming NSPR provides a callback for when a layer is removed, that should
work.
(I don't know if it does)
-- Alexei Kosut <ak...@stanford.edu> <http://www.stanford.edu/~akosut/>
Stanford University, Class of 2001 * Apache <http://www.apache.org> *
Re: Apache 2.0/NSPR
Posted by Dean Gaudet <dg...@arctic.org>.
On Wed, 9 Sep 1998, Ben Laurie wrote:
> Hmm. You made some cryptic comments, particularly with reference to
> flushing. A bit of clarity in those comments would be nice
> (alternatively a licence to ignore them will do).
Oh, the layering in NSPR doesn't provide a distinction between "flush all
the way down the stack" and "flush just this one layer". Both operations
are important when buffering is involved. Or at least I think they're
important. hmm. Maybe not, maybe I was deluded. I can't figure out why
"flush one layer" is useful now.
Dean
Re: Apache 2.0/NSPR
Posted by Ben Laurie <be...@algroup.co.uk>.
Dean Gaudet wrote:
>
> On Wed, 9 Sep 1998, Ben Laurie wrote:
>
> > I didn't want to replicate effort with the NSPR guys - but I couldn't
> > wait, so I've been talking to them anyway - looks like I'll be sorting
> > out the working directory thing, at least.
>
> Right, I don't think you'll step on them at all, they were pretty happy
> with the proposal we came up for this. I documented it in one of the
> readme files.
Yep. That's essentially what I'm planning to implement. It seems to me
to be the natural way to do it, anyway.
> > Then I was thinking of
> > looking more closely at layered I/O - what would be most valuable for me
> > is if you could make it clearer where you are at with that, before I
> > shoot us _both_ in the foot :-)
>
> I've done nothing on it...
Hmm. You made some cryptic comments, particularly with reference to
flushing. A bit of clarity in those comments would be nice
(alternatively a licence to ignore them will do).
> > Before it confuses you, I should mention I moved your Configuration to
> > Configuration.dean to avoid a nasty fight :-)
>
> Yeah I had work in progress (I forget where it is) that had a top level
> make script for folks to build nspr and apache-nspr together... never
> finished it/wasn't happy with it.
Yeah, well ... I just built against the current NSPR and it seems to
work - I'll probably tweak Configure to deal with it all at some point.
Not urgent, I feel.
> Oh other things you may want to add to your howto doc is "make
> BUILD_OPT=1" which builds the non-debugging version of NSPR.
Ah, OK. I'll add that. Of course, you can type "cvs ci" as well as I can
:-)
> I've got lots to do for my real work in the next month, so other than
> piping up and spouting my hot breath here I probably won't be committing
> much. Of course, every time I say that I write something new. ;)
Heh. Well, I'm currently standing in for two developers I haven't got,
as well as doing my own work, so count this as a displacement activity
:-)
Cheers,
Ben.
--
Ben Laurie |Phone: +44 (181) 735 0686| Apache Group member
Freelance Consultant |Fax: +44 (181) 735 0689|http://www.apache.org/
and Technical Director|Email: ben@algroup.co.uk |
A.L. Digital Ltd, |Apache-SSL author http://www.apache-ssl.org/
London, England. |"Apache: TDG" http://www.ora.com/catalog/apache/
WE'RE RECRUITING! http://www.aldigital.co.uk/
Re: Apache 2.0/NSPR
Posted by Dean Gaudet <dg...@arctic.org>.
On Wed, 9 Sep 1998, Ben Laurie wrote:
> I didn't want to replicate effort with the NSPR guys - but I couldn't
> wait, so I've been talking to them anyway - looks like I'll be sorting
> out the working directory thing, at least.
Right, I don't think you'll step on them at all, they were pretty happy
with the proposal we came up for this. I documented it in one of the
readme files.
> Then I was thinking of
> looking more closely at layered I/O - what would be most valuable for me
> is if you could make it clearer where you are at with that, before I
> shoot us _both_ in the foot :-)
I've done nothing on it...
> Before it confuses you, I should mention I moved your Configuration to
> Configuration.dean to avoid a nasty fight :-)
Yeah I had work in progress (I forget where it is) that had a top level
make script for folks to build nspr and apache-nspr together... never
finished it/wasn't happy with it.
Oh other things you may want to add to your howto doc is "make
BUILD_OPT=1" which builds the non-debugging version of NSPR.
I've got lots to do for my real work in the next month, so other than
piping up and spouting my hot breath here I probably won't be committing
much. Of course, every time I say that I write something new. ;)
Dean
Re: Apache 2.0/NSPR
Posted by Marc Slemko <ma...@znep.com>.
On Thu, 10 Sep 1998, Ben Laurie wrote:
> BTW, Alexei said some stuff about caching in between layers, which I
> seem to have mislaid. Let me take this opportunity to say that sounds
> cool and he should pursue it! I have some stuff to say about databases,
> but I'll wait until I've drunk less.
Or more. <g>
Re: Apache 2.0/NSPR
Posted by Ben Laurie <be...@algroup.co.uk>.
Alexei Kosut wrote:
>
> On Thu, 10 Sep 1998, Ben Laurie wrote:
>
> > Like I said, Dean mumbled something about flushes. Natch, if you have
> > threads, the approach is different to if you don't. I suppose the Big
> > Question is whether we are interested in implementing layered I/O for
> > non-threaded environments.
> >
> > BTW, Alexei said some stuff about caching in between layers, which I
> > seem to have mislaid. Let me take this opportunity to say that sounds
> > cool and he should pursue it! I have some stuff to say about databases,
> > but I'll wait until I've drunk less.
>
> How can you drink less? That's like saying "I'll wait until I'm less old."
That's a great idea. I'm waiting.
[snip explanation]
> Basically, that's how I think it should work. It's basically what we
> talked about in SF (sorry you weren't there Ben, you probably would have
> enjoyed it). Doesn't sound too hard to implement either, once we get the
> cache and middle end parts working perfectly (hah!).
Hehe. Anyway, the thing I wanted to say about databases (and I honestly
can't think why I let a couple of pints of wine hold me back) was this:
working out whether a cache is up-to-date for a database query is
interestingly non-trivial, but may be worth some effort. But anyway, it
illustrates the general point that to determine whether a cache is
up-to-date may require more than simply seeing what has varied, so a
general caching layer should permit a general way to query currency.
If you see what I mean.
Cheers,
Ben.
--
Ben Laurie |Phone: +44 (181) 735 0686| Apache Group member
Freelance Consultant |Fax: +44 (181) 735 0689|http://www.apache.org/
and Technical Director|Email: ben@algroup.co.uk |
A.L. Digital Ltd, |Apache-SSL author http://www.apache-ssl.org/
London, England. |"Apache: TDG" http://www.ora.com/catalog/apache/
WE'RE RECRUITING! http://www.aldigital.co.uk/
Re: Apache 2.0/NSPR
Posted by Rasmus Lerdorf <ra...@lerdorf.on.ca>.
> It's actually fairly simple to deal with: You have the PHP module tag the
> response as uncacheable unless you get specific information (via a PHP
> function or something) from the script about what dimensions it acts on.
Yup, I see no problem with this.
> P.S. A quick thought: it might not be that hard to determine the
> dimensions of a PHP script from the parser. You just need to classify each
> PHP function as to whether, passed the same arguments, it will return the
> same value every time. If it does, you can ignore it. If it doesn't, you
> need to specify what request dimensions it depends on, or if it depends on
> a non-request dimension (e.g., rand()). You should be able to put that
> together to figure out the proper metadata. But I may be missing some
> important factor.
Interesting thought. Anything that talks to an external source would of
course need to be labelled uncacheable. But you are right about the
trivial cases. A lot of people use PHP to just include some other files
and call some simple display functions.
ie.
<?
include "common.inc";
show_header();
?>
Static HTML
blah blah
<? show_footer() ?>
In a common case like this as long as the stuff done by common.inc does
not do any time, rand or external access stuff, it should be cacheable.
I just need to keep an internal state and have each uncacheable function
set it if used.
-Rasmus
Re: Apache 2.0/NSPR
Posted by Alexei Kosut <ak...@leland.Stanford.EDU>.
On Wed, 9 Sep 1998, Rasmus Lerdorf wrote:
> > (this last one can be done in the kernel, or in an otherwise streamlined
> > manner. Much speedup. And the neat thing is that neither the database nor
> > mod_php have to know *anything* about it, as long as they properly report
> > their metadata. We'll need to make that sure that's easy for module
> > authors to have their modules do.)
>
> Well, it goes a step beyond that. I would have to pass this on to the
> actual PHP script author. I can't analyze a PHP script and determine the
> metadata. Well, I could try, but there are so many different ways that a
> script can vary from one hit to the next that it wouldn't be feasible.
Yeah, that's the problem with generalized scripting languages. They're
too damn general ;)
It's actually fairly simple to deal with: You have the PHP module tag the
response as uncacheable unless you get specific information (via a PHP
function or something) from the script about what dimensions it acts on.
As I mentioned earlier, this actually is a point in favor of the plan I
outline: In the current HTTP cache system (e.g., Apache+Squid), if any
part of the response has unknown dimensions, we have to treat the whole
thing as uncacheable. With cached layering, we can at least cache the
parts of the request we can define.
P.S. A quick thought: it might not be that hard to determine the
dimensions of a PHP script from the parser. You just need to classify each
PHP function as to whether, passed the same arguments, it will return the
same value every time. If it does, you can ignore it. If it doesn't, you
need to specify what request dimensions it depends on, or if it depends on
a non-request dimension (e.g., rand()). You should be able to put that
together to figure out the proper metadata. But I may be missing some
important factor.
-- Alexei Kosut <ak...@stanford.edu> <http://www.stanford.edu/~akosut/>
Stanford University, Class of 2001 * Apache <http://www.apache.org> *
Re: Apache 2.0/NSPR
Posted by Rasmus Lerdorf <ra...@lerdorf.on.ca>.
> cache-read -> PHP -> cache-write -> client
>
> Or maybe it's the same request with the same User-Agent, and it can pull
> the whole thing out of the cache:
>
> cache-read -> client
>
> (this last one can be done in the kernel, or in an otherwise streamlined
> manner. Much speedup. And the neat thing is that neither the database nor
> mod_php have to know *anything* about it, as long as they properly report
> their metadata. We'll need to make that sure that's easy for module
> authors to have their modules do.)
Well, it goes a step beyond that. I would have to pass this on to the
actual PHP script author. I can't analyze a PHP script and determine the
metadata. Well, I could try, but there are so many different ways that a
script can vary from one hit to the next that it wouldn't be feasible.
-Rasmus
Re: Apache 2.0/NSPR
Posted by Alexei Kosut <ak...@leland.Stanford.EDU>.
On Thu, 10 Sep 1998, Ben Laurie wrote:
> Like I said, Dean mumbled something about flushes. Natch, if you have
> threads, the approach is different to if you don't. I suppose the Big
> Question is whether we are interested in implementing layered I/O for
> non-threaded environments.
>
> BTW, Alexei said some stuff about caching in between layers, which I
> seem to have mislaid. Let me take this opportunity to say that sounds
> cool and he should pursue it! I have some stuff to say about databases,
> but I'll wait until I've drunk less.
How can you drink less? That's like saying "I'll wait until I'm less old."
You already drank it. Too late now. Unless you drink enough to overflow
the drink counter and it goes back to zero. You might try that.
Anyway, the 'layers' I was talking about do not have a one-to-one
correspondence to the layers of an I/O stack. I was talking about the
layering of modules to create a response. Which would be implemented, of
course, using one or more I/O layers for each module. So yeah, that's
basically correct.
I think that would work something like this: Let's say a response starts
with the result of database access, which is then parsed through PHP, and
then sent. So the response looks something like this. This layering is
constructed by the "middle end" of the server based on the URL, server
config, whatever:
database -> PHP -> client
Each of those involve one (or more) NSPR layers. But we insert a cache
layer between each module:
database -> cache-write -> PHP -> cache-write -> client
The cache layer is write-through and just records, if appropriate, the
output from each module, along with meta-data it gets (from some other
method) about what each module layer does to the parameters of the
response (e.g., what it negotiates on). It uses that data to decides if
that layer is worth caching. If not, it just passes on the data. Either
way, it's transparent; it doesn't affect this response at all.
Anyway, next time a request comes in, the middle end consults the cache,
and figures out how much of that request is cached. It might discover that
the database access is valid for the new request, but the PHP layer
depended on User-Agent, which is different. So it constructs the new
response like this:
cache-read -> PHP -> cache-write -> client
Or maybe it's the same request with the same User-Agent, and it can pull
the whole thing out of the cache:
cache-read -> client
(this last one can be done in the kernel, or in an otherwise streamlined
manner. Much speedup. And the neat thing is that neither the database nor
mod_php have to know *anything* about it, as long as they properly report
their metadata. We'll need to make that sure that's easy for module
authors to have their modules do.)
Basically, that's how I think it should work. It's basically what we
talked about in SF (sorry you weren't there Ben, you probably would have
enjoyed it). Doesn't sound too hard to implement either, once we get the
cache and middle end parts working perfectly (hah!).
-- Alexei Kosut <ak...@stanford.edu> <http://www.stanford.edu/~akosut/>
Stanford University, Class of 2001 * Apache <http://www.apache.org> *
Re: Apache 2.0/NSPR
Posted by Ben Laurie <be...@algroup.co.uk>.
Ed Korthof wrote:
>
> On Wed, 9 Sep 1998, Ben Laurie wrote:
>
> > Dean Gaudet wrote:
> > >
> > > Please don't wait for me... everything I had in progress is now in the
> > > repository. I'd be delighted to have other folks working on apache-nspr.
> > > I'll make sure to post when I start tackling the TODOs.
> >
> > I didn't want to replicate effort with the NSPR guys - but I couldn't
> > wait, so I've been talking to them anyway - looks like I'll be sorting
> > out the working directory thing, at least. Then I was thinking of
> > looking more closely at layered I/O - what would be most valuable for me
> > is if you could make it clearer where you are at with that, before I
> > shoot us _both_ in the foot :-)
>
> I'd be happy to work on layered I/O, so long as I'm not stepping on
> anyone's feet and don't have to deal with merging in changes from 1.3.x.
> :p
>
> I haven't done anything with this since Alexei and I wrote up the proposal
> for layered I/O using modified BUFFs, as I understood that NSPR has some
> (or all) of the functionality which Apache needs there -- but if more
> should be done here, I'd be interested in helping with it.
Like I said, Dean mumbled something about flushes. Natch, if you have
threads, the approach is different to if you don't. I suppose the Big
Question is whether we are interested in implementing layered I/O for
non-threaded environments.
BTW, Alexei said some stuff about caching in between layers, which I
seem to have mislaid. Let me take this opportunity to say that sounds
cool and he should pursue it! I have some stuff to say about databases,
but I'll wait until I've drunk less.
Cheers,
Ben.
--
Ben Laurie |Phone: +44 (181) 735 0686| Apache Group member
Freelance Consultant |Fax: +44 (181) 735 0689|http://www.apache.org/
and Technical Director|Email: ben@algroup.co.uk |
A.L. Digital Ltd, |Apache-SSL author http://www.apache-ssl.org/
London, England. |"Apache: TDG" http://www.ora.com/catalog/apache/
WE'RE RECRUITING! http://www.aldigital.co.uk/
Re: Apache 2.0/NSPR
Posted by Ed Korthof <ed...@ultimanet.com>.
On Wed, 9 Sep 1998, Ben Laurie wrote:
> Dean Gaudet wrote:
> >
> > Please don't wait for me... everything I had in progress is now in the
> > repository. I'd be delighted to have other folks working on apache-nspr.
> > I'll make sure to post when I start tackling the TODOs.
>
> I didn't want to replicate effort with the NSPR guys - but I couldn't
> wait, so I've been talking to them anyway - looks like I'll be sorting
> out the working directory thing, at least. Then I was thinking of
> looking more closely at layered I/O - what would be most valuable for me
> is if you could make it clearer where you are at with that, before I
> shoot us _both_ in the foot :-)
I'd be happy to work on layered I/O, so long as I'm not stepping on
anyone's feet and don't have to deal with merging in changes from 1.3.x.
:p
I haven't done anything with this since Alexei and I wrote up the proposal
for layered I/O using modified BUFFs, as I understood that NSPR has some
(or all) of the functionality which Apache needs there -- but if more
should be done here, I'd be interested in helping with it.
Ed, reading through the references which Dean and others were helpful
enough to provide re: zero-copy TCP and page-flipping
Re: Apache 2.0/NSPR
Posted by Simon Spero <se...@mossflower.oit.unc.edu>.
On Wed, 9 Sep 1998, Alexei Kosut wrote:
> P.S. One note, though. Although some (hi, Simon) might disagree, I'm not
> talking about using Harvest as a whole, just taking out the cache bit and
> using that. As an alternative to writing our own cache. I tried that once
> before and we ended up with the mess of a proxy we have now.
I'm paying attention ; I'm not sure if I disagree or not, since the
mechanical parts of the cache are pretty generic, and could well be worth
lifting. The key thing is that the cache be write-thru, and capable of
being kernelised.
I was actually writing some stuff on this the past few nights. Of course,
I was on pain-killers so I can't read most of what I wrote. I'll post what
I can decipher
Simon // hey, I lit a candle and pretended I was at burning man.
Re: Apache 2.0/NSPR
Posted by Alexei Kosut <ak...@leland.Stanford.EDU>.
On Wed, 9 Sep 1998, Ben Laurie wrote:
> Alexei Kosut wrote:
> > By the way, I think I'm going to take advantage of some time next week
> > between when I stop working for C2Net and when I start school to play with
> > apache-nspr, and to see how the Magic Cache idea we talked about at the
> > core meeting last July might work with it. I've started looking at some
> > freely-available cache engines like Harvest, seeing if perhaps there's
> > code available somewhere we can use instead of writing our own.
>
> Do things like Harvest really help? If the "Magic Cache" ends up on disk
> like a Harvest cache doesn't that rather defeat the object?
Harvest uses a RAM cache for meta-information and "hot objects", just like
Squid. And yes, an on-disk cache does help. Obviously not for flat files,
but for most other kinds, it does. That was Cliff's point, IIRC: If you
have a file backed by a very slow database, for example, you want to be
able to cache the result of that lookup. Even if it's stored on disk,
Apache's quite good at serving files from disk.
And imagine this scenario: You're pulling a file template out of a slow
database, then filtering that output through SSI or PHP or something,
filling it in with some on-the-fly information like the time of day or
cookie-customized colors or something (I dunno; use your imagination).
Let's say the database file only changes every hour, but the on-the-fly
stuff is different for each request.
In this case, the advantages of having a cache act as the mediator between
all stages of request fulfillment become clear: The final output document
is not cacheable, so any external cache, proxy or accelerator (e.g.,
Squid) cannot help. But if Apache caches (on disk or wherever) the
layers of each response, then that database access will only occur once
every hour, instead of once per request.
*shrug* Sounds good to me. As I said, I'm planning to look into it.
Hopefully, we'll be able to figure out if the usefulness of it once we can
come up with some working code.
P.S. One note, though. Although some (hi, Simon) might disagree, I'm not
talking about using Harvest as a whole, just taking out the cache bit and
using that. As an alternative to writing our own cache. I tried that once
before and we ended up with the mess of a proxy we have now.
-- Alexei Kosut <ak...@stanford.edu> <http://www.stanford.edu/~akosut/>
Stanford University, Class of 2001 * Apache <http://www.apache.org> *
Re: Apache 2.0/NSPR
Posted by Dean Gaudet <dg...@arctic.org>.
On Thu, 10 Sep 1998, Simon Spero wrote:
> 7) TCP changes: There are several changes that would be useful for a web
> server.
You forgot:
7.n) Get rid of Nagle, and get rid of auto-flush-on-write(). Allow the
app to tell the kernel when it's at the end of what it's sending...
there's no need for silly timers and that crud like in Nagle when the app
is well behaved.
Dean
Re: Apache 2.0/NSPR
Posted by Dean Gaudet <dg...@arctic.org>.
On Sat, 12 Sep 1998, Ben Laurie wrote:
> useful thing. Also a great way to leak memory, of course :-) Did I ever
> mention that there's this great language that helps with that?
You mean C, right?
Dean
Re: Apache 2.0/NSPR
Posted by Ben Laurie <be...@algroup.co.uk>.
Dean Gaudet wrote:
>
> On Fri, 11 Sep 1998, Ben Laurie wrote:
>
> > Do we really need reference counting? Can't they have a common
> > allocation/deallocation method, and get ownership handed through the
> > layers - last one out turns off the lights (err, lowest layer releases
> > the buffer).
>
> If it's in a mmap() cache you don't ever want to give up ownership... the
> cache owns it.
Hmmm - spose we have for each memory block a handler - the lowest layer
calls the releaser in the handler - if it is a mmap() cache, the
releaser simply recycles it. Hmmm - this may be a good idea even if we
do need reference counts... actually, I'll concede reference counts, coz
even if we can avoid them in this special case, they are a generally
useful thing. Also a great way to leak memory, of course :-) Did I ever
mention that there's this great language that helps with that?
Cheers,
Ben.
--
Ben Laurie |Phone: +44 (181) 735 0686| Apache Group member
Freelance Consultant |Fax: +44 (181) 735 0689|http://www.apache.org/
and Technical Director|Email: ben@algroup.co.uk |
A.L. Digital Ltd, |Apache-SSL author http://www.apache-ssl.org/
London, England. |"Apache: TDG" http://www.ora.com/catalog/apache/
WE'RE RECRUITING! http://www.aldigital.co.uk/
Re: Apache 2.0/NSPR
Posted by Dean Gaudet <dg...@arctic.org>.
On Fri, 11 Sep 1998, Ben Laurie wrote:
> Do we really need reference counting? Can't they have a common
> allocation/deallocation method, and get ownership handed through the
> layers - last one out turns off the lights (err, lowest layer releases
> the buffer).
If it's in a mmap() cache you don't ever want to give up ownership... the
cache owns it.
> Are you saying that all interlayer comms should be in terms of URIs? And
> qualifying headers? (I'm thinking this integrated neatly with Magic
> Cache).
Yeah I think that's what I'm saying. At least it sounds good :)
Dean
Re: Apache 2.0/NSPR
Posted by Ben Laurie <be...@algroup.co.uk>.
Dean Gaudet wrote:
>
> On Thu, 10 Sep 1998, Simon Spero wrote:
>
> > 2) Layering is dangerous to performance, and should be collapsed as much
> > as possible. This is part of the job of the middle end.
>
> This was part of why I was prompted to say we should look at zero-copy,
> "page based" i/o... in particular, the chunking layer doesn't need to
> modify anything. It just needs to build iovecs. But without reference
> counting on the buffers we have to actually copy the contents rather than
> just pass them around...
Do we really need reference counting? Can't they have a common
allocation/deallocation method, and get ownership handed through the
layers - last one out turns off the lights (err, lowest layer releases
the buffer).
> > 4) A lot of the cache could be made kernel resident. This means that
> > although cache invalidation can be complicated (objects can have a
> > validate method), simple tests should be handlable simply and predictably-
> > for example file modification date, eq checks for negotiation etc.
> > Otherwise you have to leave the kernel to validate.
>
> For some sites it'd be sufficient to invalidate the entire cache on a
> regular basis. That's pretty easy to do. But yeah invalidation is in
> general a painful problem...
>
> > 5) The cache allows more objects than normal to be treated as files (i.e.
> > they have finite length, rather than being data streams). This makes it a
> > lot easier to attach file system protocol front ends to the middle-end
> > namespace (in particular code from Samba and the old user mode NFS
> > server). This allows the server to mediate *all* access to the data store,
> > making it easy to use *active* invalidation, with no validation checks at
> > all in the fast path.
>
> Hmmm... interesting.
>
> On a related note, we need to abstract those bits of the filesystem which
> we need to serve HTTP so that backing store doesn't have to be a
> filesystem. I'd say this should even go as far as modules being able to
> open() a URI and read from it -- so that it can be as transparent as
> possible. So rather than use ap_bopenf() directly (the apache-nspr
> equivalent of fopen()), modules open a URI through some other interface,
> and that may go to the filesystem or go to a database/whatever.
>
> A difficulty in this is access control -- there's a difference between an
> external request requesting some files, and an internal module requesting
> them. Different rights.
Are you saying that all interlayer comms should be in terms of URIs? And
qualifying headers? (I'm thinking this integrated neatly with Magic
Cache).
> > 6) This implies that the namespace model should be mappable in terms of
> > directories, files, and specials (cgi-scripts, etc). This gives the
> > hierarchical component of the resolution process a higher priority than
> > the other phases.
>
> I'd like to see the namespace have "mount points" somewhat like the unix
> filesystem. This controls the hierarchy as far as what the underlying
> store is... and it's a simple system, easy to optimize for. i.e. I'd
> really like to avoid "if the URI matches this regex then it's served by
> database foobar". That's far too general.
Sounds like a plan.
Cheers,
Ben.
--
Ben Laurie |Phone: +44 (181) 735 0686| Apache Group member
Freelance Consultant |Fax: +44 (181) 735 0689|http://www.apache.org/
and Technical Director|Email: ben@algroup.co.uk |
A.L. Digital Ltd, |Apache-SSL author http://www.apache-ssl.org/
London, England. |"Apache: TDG" http://www.ora.com/catalog/apache/
WE'RE RECRUITING! http://www.aldigital.co.uk/
Re: Apache 2.0/NSPR
Posted by Manoj Kasichainula <ma...@io.com>.
On Fri, Sep 11, 1998 at 09:44:38AM -0700, Alexei Kosut wrote:
> 3. AFS-like "I'll tell you when it's changed" invalidation. I'm not quite
> sure how we'd make this one work. Probably something to do with
> threads. But it's probably the best way to ensure that the server
> nevers delivers out-of-date content for an arbitrarily changing object
Does AFS export its callback APIs to userland?
For other filesystems, having kernel help for cache invalidation would
make sense when we have in-kernel caching.
--
Manoj Kasichainula - manojk at io dot com - http://www.io.com/~manojk/
"Even on radio, their new logo sucks" -- Aaron Dickey concerning the BBC World
Service, 1997-11-22
Re: Apache 2.0/NSPR
Posted by Ben Laurie <be...@algroup.co.uk>.
Alexei Kosut wrote:
> On Fri, 11 Sep 1998, Dean Gaudet wrote:
> > I'd like to see the namespace have "mount points" somewhat like the unix
> > filesystem. This controls the hierarchy as far as what the underlying
> > store is... and it's a simple system, easy to optimize for. i.e. I'd
> > really like to avoid "if the URI matches this regex then it's served by
> > database foobar". That's far too general.
>
> I'm sure you would like to see that, Dean. :) I doubt it will happen,
> though. Users like to be able to do odd things with the URLs they type in.
> And because URIs and URNs never took off, we're stuck with
> resource-specific URLs, like http://foobar/file.html. If you have a
> widely-publicized file like that, and suddenly decided you want to
> exchange it for a database-backed file, it's hard to convince everyone,
> using ESP perhaps, to instead go to http://foobar/database/notafile.html.
>
> I think we're going to have to add generic, regex-like matching mechanisms
> at some level of the URI parsing code. It may be at a higher level than it
> is right now (i.e., perhaps URI rewriting from external URIs to internal
> not-quite-URIs), but it'll still have to be there.
I thought that was what he said? If he didn't he should've - that is,
strict "mount-point" based URIs are only internal - external URL
rewriting is as whacky as it always has been...
Cheers,
Ben.
--
Ben Laurie |Phone: +44 (181) 735 0686| Apache Group member
Freelance Consultant |Fax: +44 (181) 735 0689|http://www.apache.org/
and Technical Director|Email: ben@algroup.co.uk |
A.L. Digital Ltd, |Apache-SSL author http://www.apache-ssl.org/
London, England. |"Apache: TDG" http://www.ora.com/catalog/apache/
WE'RE RECRUITING! http://www.aldigital.co.uk/
Re: Apache 2.0/NSPR
Posted by Dean Gaudet <dg...@arctic.org>.
On Fri, 11 Sep 1998, Alexei Kosut wrote:
> I think we're going to have to add generic, regex-like matching mechanisms
> at some level of the URI parsing code. It may be at a higher level than it
> is right now (i.e., perhaps URI rewriting from external URIs to internal
> not-quite-URIs), but it'll still have to be there.
mount a "regex remapping backing store" on /. At the lowest level I'd
like the simplicity of mount points, rather than the complexity of regular
expressions.
Besides, if people are doing a reorg from the filesystem to a database
they'll probably end up changing URLs... unless you were implying that
/file.html was in a database already, in which case just mount the
database on /.
Dean
Re: Apache 2.0/NSPR
Posted by Alexei Kosut <ak...@leland.Stanford.EDU>.
On Fri, 11 Sep 1998, Dean Gaudet wrote:
> > 4) A lot of the cache could be made kernel resident. This means that
> > although cache invalidation can be complicated (objects can have a
> > validate method), simple tests should be handlable simply and predictably-
> > for example file modification date, eq checks for negotiation etc.
> > Otherwise you have to leave the kernel to validate.
>
> For some sites it'd be sufficient to invalidate the entire cache on a
> regular basis. That's pretty easy to do. But yeah invalidation is in
> general a painful problem...
Yep... Hey; I've got a brilliant idea. Let's redefine some terms. Apache
only serves valid, up-to-date data. Therefore, anything Apache isn't
serving must be invalid or old. So we don't need to expire anything,
because Apache is perfect :)
Probably what we need are some generic cache-invalidation methods. I can
think of three:
1. Algorithms, based on object metainformation. i.e., like the Expires
header (when the date becomes later, it's invalid), though you could go
a bit more complex than that.
2. Invalidation callbacks, a la HTTP's If-Modified-Since; a function to
call to determine if the item is still valid. Some content
will require this, although it kills any chance of doing real fast
kernel-style delivery of the cached item.
3. AFS-like "I'll tell you when it's changed" invalidation. I'm not quite
sure how we'd make this one work. Probably something to do with
threads. But it's probably the best way to ensure that the server
nevers delivers out-of-date content for an arbitrarily changing object
[...]
> On a related note, we need to abstract those bits of the filesystem which
> we need to serve HTTP so that backing store doesn't have to be a
> filesystem. I'd say this should even go as far as modules being able to
> open() a URI and read from it -- so that it can be as transparent as
> possible. So rather than use ap_bopenf() directly (the apache-nspr
> equivalent of fopen()), modules open a URI through some other interface,
> and that may go to the filesystem or go to a database/whatever.
Yes! That's exactly what I was thinking last night. This not only gives us
the advantage of allowing other backing stores, it means all of those
objects are cached, and also that you can apply filters to them, just like
any file.
This brings up some interesting possiblities, if every file is treated
like this, even, say, the config files. Because you wouldn't need any
special hooks or *anything* special to enable, say, a macro language, or
even a completely different, GUI-based config format. You just configure
the server (I'll leave out the obvious chicken-vs-egg question for now) to
pass the config file URIs through a module that does whatever you want.
Neat.
> A difficulty in this is access control -- there's a difference between an
> external request requesting some files, and an internal module requesting
> them. Different rights.
Hmm. Hadn't really thought of that. What we really need to do, as Simon
alluded, is apply proper file system semantics to the URI space. With
classes (groups) of users and varying access privilidges.
Of course, Microsoft's already thought of this. But instead of building a
filesystem-like access control and object retrieval system into IIS, they
just modified the NT filesystem. I suggest we not do that.
> > 6) This implies that the namespace model should be mappable in terms of
> > directories, files, and specials (cgi-scripts, etc). This gives the
> > hierarchical component of the resolution process a higher priority than
> > the other phases.
>
> I'd like to see the namespace have "mount points" somewhat like the unix
> filesystem. This controls the hierarchy as far as what the underlying
> store is... and it's a simple system, easy to optimize for. i.e. I'd
> really like to avoid "if the URI matches this regex then it's served by
> database foobar". That's far too general.
I'm sure you would like to see that, Dean. :) I doubt it will happen,
though. Users like to be able to do odd things with the URLs they type in.
And because URIs and URNs never took off, we're stuck with
resource-specific URLs, like http://foobar/file.html. If you have a
widely-publicized file like that, and suddenly decided you want to
exchange it for a database-backed file, it's hard to convince everyone,
using ESP perhaps, to instead go to http://foobar/database/notafile.html.
I think we're going to have to add generic, regex-like matching mechanisms
at some level of the URI parsing code. It may be at a higher level than it
is right now (i.e., perhaps URI rewriting from external URIs to internal
not-quite-URIs), but it'll still have to be there.
-- Alexei Kosut <ak...@stanford.edu> <http://www.stanford.edu/~akosut/>
Stanford University, Class of 2001 * Apache <http://www.apache.org> *
Re: Apache 2.0/NSPR
Posted by Dean Gaudet <dg...@arctic.org>.
On Thu, 10 Sep 1998, Simon Spero wrote:
> 2) Layering is dangerous to performance, and should be collapsed as much
> as possible. This is part of the job of the middle end.
This was part of why I was prompted to say we should look at zero-copy,
"page based" i/o... in particular, the chunking layer doesn't need to
modify anything. It just needs to build iovecs. But without reference
counting on the buffers we have to actually copy the contents rather than
just pass them around...
> 4) A lot of the cache could be made kernel resident. This means that
> although cache invalidation can be complicated (objects can have a
> validate method), simple tests should be handlable simply and predictably-
> for example file modification date, eq checks for negotiation etc.
> Otherwise you have to leave the kernel to validate.
For some sites it'd be sufficient to invalidate the entire cache on a
regular basis. That's pretty easy to do. But yeah invalidation is in
general a painful problem...
> 5) The cache allows more objects than normal to be treated as files (i.e.
> they have finite length, rather than being data streams). This makes it a
> lot easier to attach file system protocol front ends to the middle-end
> namespace (in particular code from Samba and the old user mode NFS
> server). This allows the server to mediate *all* access to the data store,
> making it easy to use *active* invalidation, with no validation checks at
> all in the fast path.
Hmmm... interesting.
On a related note, we need to abstract those bits of the filesystem which
we need to serve HTTP so that backing store doesn't have to be a
filesystem. I'd say this should even go as far as modules being able to
open() a URI and read from it -- so that it can be as transparent as
possible. So rather than use ap_bopenf() directly (the apache-nspr
equivalent of fopen()), modules open a URI through some other interface,
and that may go to the filesystem or go to a database/whatever.
A difficulty in this is access control -- there's a difference between an
external request requesting some files, and an internal module requesting
them. Different rights.
> 6) This implies that the namespace model should be mappable in terms of
> directories, files, and specials (cgi-scripts, etc). This gives the
> hierarchical component of the resolution process a higher priority than
> the other phases.
I'd like to see the namespace have "mount points" somewhat like the unix
filesystem. This controls the hierarchy as far as what the underlying
store is... and it's a simple system, easy to optimize for. i.e. I'd
really like to avoid "if the URI matches this regex then it's served by
database foobar". That's far too general.
On the other hand, I'm happy if filters are able to do "wildcarding" like
that. But I think it's wrong for them to do it based on the URI --
instead they should be based on content-types, or other (abstract)
attributes. For the filesystem underlying store we'll still derive
content-type from the filename (well, on the Mac or OS/2 I could see it
going into the metadata). But from the point of view of the server,
that's an implementation detail in a lower level backing store.
> 7) TCP changes: There are several changes that would be useful for a web
> server.
> 7.3) Use Zero-copy w/ page flipping. Especially if you have a
> kernel cache.
It's so nice that yummy hardware capable of doing this will be commodity
on PCs in a year or two... gigabit ether cards have all the on-card smarts
to do zero-copy.
> 7.4) Allow connect_and_write (send data with syn)
> 7.5) Allow accept_and_read( read data with syn, delay syn-ack).
I disagree... but only because I'm a linuxhead and syscalls are cheap ;)
Although there are a lot of similar optimizations that I would still like
to see -- specifically in the way of socket/fd options. It's annoying
that every new connection has to be set to no-nagle, and needs to be set
non-blocking. That stuff should be inheritable, or be part of the
accept().
accept() is somewhat of a poor child compared to open() ... open() gets to
have flags associated with the FD for free.
Proxies want the same treatment for socket().
Plus there's a need to handle closing thousands of FDs on a
fork()/exec()... unix FDs just weren't designed for big threaded servers
with thousands of FDs. You either pay at every socket creation, or you
pay hard at each fork()... The problem is that fork() isn't the primitive
we want -- we want CreateProcess().
How many programs actually *use* fork() ? Sure Apache 1.x does. In some
sense so does inetd and related ilk. But they don't have to. They'd be
much better off with threads and CreateProcess(). NT gets this right.
Anyhow I had a proposal for this which adds a new set of "extended" file
handles... which have the exact semantics that threaded servers want.
Essentially they're shared (not copied!) across fork()/clone() but not
across exec(). So a CreateProcess() amounts to a fork(), a few dup2()s,
and an exec(), reducing the overhead.
(Why do I care about CGIs? 'cause they're something that gets
benchmarked...)
Dean
Re: Apache 2.0/NSPR
Posted by Simon Spero <se...@mossflower.oit.unc.edu>.
On Wed, 9 Sep 1998, Ben Laurie wrote:
> Marc Slemko wrote:
> > On Wed, 9 Sep 1998, Ben Laurie wrote:
> > > Marc Slemko wrote:
> > > > Memory and disk are the same thing.
> > > Bollocks. :-)
> > No, honest. Especially with modern vm systems, the most efficient way for
> > your application to differentiate can sometimes be to not.
> That I can agree with, so long as we remember the "sometimes". However,
> sometimes it isn't the most efficient. Certainly a pointer is quicker
[just wanted to keep the bollocks. A few disparate points - I screwed up
my linux box and scattered my mail all over the place.]
1) In the situation we're talking about, disk is most definitely not
memory. The cache subsystem is part of the server kernel- the server has a
far better handle on page replacement policies than BSD or Linux. The
memory cache should be locked; otherwise you can't tune performance at
all. [And this is coming from me, an object database freak]
2) Layering is dangerous to performance, and should be collapsed as much
as possible. This is part of the job of the middle end.
3) All object access is mediated by the cache. Thus, any sub objects
referenced by another object are cached. (For example, part of the
namespace might wrap objects with headers and footers. The header, footer,
page body and end result would all be cachable).
4) A lot of the cache could be made kernel resident. This means that
although cache invalidation can be complicated (objects can have a
validate method), simple tests should be handlable simply and predictably-
for example file modification date, eq checks for negotiation etc.
Otherwise you have to leave the kernel to validate.
5) The cache allows more objects than normal to be treated as files (i.e.
they have finite length, rather than being data streams). This makes it a
lot easier to attach file system protocol front ends to the middle-end
namespace (in particular code from Samba and the old user mode NFS
server). This allows the server to mediate *all* access to the data store,
making it easy to use *active* invalidation, with no validation checks at
all in the fast path.
6) This implies that the namespace model should be mappable in terms of
directories, files, and specials (cgi-scripts, etc). This gives the
hierarchical component of the resolution process a higher priority than
the other phases.
7) TCP changes: There are several changes that would be useful for a web
server.
7.1)The most important of these is probably sharing congestion
window information between connections to the same host. This
gives you a lot of the performance win and anti-MARCA congestion
avoidance of SCP/HTTP-NG. Someone has done this for BSD
(see end2end )
7.2) Use shared send window space (you need 1 BDP total for all
the sub-connections. This gives you the memory performance of
SCP/HTTP-NG (lets you use honking big buffers iff you have that
many packets in flight; otherwise, throttle back the server. Not
known to be implemented ATM (see end2end).
7.3) Use Zero-copy w/ page flipping. Especially if you have a
kernel cache.
7.4) Allow connect_and_write (send data with syn)
7.5) Allow accept_and_read( read data with syn, delay syn-ack).
7.6) Use ultra-aggressive tcb search.
There's more, but I'm sleepy and sick.
Simon
Re: Apache 2.0/NSPR
Posted by Ben Laurie <be...@algroup.co.uk>.
Marc Slemko wrote:
>
> On Wed, 9 Sep 1998, Ben Laurie wrote:
>
> > Marc Slemko wrote:
> > > Memory and disk are the same thing.
> >
> > Bollocks. :-)
>
> No, honest. Especially with modern vm systems, the most efficient way for
> your application to differentiate can sometimes be to not.
That I can agree with, so long as we remember the "sometimes". However,
sometimes it isn't the most efficient. Certainly a pointer is quicker
than an absolute pathname, even if the thing the pointer points at got
swapped. You know this. I know this. There are young minds out there.
Let's not corrupt them. :-)
Cheers,
Ben.
--
Ben Laurie |Phone: +44 (181) 735 0686| Apache Group member
Freelance Consultant |Fax: +44 (181) 735 0689|http://www.apache.org/
and Technical Director|Email: ben@algroup.co.uk |
A.L. Digital Ltd, |Apache-SSL author http://www.apache-ssl.org/
London, England. |"Apache: TDG" http://www.ora.com/catalog/apache/
WE'RE RECRUITING! http://www.aldigital.co.uk/
Re: Apache 2.0/NSPR
Posted by Marc Slemko <ma...@znep.com>.
On Wed, 9 Sep 1998, Ben Laurie wrote:
> Marc Slemko wrote:
> >
> > On Wed, 9 Sep 1998, Ben Laurie wrote:
> >
> > > Alexei Kosut wrote:
> > > > By the way, I think I'm going to take advantage of some time next week
> > > > between when I stop working for C2Net and when I start school to play with
> > > > apache-nspr, and to see how the Magic Cache idea we talked about at the
> > > > core meeting last July might work with it. I've started looking at some
> > > > freely-available cache engines like Harvest, seeing if perhaps there's
> > > > code available somewhere we can use instead of writing our own.
> > >
> > > Do things like Harvest really help? If the "Magic Cache" ends up on disk
> > > like a Harvest cache doesn't that rather defeat the object?
> >
> > Memory and disk are the same thing.
>
> Bollocks. :-)
No, honest. Especially with modern vm systems, the most efficient way for
your application to differentiate can sometimes be to not.
>
> > Heck, even FreeBSD's malloc() was designed for a "disk" first:
> >
> > The present allocation implementation started out as a filesystem for a
> > drum attached to a 20bit binary challenged computer which was built with
> > discrete germanium transistors. It has since graduated to handle primary
> > storage rather than secondary. It first appeared in its new shape and
> > ability in FreeBSD release 2.2.
> >
> > <g>
>
> I've got this hammer. It was designed for nails. Shall we see what it
> does to fingers? :-)
It works fine, you just have to sharpen the fingers first or else they
don't go in very well and tend to bend a bit eaily.
Re: Apache 2.0/NSPR
Posted by Ben Laurie <be...@algroup.co.uk>.
Marc Slemko wrote:
>
> On Wed, 9 Sep 1998, Ben Laurie wrote:
>
> > Alexei Kosut wrote:
> > > By the way, I think I'm going to take advantage of some time next week
> > > between when I stop working for C2Net and when I start school to play with
> > > apache-nspr, and to see how the Magic Cache idea we talked about at the
> > > core meeting last July might work with it. I've started looking at some
> > > freely-available cache engines like Harvest, seeing if perhaps there's
> > > code available somewhere we can use instead of writing our own.
> >
> > Do things like Harvest really help? If the "Magic Cache" ends up on disk
> > like a Harvest cache doesn't that rather defeat the object?
>
> Memory and disk are the same thing.
Bollocks. :-)
> Heck, even FreeBSD's malloc() was designed for a "disk" first:
>
> The present allocation implementation started out as a filesystem for a
> drum attached to a 20bit binary challenged computer which was built with
> discrete germanium transistors. It has since graduated to handle primary
> storage rather than secondary. It first appeared in its new shape and
> ability in FreeBSD release 2.2.
>
> <g>
I've got this hammer. It was designed for nails. Shall we see what it
does to fingers? :-)
Cheers,
Ben.
--
Ben Laurie |Phone: +44 (181) 735 0686| Apache Group member
Freelance Consultant |Fax: +44 (181) 735 0689|http://www.apache.org/
and Technical Director|Email: ben@algroup.co.uk |
A.L. Digital Ltd, |Apache-SSL author http://www.apache-ssl.org/
London, England. |"Apache: TDG" http://www.ora.com/catalog/apache/
WE'RE RECRUITING! http://www.aldigital.co.uk/
Re: Apache 2.0/NSPR
Posted by Marc Slemko <ma...@worldgate.com>.
On Wed, 9 Sep 1998, Ben Laurie wrote:
> Alexei Kosut wrote:
> > By the way, I think I'm going to take advantage of some time next week
> > between when I stop working for C2Net and when I start school to play with
> > apache-nspr, and to see how the Magic Cache idea we talked about at the
> > core meeting last July might work with it. I've started looking at some
> > freely-available cache engines like Harvest, seeing if perhaps there's
> > code available somewhere we can use instead of writing our own.
>
> Do things like Harvest really help? If the "Magic Cache" ends up on disk
> like a Harvest cache doesn't that rather defeat the object?
Memory and disk are the same thing.
Heck, even FreeBSD's malloc() was designed for a "disk" first:
The present allocation implementation started out as a filesystem for a
drum attached to a 20bit binary challenged computer which was built with
discrete germanium transistors. It has since graduated to handle primary
storage rather than secondary. It first appeared in its new shape and
ability in FreeBSD release 2.2.
<g>
Re: Apache 2.0/NSPR
Posted by Ben Laurie <be...@algroup.co.uk>.
Alexei Kosut wrote:
> By the way, I think I'm going to take advantage of some time next week
> between when I stop working for C2Net and when I start school to play with
> apache-nspr, and to see how the Magic Cache idea we talked about at the
> core meeting last July might work with it. I've started looking at some
> freely-available cache engines like Harvest, seeing if perhaps there's
> code available somewhere we can use instead of writing our own.
Do things like Harvest really help? If the "Magic Cache" ends up on disk
like a Harvest cache doesn't that rather defeat the object?
Cheers,
Ben.
--
Ben Laurie |Phone: +44 (181) 735 0686| Apache Group member
Freelance Consultant |Fax: +44 (181) 735 0689|http://www.apache.org/
and Technical Director|Email: ben@algroup.co.uk |
A.L. Digital Ltd, |Apache-SSL author http://www.apache-ssl.org/
London, England. |"Apache: TDG" http://www.ora.com/catalog/apache/
WE'RE RECRUITING! http://www.aldigital.co.uk/
Re: Apache 2.0/NSPR
Posted by Alexei Kosut <ak...@leland.Stanford.EDU>.
On Wed, 9 Sep 1998, Ben Laurie wrote:
> Dean Gaudet wrote:
> >
> > Please don't wait for me... everything I had in progress is now in the
> > repository. I'd be delighted to have other folks working on apache-nspr.
> > I'll make sure to post when I start tackling the TODOs.
>
> I didn't want to replicate effort with the NSPR guys - but I couldn't
> wait, so I've been talking to them anyway - looks like I'll be sorting
> out the working directory thing, at least. Then I was thinking of
> looking more closely at layered I/O - what would be most valuable for me
> is if you could make it clearer where you are at with that, before I
> shoot us _both_ in the foot :-)
Go on! Shoot everybody in the foot. Then, while we're all hobbling about,
you can secretly switch in the C++ version of Apache you've got hiding in
your back pocket :)
By the way, I think I'm going to take advantage of some time next week
between when I stop working for C2Net and when I start school to play with
apache-nspr, and to see how the Magic Cache idea we talked about at the
core meeting last July might work with it. I've started looking at some
freely-available cache engines like Harvest, seeing if perhaps there's
code available somewhere we can use instead of writing our own.
(Unfortunately, my first choice for code, Squid, isn't usable, because
it's under the GNU GPL. But I won't open that ball of wax right now. Maybe
tomorrow *grin*).
-- Alexei Kosut <ak...@stanford.edu> <http://www.stanford.edu/~akosut/>
Stanford University, Class of 2001 * Apache <http://www.apache.org> *
Re: Apache 2.0/NSPR
Posted by Ben Laurie <be...@algroup.co.uk>.
Dean Gaudet wrote:
>
> Please don't wait for me... everything I had in progress is now in the
> repository. I'd be delighted to have other folks working on apache-nspr.
> I'll make sure to post when I start tackling the TODOs.
I didn't want to replicate effort with the NSPR guys - but I couldn't
wait, so I've been talking to them anyway - looks like I'll be sorting
out the working directory thing, at least. Then I was thinking of
looking more closely at layered I/O - what would be most valuable for me
is if you could make it clearer where you are at with that, before I
shoot us _both_ in the foot :-)
>
> I still haven't read my mass of apache-cvs mail so I don't know if you did
> a merge up to the latest apache-1.3.2-dev code base... (and I'm guessing
> Ken has been going around doing yet more code cleanups based on some
> new-httpd messages I've read so this is going to be another fucking pain
> in the ass merge). If you want to leave that to me, that's fine... I'll
> try to get to it in the next week or so.
I haven't done the merge, and I'm more than happy to leave it to you - I
did fix a small cockup in 2.0 and backport to 1.3, but that shouldn't
bite you too badly.
Before it confuses you, I should mention I moved your Configuration to
Configuration.dean to avoid a nasty fight :-)
Cheers,
Ben.
--
Ben Laurie |Phone: +44 (181) 735 0686| Apache Group member
Freelance Consultant |Fax: +44 (181) 735 0689|http://www.apache.org/
and Technical Director|Email: ben@algroup.co.uk |
A.L. Digital Ltd, |Apache-SSL author http://www.apache-ssl.org/
London, England. |"Apache: TDG" http://www.ora.com/catalog/apache/
WE'RE RECRUITING! http://www.aldigital.co.uk/
Re: Apache 2.0/NSPR
Posted by Dean Gaudet <dg...@arctic.org>.
Please don't wait for me... everything I had in progress is now in the
repository. I'd be delighted to have other folks working on apache-nspr.
I'll make sure to post when I start tackling the TODOs.
I still haven't read my mass of apache-cvs mail so I don't know if you did
a merge up to the latest apache-1.3.2-dev code base... (and I'm guessing
Ken has been going around doing yet more code cleanups based on some
new-httpd messages I've read so this is going to be another fucking pain
in the ass merge). If you want to leave that to me, that's fine... I'll
try to get to it in the next week or so.
Dean
On Sun, 6 Sep 1998, Ben Laurie wrote:
> As some of you will have noticed, I was playing with Apache 2.0 (nice
> work, Dean). So now I'm wondering what the status of NSPR is - I know
> Dean was after some changes, and the one that is bugging me is the
> working directory of subprocesses - I know Dean was talking to the NSPR
> team, but what's the state of play? Anyone know? Or do I wait for Dean?
>
> Cheers,
>
> Ben.
>
> --
> Ben Laurie |Phone: +44 (181) 735 0686| Apache Group member
> Freelance Consultant |Fax: +44 (181) 735 0689|http://www.apache.org/
> and Technical Director|Email: ben@algroup.co.uk |
> A.L. Digital Ltd, |Apache-SSL author http://www.apache-ssl.org/
> London, England. |"Apache: TDG" http://www.ora.com/catalog/apache/
>
> WE'RE RECRUITING! http://www.aldigital.co.uk/
>