You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by Paul A Houle <ph...@cornell.edu> on 2005/08/22 19:07:09 UTC
httpd has no reason to improve...
Ben Collins-Sussman wrote:
> I see a lot of frustration going on. The thing is, httpd's
> development process is nearly identical to Subversion's process... we
> stole most of it from you folks! So why all the angst in httpd-land,
> but not in Subversion-land?
It's really a lack of direction.
Apache already is the market leader in the web server space and has
no potential for further growth -- Windows users aren't going to switch
to Apache in significant numbers. (They never were, so developing a
good NT port was a waste of resources that has helped paralyze the
development of Apache for it's core users.) This in contrast to svn,
which has a long way to go in growth.
There is very little going on in Apache to make it be a better web
server -- the frontiers of development are trying to make it an ftp
server, an smtp server, a ntp server, a snpd server, a bgp
server... Efforts that 99.44% of httpd users are completely indifferent
to (Ok, maybe a few of them will want to turn on an ftp server because
they're not getting enough support calls from people who are behind NAT
or firewalls.)
The module mechanism means that significant improvements in Apache
can be made without making changes to the Apache distribution -- people
are who are serious about improving Apache as a web server are taking
this route and not adding things to the Apache distribution. (Any
change which affects the core use of Apache is politically controversial
and gets an automatic -1.)
Pluggable MPMs sound like a good idea in theory, but in practice
they're a failure. It's hard enough writing code that runs in a single
concurrent environment, but impossible if the concurrency environment
is unknown and unknowable. It's hard enough keeping the modules that
are part of the distribution working under all MPMs, even harder to
support third-party modules (mod_php) and even harder to support all the
libraries that link against third-party modules (kiss of death for
mod_php and mod_perl in worker.)
-----------
People who want to wake Apache up would be best to try a fork.
Throw things that are irrelevant to running httpd on Posix overboard.
Pick ONE mpm that offers a significant advance in functionality
(mod_event or mod_perchild that actually works) and qualify everything
to run against it. Add mod_macro.
But why? Apache is good enough for most people, and you can fix
any real problems for a particular project by adding modules or making
little source code patches. There are 1,000 or so sites in the world
that need an MPM faster than prefork, and they'd be better off going to
a cluster strategy for availability anyway.
Apache 2 was a rough ride, but Apache 2.0.54 runs smooth on every
installation I have... Why change anything?
Re: httpd has no reason to improve...
Posted by Nick Kew <ni...@webthing.com>.
Paul A Houle wrote:
> Ben Collins-Sussman wrote:
>
>> I see a lot of frustration going on. The thing is, httpd's
>> development process is nearly identical to Subversion's process... we
>> stole most of it from you folks! So why all the angst in httpd-land,
>> but not in Subversion-land?
Too many versions. And conflicting views on how to deal with them.
> It's really a lack of direction.
Lack of what?
Support for major up-and-coming technologies like LDAP?
A working cacheing framework?
The biggest improvement in database management since mod_perl
pioneered the LAMP architecture ten years ago?
Untying authentication protocols from provider implementations?
I could go on, but those seem like worthwhile directions.
All happening because people want them.
> There is very little going on in Apache to make it be a better web
> server
Nonsense.
-- the frontiers of development are trying to make it an ftp
> server, an smtp server, a ntp server, a snpd server, a bgp
Erm, I know we're doing SMTP: that's to harness the input filter chain
for spam filtering. FTP has been around for some time. Where are all
those other protocols?
> The module mechanism means that significant improvements in Apache
> can be made without making changes to the Apache distribution -- people
> are who are serious about improving Apache as a web server are taking
> this route and not adding things to the Apache distribution. (Any
Indeed, we can do a lot with modules. A number of us who are now core
developers started out as application developers writing modules. The
more modules you write, the more you see how the core product can be
improved, the more you contribute. Well, that's (broadly speaking) how
I came to Apache, and I'm not the only one.
Before 2.0 - specifically filters - Apache didn't cut it for
applications. The payback for writing modules (as opposed to
apps running under CGI, mod_perl, etc) wasn't worth the effort
from my PoV.
> People who want to wake Apache up would be best to try a fork. Throw
> things that are irrelevant to running httpd on Posix overboard. Pick
> ONE mpm that offers a significant advance in functionality (mod_event or
> mod_perchild that actually works) and qualify everything to run against
> it. Add mod_macro.
Feel free to do exactly that. I don't find it a problem to write
modules that work across platforms and MPMs, and I value the fact that
modules work cross-platform without hair-tearing (and with less trouble
and overhead than many cross-platform environments like Java).
> But why? Apache is good enough for most people, and you can fix any
> real problems for a particular project by adding modules or making
> little source code patches. There are 1,000 or so sites in the world
> that need an MPM faster than prefork, and they'd be better off going to
> a cluster strategy for availability anyway.
What's "faster than" got to do with it? You want a justification for
threaded MPMs, just look at connection pooling apps, and the first
big advance in scalability since LAMP.
> Apache 2 was a rough ride, but Apache 2.0.54 runs smooth on every
> installation I have... Why change anything?
I hope you don't run those new-fangled gadgets like a mouse?
--
Nick Kew
Re: httpd has no reason to improve...
Posted by Brian Akins <br...@turner.com>.
Paul A Houle wrote:
>
>
> I don't see end users clamoring for mod_ftp, or mod_snmpd. What's
> the point of writing a squid replacement unless you can actually make
> something better?
That's why I have been posting so many cache patches.
--
Brian Akins
Lead Systems Engineer
CNN Internet Technologies
Re: httpd has no reason to improve...
Posted by Joe Schaefer <jo...@sunstarsys.com>.
Paul A Houle <ph...@cornell.edu> writes:
> I don't see excellence coming from "swiss army knife" frameworks
> that do everything, but from systems that are developed from a whole
> system viewpoint, that have a good amount of codesign between layers of
> the system -- if you build a system that lets you plug in arbitrary
> junk, you're going to get arbitrary performance and reliability.
I don't think the ASF infrastructure team feels that way
about qpsmtpd + mod_perl. Without mod_perl, qpsmtpd simply
couldn't handle our load. With mod_perl, it handles the load
rather easily. The glue code to hook qpsmtpd into mod_perl is
rather small.
I consider that a remarkable real-world success of the 2.x
architecture, warts and all.
[...]
> Were Apache development targeting real problems that real users
> have, I think things would be quite different.
My point remains this: we target power users, because those are
the people that contribute to the project. Mass-virtual-hosting
ISP's rarely have anything to contribute here; yet they form the
dominant portion of our "market share".
>
> A big part of the problem is that the Apache project has settled
> into a local equilibrium -- this explains the paradox of a product that
> obviously satisfies end user's needs well (no competition has emerged)
> but has a moribund development process. Any real innovation in the web
> server space will need to be disruptive, to break things. Apache, as
> we know it, just can't do that.
As an apreq developer, I couldn't disagree more. 2.0 solved our
real-world problems that *our* users face, because it allows better
modularity. No longer do apreq users have to use apreq consistently
throughout their codebase in order to reap its benefits.
--
Joe Schaefer
Re: httpd has no reason to improve...
Posted by phildietz <Ph...@cox.net>.
My 2 cents says the mod_cgi in 1.3 is better than the 2.0 series...
Custom changes to mod_cgi are much easier to perform in the 1.3 code base.
It wasnt until 2.0.54 or whatever until the stderr deadlocks were fixed...
and I'm still uncertain if mod_cgi and mod_cgid perform identical with
stderr present before or after a Content-Type: text/html header
...
Re: httpd has no reason to improve...
Posted by r....@t-online.de.
Paul A Houle wrote:
[..cut..]
> very little to offer end users. A few people thought it would be great
> to have pluggable MPM's, and a few other people introduced half-baked
> systems such as mod_cache and filters. You know a tree by its fruit,
What is half-baked about filters? Without them many of the things that also
3rd party modules use would not be possible or much harder to do. You are right
one of the good thing is that you can do many things with modules in httpd
but this requires a good and flexible framework and API which can be always
improved.
mod_cache is experimental in 2.0.x and has improved much in the trunk. It is much more
any caching (reverse) proxy could do as it is more of a caching framework. If you
would have taken a look at the background of the people who submitted many of the mod_cache
patches during recent times you would have noticed that these are people which
use mod_cache at big sites and are real power users of httpd.
> and the fruit I care about is performance and reliability. Apache
> 2.0.54 is a great server, but the fact that it took 54 revisions to get
> there isn't a good indication about the design and development process.
>
Version numbers are cheap and say nothing here. 2.0.35 was the first GA release,
and thus 2.0.54 is not that high as it seems.
Regards
Rüdiger
Re: httpd has no reason to improve...
Posted by Garrett Rooney <ro...@electricjellyfish.net>.
Paul A Houle wrote:
> I don't see end users clamoring for mod_ftp, or mod_snmpd. What's
> the point of writing a squid replacement unless you can actually make
> something better?
You seem to be under the mistaken impression that it's the users who
decide what the project does. Apache projects (along with most other
open source projects) are developer driven, if there are developers
interested in working on something, be it mod_ftp, mod_smtpd, or
whatever, it will get done, otherwise it won't. If the users want
something to happen and people aren't working on it, they can work on it
themselves or hire someone to do it for them, but just saying "nobody
wants this" is meaningless, since clearly the developers who are writing
the code do want it, and that's all that really matters.
-garrett
Re: httpd has no reason to improve...
Posted by Paul A Houle <ph...@cornell.edu>.
Joe Schaefer wrote:
>Paul A Houle <ph...@cornell.edu> writes:
>
>IMO market-share doesn't relate to project activity. The
>word I most associate with apache development is empowerment;
>cheifly to empower users to build better web stuff. Users that
>need to tweak the software in order to make that happen become
>committers, who are further empowered to make commits, and eventually
>make decisions about the project as a whole when they join the pmc.
>
>
>
Market share does matter. It's a function of a product having a niche.
A project that's thinking about market share is thinking about end
users. A project that isn't thinking about end users is a project of
people working on something for their own amusement.
>What I personally enjoy seeing on dev@httpd is "piling on", where
>one person implements something, then two or three other people
>(not always committers) jump on the bandwagon and take it in a better
>direction. It's good to see more of that happening lately.
>
>
>
Yeah, soon httpd will have a second- or third-rate implementation
of every protocol under the sun.
mod_ftp will underperform mainstream ftp servers so long as its
running under prefork. Similarly, cacheing proxy servers will
outperform squid.
I don't see end users clamoring for mod_ftp, or mod_snmpd. What's
the point of writing a squid replacement unless you can actually make
something better?
>According to the statistics, most users are content with 1.3. But
>that is a very short-sighted way to measure progress, because the
>situation will change as the *nix distros move away from 1.3.
>
And it's got nothing to do with how content users are with 1.3, but
with how, after 54 revisions, Apache has finally produced a server
that doesn't have serious problems running in production environments.
>Exploring
>new protocols within httpd will almost certainly will make the internal
>architecture better, even if none of those modules you mention are
>distributed with httpd. IMO mod_smtpd will be a fine example of that,
>because unlike httpd, almost all of the real action will be on the
>input_filter side (which IMO hasn't received the same amount of polish
>that the output_filter side has).
>
>
>
A server framework that can implement any protocol under the sun
might be a nice PhD thesis, but what does it do for end users?
I don't see excellence coming from "swiss army knife" frameworks
that do everything, but from systems that are developed from a whole
system viewpoint, that have a good amount of codesign between layers of
the system -- if you build a system that lets you plug in arbitrary
junk, you're going to get arbitrary performance and reliability.
(I think of the old days of Java Applet programming, where people
who were just learning OO techniques were writing great complicated
classes that were trying to be infinitely reusable, and it took them a
long time to realize that people were going to have download every byte
of the class files they were generating... But it's not just the old
"bloat" argument, but the burden of maintaining superfluous code.)
>To sum it up:
>
>better server architecture => better modules => more toys for users
>=> more interest in the 2.x internals => more patches => more activity
>=> better server architecture ...
>
>
Sure, I appreciate mod_ssl and mod_dav in Apache 2 -- but the
reason why so many people have stuck with 1.3 for so long as that 2.0
has had very little to offer end users. A few people thought it would
be great to have pluggable MPM's, and a few other people introduced
half-baked systems such as mod_cache and filters. You know a tree by
its fruit, and the fruit I care about is performance and reliability.
Apache 2.0.54 is a great server, but the fact that it took 54 revisions
to get there isn't a good indication about the design and development
process.
Were Apache development targeting real problems that real users
have, I think things would be quite different.
A big part of the problem is that the Apache project has settled
into a local equilibrium -- this explains the paradox of a product that
obviously satisfies end user's needs well (no competition has emerged)
but has a moribund development process. Any real innovation in the web
server space will need to be disruptive, to break things. Apache, as
we know it, just can't do that.
Re: httpd has no reason to improve...
Posted by Joe Schaefer <jo...@sunstarsys.com>.
Paul A Houle <ph...@cornell.edu> writes:
> Ben Collins-Sussman wrote:
>
>> I see a lot of frustration going on. The thing is, httpd's development
>> process is nearly identical to Subversion's process... we stole most
>> of it from you folks! So why all the angst in httpd-land, but not in
>> Subversion-land?
>
> It's really a lack of direction.
IMO market-share doesn't relate to project activity. The
word I most associate with apache development is empowerment;
cheifly to empower users to build better web stuff. Users that
need to tweak the software in order to make that happen become
committers, who are further empowered to make commits, and eventually
make decisions about the project as a whole when they join the pmc.
What I personally enjoy seeing on dev@httpd is "piling on", where
one person implements something, then two or three other people
(not always committers) jump on the bandwagon and take it in a better
direction. It's good to see more of that happening lately.
> There is very little going on in Apache to make it be a better web
> server -- the frontiers of development are trying to make it an ftp
> server, an smtp server, a ntp server, a snpd server, a bgp
> server... Efforts that 99.44% of httpd users are completely indifferent
> to
According to the statistics, most users are content with 1.3. But
that is a very short-sighted way to measure progress, because the
situation will change as the *nix distros move away from 1.3. Exploring
new protocols within httpd will almost certainly will make the internal
architecture better, even if none of those modules you mention are
distributed with httpd. IMO mod_smtpd will be a fine example of that,
because unlike httpd, almost all of the real action will be on the
input_filter side (which IMO hasn't received the same amount of polish
that the output_filter side has).
To sum it up:
better server architecture => better modules => more toys for users
=> more interest in the 2.x internals => more patches => more activity
=> better server architecture ...
--
Joe Schaefer