You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by Stas Bekman <st...@stason.org> on 2003/11/11 02:14:35 UTC
the wheel of httpd-dev life is surely slowing down, solutions please
I have several reasons to believe that the wheel of httpd-dev life is
slowing down and something has to be done to get this wheel up to the
speed like in the good old days. The following observation are listed
in no particular order. I've also tried to offer suggestions how to
resolve problems.
======================================================================
1) Bugs
searching for NEW and REOPENED bugs in httpd-2.0 returns: 420 entries
http://nagoya.apache.org/bugzilla/buglist.cgi?bug_status=NEW&bug_status=REOPENED&product=Apache+httpd-2.0
Suggestion: make bugzilla CC bug-reports to the dev list. Most
developers won't just go and check the bugs database to see if there
is something to fix, both because they don't have time and because
there are too many things to fix. Posting the reports to the list
raises a chance for the developer to have a free minute and may be to
resolve the bug immediately or close it.
======================================================================
2) Lack of Design
In my personal opinion, the move to CTR from RTC had very bad
implications on the design of httpd 2.1.
2a). Design of new on-going features (and changes/fixes) of the
existing features is not discussed before it's put to the code. When
it's committed it's usually too late to have any incentive to give a
design feedback, after all it's already working and we are too busy
with other things, so whatever.
The worst part is that it's now easy to sneak in code which otherwise
would never be accepted (and backport it to 2.0). I don't have any
examples, but I think the danger is there.
2b). As a side-effect when someone asks a design question (e.g. me)
it's not being answered because people tell: we are in the CRT mode,
go ahead code it and commit it. But if the poster doesn't know what's
the right design, if she did she won't have asked in first place.
2c). Future design. There is no clear roadmap of where do we go with
Apache 2.1. People scratch their itches with new ideas which is cool,
but if there was a plan to work on things, this may have invited new
developers to scratch their itches.
2d). CRT seemed to come as a replacement for design discussions. It's
very easy to observe from the traffic numbers:
http://marc.theaimsgroup.com/?l=apache-httpd-dev
2003-11-01 - 2003-12-01 (118 messages)
2003-10-01 - 2003-11-01 (336 messages)
2003-09-01 - 2003-10-01 (275 messages)
2003-08-01 - 2003-09-01 (264 messages)
2003-07-01 - 2003-08-01 (321 messages)
2003-06-01 - 2003-07-01 (430 messages)
2003-05-01 - 2003-06-01 (450 messages)
2003-04-01 - 2003-05-01 (359 messages)
2003-03-01 - 2003-04-01 (696 messages)
2003-02-01 - 2003-03-01 (573 messages)
2003-01-01 - 2003-02-01 (546 messages)
2002-12-01 - 2003-01-01 (436 messages)
2002-11-01 - 2002-12-01 (538 messages)
2002-10-01 - 2002-11-01 (763 messages)
2002-09-01 - 2002-10-01 (894 messages)
2002-08-01 - 2002-09-01 (815 messages)
2002-07-01 - 2002-08-01 (721 messages)
2002-06-01 - 2002-07-01 (916 messages)
2002-05-01 - 2002-06-01 (1028 messages)
2002-04-01 - 2002-05-01 (1380 messages)
2002-03-01 - 2002-04-01 (1094 messages)
2002-02-01 - 2002-03-01 (1155 messages)
Quiz: based on this input, tell which date CRT was introduced at.
You don't need a fancy graph to see a clear decline in the last 1.5
years. Quite a few people suggested that this is due to the market
decline. I won't doubt them, but it's quite possible that the market
hasn't changed to the worst in the last 1.5 years (it is more likely
that we should have seen this dip in 2001-2002, which I can't see from
the numbers at the above URL).
The cvs commit average rate hasn't changed much, which shows that
development continues though mainly behind the scenes.
http://marc.theaimsgroup.com/?l=apache-cvs&r=1&w=2
======================================================================
3). Contributions
I don't have numbers to support my clause, but I have a strong feeling
that nowadays we see a much smaller number of posts with contributions
from non-developers, since most contributions are simply ignored and
it's a known thing among Apache community that posting fixes and
suggestions to httpd-dev list is usually a waste of time. (This is
based on my personal experience and discussions with other developers
who aren't httpd-core coders). I realize that I'm not entirely correct
saying that, since some things are picked up, so I apologize to those
folks who do try hard to pick those things.
The obvious problem is that most of those things are unsexy, usually
don't fit someones itch and they sometimes require a lot of
communication overhead with the reporter/contributor just to
understand what exactly is going on.
Solutions:
a. certain (most?) chunks of httpd lack owners. If httpd was to have
clear owners of certain code bases then it'll be easy to take care of
bug reports and contributions/patches. Even though httpd is
collectively owned, it doesn't preclude champions in certain areas,
who can see that good things happen to the areas they overlook.
b. similar to the root rotation we have on the infrastructure, I
suggest to have a voluntary weekly rotation of httpd-dev list champion
whose responsibility is to make sure that most (all?) bug-reports and
contributions are dealts with: assigned to those who those champions
who know how to fix/review/adopt things (See (a) above), asking for
more input if needed, etc. You don't have to know httpd as your five
fingers to be able to do a great rewarding job.
c. turn the dealing with contributions and bugs into a sexy
thing. Everybody wants to feed their ego, there is no better way to
make your ego happy then getting a positive feedback from a person you
have helped to resolve a bug or handhold to provide a good patch for
the new feature, they spent so much time working on.
3a) I can hardly see new developers joining the team, should we blame
the economy or the lack of encouragement for people to contribute. I
think we now have a higher rate of developers who leave the project
(even though that technically they are still committers and all) the
those who join the project. Which obviously has clear effects on the
productivity of the dev list.
======================================================================
4). Feedback Preference and List Karma
List Karma is a known issue in all Open Source projects. It's obvious
that known and respected members of the httpd-dev list come to a
higher preference when it comes to posting feedback and any answers at
all. It's very easy to see that questions from most known httpd
developers almost always have a long thread with resolution. Other
posters are usually lucky to get any responses at all, and if they do
usually the thread would die out without any resolutions.
I'm not questioning the validity of the list karma, this is something
that I happen to stick to myself on the lists where I can give useful
feedback: I usually answer first questions from people whom I know for
good and bad. What I want to talk about is about the karma of the
posters who don't ask to solve their "personal" problems, but act on
behalf sub-projects built upon httpd. These posters try to solve a
problem for hundreds, thousands and sometimes millions of people, who
won't use Apache at all if they all they needed is to serve static
pages -- there are several other web servers which outperform Apache
on this front. They use Apache because of the 3rd party modules that
build upon Apache. It's those 3rd party modules that Apache should be
thankful to to its current world domination in the web server sector.
Therefore I think httpd developers should stop thinking, "this
poster's question is not httpd's problem, I have better things to
do". I'd like to suggest that it's a very own httpd problem, because
usually you get questions from 3rd party developers when something in
httpd doesn't cooperate with those 3rd party modules and needs to be
fixed. I think it's much less important to work on Apache 2.1 and much
more important to polish 2.0 and make 3rd party developers
happy. (wearing the 3rd party module developer hat) we don't need no
Apache 2.1 in the sky, we need Apache 2.0 on steady on the ground, so
we can run our modules on.
and if I may add that giving those 3rd party developers a commit
access is not a solution to the problem, because they hardly survive
coping with their own projects' bugs, support issues and feature
demands. So telling them: "hey, we gave you a commit access, just go
and code/fix it". is not always working.
__________________________________________________________________
Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide ---> http://perl.apache.org
mailto:stas@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org http://ticketmaster.com
Re: SPAM and CHANGES [was: the wheel of httpd-dev]
Posted by Lars Eilebrecht <la...@hyperreal.org>.
According to William A. Rowe, Jr.:
> Here is a simple suggestion; does anyone mind if I run CHANGES in 1.3,
> 2.0 and 2.1 through the following filter?
>
> perl -e "while(<stdin>){s#(<[^ @>]*)@([^ @>]*>)#$1 $2#g;print $_;}"
+1
ciao...
--
Lars Eilebrecht - Never put off until tomorrow what you can
lars@hyperreal.org - do the day after tomorrow. (Mark Twain)
Re: SPAM and CHANGES [was: the wheel of httpd-dev]
Posted by Jeff Trawick <tr...@attglobal.net>.
William A. Rowe, Jr. wrote:
> instead of [William Rowe <wr...@apache.org>] I end up with something
> like [William Rowe <wrowe apache.org>]
fine with me if you remove the @ sign by whatever mechanism... cvs diff can
help you whether or not that one-liner did the trick
SPAM and CHANGES [was: the wheel of httpd-dev]
Posted by "William A. Rowe, Jr." <wr...@apache.org>.
Here is a simple suggestion; does anyone mind if I run CHANGES in 1.3,
2.0 and 2.1 through the following filter?
perl -e "while(<stdin>){s#(<[^ @>]*)@([^ @>]*>)#$1 $2#g;print $_;}"
instead of [William Rowe <wr...@apache.org>] I end up with something
like [William Rowe <wrowe apache.org>]
Anyone can figure this out, but the bots won't. Specific harvesting isn't
prevented, but the general harvesting, that most of us are victims of, would be.
Bill
At 11:20 AM 11/11/2003, Colm MacCarthaigh wrote:
>On Mon, Nov 10, 2003 at 05:14:35PM -0800, Stas Bekman wrote:
>> 3). Contributions
>>
>> I don't have numbers to support my clause, but I have a strong feeling
>> that nowadays we see a much smaller number of posts with contributions
>> from non-developers
>
>More facetious than anything else, I'm going to hijack this part
>to make a small suggestion; give non-committers the option of
>not having their e-mail in the CHANGES/STATUS files, or at least
>of having them obfuscated in some fashion.
>
>The ammount of spam I get due to the httpd CHANGES file is simply
>unreal, most of it nonsense - with the forged from headers of other
>users of this list.
>
>That's about the only thing that ever makes me think twice about
>posting a patch/contrib.
>
>PS. This is not a complaint, it's my own silly fault for never
>remembering to ask someone about this.
>
>--
>Colm MacCárthaigh Public Key: colm+pgp@stdlib.net
Re: the wheel of httpd-dev life is surely slowing down, solutions please
Posted by Albert Chin <ht...@mlists.thewrittenword.com>.
On Thu, Nov 13, 2003 at 02:17:01AM +0100, Erik Abele wrote:
> Hmm, don't know, but I don't think so...
> Is bugzilla able to be fully controlled through email?
GCC switched from GNATS to Bugzilla. I think Daniel Berlin
<db...@dberlin.org> managed the transition and did some work on an
email <-> Bugzilla gateway (though I might be mistaken). Ping the GCC
folks for info.
--
albert chin (china@thewrittenword.com)
Re: the wheel of httpd-dev life is surely slowing down, solutions please
Posted by Aaron Bannert <aa...@clove.org>.
On Thu, Nov 13, 2003 at 02:17:01AM +0100, Erik Abele wrote:
> >My main requirement is that the bug tracking system be fully-accessible
> >through email. Having a full web interface is great, but not at the
> >expense of usable offline replies to bug reports.
>
> Okay, I can understand that.
>
> >(Do either of these bug tracking systems support this?)
>
> Hmm, don't know, but I don't think so...
> Is bugzilla able to be fully controlled through email?
Nope, but it sucks for other reasons too. ;)
We used to use gnats until someone suggested we try Bugzilla.
I suggested today that we might look into using the homegrown
bug tracking system used by debian. It is email based with a web
interface.
-aaron
Re: the wheel of httpd-dev life is surely slowing down, solutions please
Posted by Erik Abele <er...@codefaktor.de>.
On 13.11.2003, at 02:10, Aaron Bannert wrote:
> On Thu, Nov 13, 2003 at 02:04:31AM +0100, Erik Abele wrote:
>> I'd be very in favour of exploring other forms of bug-tracking. For
>> example
>> we'll have a full replication of BugZilla in Jira (/me hides) in the
>> near future
>> here on ASF hardware (see http://nagoya.apache.org/jira/ for a
>> preview).
>>
>> I've to admit that I really like it and especially as a patch-tracking
>> tool
>> it could be useful, at least IMHO.
>>
>> Btw, Scarab (http://nagoya.apache.org/scarab/issues) is also available
>> but I
>> doubt that it is used by a single project ;-)
>
> My main requirement is that the bug tracking system be fully-accessible
> through email. Having a full web interface is great, but not at the
> expense of usable offline replies to bug reports.
Okay, I can understand that.
> (Do either of these bug tracking systems support this?)
Hmm, don't know, but I don't think so...
Is bugzilla able to be fully controlled through email?
Cheers,
Erik
> -aaron
>
Re: the wheel of httpd-dev life is surely slowing down, solutions
please
Posted by Stas Bekman <st...@stason.org>.
Mads Toftum wrote:
> On Wed, Nov 12, 2003 at 05:10:24PM -0800, Aaron Bannert wrote:
>
>>My main requirement is that the bug tracking system be fully-accessible
>>through email. Having a full web interface is great, but not at the
>>expense of usable offline replies to bug reports.
>>
>
> RT could be what you're looking for - http://www.bestpractical.com/rt
>
> An example is http://rt.cpan.org
Seconded. The perl project uses RT with a great success (http://rt.perl.org/)
__________________________________________________________________
Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide ---> http://perl.apache.org
mailto:stas@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org http://ticketmaster.com
Re: the wheel of httpd-dev life is surely slowing down, solutions please
Posted by Mads Toftum <ma...@toftum.dk>.
On Wed, Nov 12, 2003 at 05:10:24PM -0800, Aaron Bannert wrote:
>
> My main requirement is that the bug tracking system be fully-accessible
> through email. Having a full web interface is great, but not at the
> expense of usable offline replies to bug reports.
>
RT could be what you're looking for - http://www.bestpractical.com/rt
An example is http://rt.cpan.org
vh
Mads Toftum
--
`Darn it, who spiked my coffee with water?!' - lwall
Re: the wheel of httpd-dev life is surely slowing down, solutions please
Posted by Aaron Bannert <aa...@clove.org>.
On Thu, Nov 13, 2003 at 02:04:31AM +0100, Erik Abele wrote:
> I'd be very in favour of exploring other forms of bug-tracking. For
> example
> we'll have a full replication of BugZilla in Jira (/me hides) in the
> near future
> here on ASF hardware (see http://nagoya.apache.org/jira/ for a preview).
>
> I've to admit that I really like it and especially as a patch-tracking
> tool
> it could be useful, at least IMHO.
>
> Btw, Scarab (http://nagoya.apache.org/scarab/issues) is also available
> but I
> doubt that it is used by a single project ;-)
My main requirement is that the bug tracking system be fully-accessible
through email. Having a full web interface is great, but not at the
expense of usable offline replies to bug reports.
(Do either of these bug tracking systems support this?)
-aaron
Re: the wheel of httpd-dev life is surely slowing down, solutions please
Posted by Erik Abele <er...@codefaktor.de>.
On 13.11.2003, at 01:29, Aaron Bannert wrote:
> On Mon, Nov 10, 2003 at 05:14:35PM -0800, Stas Bekman wrote:
>> ======================================================================
>> 1) Bugs
>>
>> searching for NEW and REOPENED bugs in httpd-2.0 returns: 420 entries
>> http://nagoya.apache.org/bugzilla/buglist.cgi?
>> bug_status=NEW&bug_status=REOPENED&product=Apache+httpd-2.0
>>
>> Suggestion: make bugzilla CC bug-reports to the dev list. Most
>> developers won't just go and check the bugs database to see if there
>> is something to fix, both because they don't have time and because
>> there are too many things to fix. Posting the reports to the list
>> raises a chance for the developer to have a free minute and may be to
>> resolve the bug immediately or close it.
>
> Would anyone else be in favor of exploring other forms of bug-tracking?
> The old system (gnats) was completely email-oriented, which was good
> and bad. Bugzilla may be more usable for web-oriented users, but it
> it definitely more difficult to use offline (ie. impossible). Perhaps
> we could try out the bug tracking system used by the debian project
> (http://www.debian.org/Bugs/).
I'd be very in favour of exploring other forms of bug-tracking. For
example
we'll have a full replication of BugZilla in Jira (/me hides) in the
near future
here on ASF hardware (see http://nagoya.apache.org/jira/ for a preview).
I've to admit that I really like it and especially as a patch-tracking
tool
it could be useful, at least IMHO.
Btw, Scarab (http://nagoya.apache.org/scarab/issues) is also available
but I
doubt that it is used by a single project ;-)
Cheers,
Erik
Re: the wheel of httpd-dev life is surely slowing down, solutions please
Posted by Aaron Bannert <aa...@clove.org>.
On Mon, Nov 10, 2003 at 05:14:35PM -0800, Stas Bekman wrote:
> ======================================================================
> 1) Bugs
>
> searching for NEW and REOPENED bugs in httpd-2.0 returns: 420 entries
> http://nagoya.apache.org/bugzilla/buglist.cgi?bug_status=NEW&bug_status=REOPENED&product=Apache+httpd-2.0
>
> Suggestion: make bugzilla CC bug-reports to the dev list. Most
> developers won't just go and check the bugs database to see if there
> is something to fix, both because they don't have time and because
> there are too many things to fix. Posting the reports to the list
> raises a chance for the developer to have a free minute and may be to
> resolve the bug immediately or close it.
Would anyone else be in favor of exploring other forms of bug-tracking?
The old system (gnats) was completely email-oriented, which was good
and bad. Bugzilla may be more usable for web-oriented users, but it
it definitely more difficult to use offline (ie. impossible). Perhaps
we could try out the bug tracking system used by the debian project
(http://www.debian.org/Bugs/).
> ======================================================================
> 2) Lack of Design
>
> In my personal opinion, the move to CTR from RTC had very bad
> implications on the design of httpd 2.1.
>
> 2a). Design of new on-going features (and changes/fixes) of the
> existing features is not discussed before it's put to the code. When
> it's committed it's usually too late to have any incentive to give a
> design feedback, after all it's already working and we are too busy
> with other things, so whatever.
I agree with the premise but not your conclusion. Design is being
adversely affected when people make large design-changing commits.
These people should be posting their design changes (code and all)
to the mailing list before committing.
However, if people are making commits and others aren't reviewing
those commits, then the responsibility falls to the reviewer. Nobody
should be obligated to hold on to a patch because other people don't have
the time to review.
> The worst part is that it's now easy to sneak in code which otherwise
> would never be accepted (and backport it to 2.0). I don't have any
> examples, but I think the danger is there.
I have not seen any evidence of this.
> 2b). As a side-effect when someone asks a design question (e.g. me)
> it's not being answered because people tell: we are in the CRT mode,
> go ahead code it and commit it. But if the poster doesn't know what's
> the right design, if she did she won't have asked in first place.
There is not a fine line between those types of changes that warrant
discussion and those that can be simply committed, so this is a difficult
issue to address.
It is not uncommon for design questions to go unanswered on dev@httpd,
and it has been that way as long as I can remember. Patches, on the other
hand, speak loudest.
> 2c). Future design. There is no clear roadmap of where do we go with
> Apache 2.1. People scratch their itches with new ideas which is cool,
> but if there was a plan to work on things, this may have invited new
> developers to scratch their itches.
Make proposals (or better yet, add them to the 2.1 STATUS file. :)
I would personally like to see:
a) httpd.conf rewrite
- normalized syntax (structured, maybe even *gasp* XML)
- normalized parsing in apache (defined passes, hooks, restart semantics, etc)
- other ways to retrieve a config (not just from a file, eg. LDAP)
b) sendfile improvements for 64bit memory addressing machines
(eg. can we mmap/sendfile a bunch of DVD images at the same time and
not crash? Do we see improvements?)
c) simplified filters
(it's been too long since I've thought about this, but basicly writing
filters is too difficult and should be easier).
> 2d). CRT seemed to come as a replacement for design discussions. It's
> very easy to observe from the traffic numbers:
I think it's actually the opposite. The amount of design discussions in
general has dramatically decreased since RTC went into effect in 2.0.
I recall very few discussions around 2.1 in general.
> ======================================================================
> 3). Contributions
>
> I don't have numbers to support my clause, but I have a strong feeling
> that nowadays we see a much smaller number of posts with contributions
> from non-developers, since most contributions are simply ignored and
> it's a known thing among Apache community that posting fixes and
> suggestions to httpd-dev list is usually a waste of time. (This is
> based on my personal experience and discussions with other developers
> who aren't httpd-core coders). I realize that I'm not entirely correct
> saying that, since some things are picked up, so I apologize to those
> folks who do try hard to pick those things.
>
> The obvious problem is that most of those things are unsexy, usually
> don't fit someones itch and they sometimes require a lot of
> communication overhead with the reporter/contributor just to
> understand what exactly is going on.
This is a problem that has always plagued Apache (and probably most open
source projects in general). So yes, I agree that it is a problem.
> Solutions:
>
> a. certain (most?) chunks of httpd lack owners. If httpd was to have
> clear owners of certain code bases then it'll be easy to take care of
> bug reports and contributions/patches. Even though httpd is
> collectively owned, it doesn't preclude champions in certain areas,
> who can see that good things happen to the areas they overlook.
-1
Everyone should be encouraged to work on whatever they want, and shouldn't
have to wait for permission from an "owner" to work on their piece.
It's easy to be a champion right now. Take a piece that you think needs
fixing and start committing those fixes. Your changes will be noticed.
> c. turn the dealing with contributions and bugs into a sexy
> thing. Everybody wants to feed their ego, there is no better way to
> make your ego happy then getting a positive feedback from a person you
> have helped to resolve a bug or handhold to provide a good patch for
> the new feature, they spent so much time working on.
IMHO that incentive is there right now, but the tools get in the way.
> 3a) I can hardly see new developers joining the team, should we blame
> the economy or the lack of encouragement for people to contribute. I
> think we now have a higher rate of developers who leave the project
> (even though that technically they are still committers and all) the
> those who join the project. Which obviously has clear effects on the
> productivity of the dev list.
That in and of itself is not a problem, in my opinion.
-aaron
Re[2]: the wheel of httpd-dev life is surely slowing down, solutions please
Posted by Astrid Keßler <ke...@kess-net.de>.
> Read "The Cathedral & The Bazaar" from ESR [2]. It provides a good
> insight into the social structures of the "community": motivations,
> incentives, .. It's not always the money, you know ;) Market share?
> Who cares! Customers? Who cares! I want my name to be a three letter
> acronym everyone recognizes! RMS? ESR?
How pity. Hunting for honor is just a short term motivation. What about
the next hype? Will you jump on the new train?
This is not what makes HTTP. This doesn't bring the market share. This
doesn't bring the trust, customers have. It may lead into some good
features - mostly maintained by others after some time.
Sorry, this is your motivation. It's ok for you. But don't believe, this
drives everybody. For myself, I joined the project, because I have been
sad to tell people where to find the right thing within the
documentation. I want a good software to be used. This is why I does not
code. There are lots of programmers. But less documentation writers.
I do not need my name written on the sky. Sure, honor is nice. But good
teamwork is nicer. And THIS is the discussion about. How to improve the
teamwork to get better results.
Just my 2 cents
Kess
Re: the wheel of httpd-dev life is surely slowing down, solutions
please
Posted by Henri Gomez <hg...@apache.org>.
Daniel Lorch a écrit :
> hi,
>
>> I'm not sure http-dev is the place to flam ASF and its commiters.
>
>
> I don't think it was Peter's intention to flame anyone. The ASF has done
> a great job to deliver a fantastic, widely-deployed webserver.
> Consindering though that Apache 2 is mostly a refactored 1.3.x, which
> doesn't provide anything spectacularly new. Yes improved X and better Y,
> but hey, it still is a webserver and it doesn't make coffee for you.
Yes, HTTPD 2.0 is still a webserver but the refactory was impressive
and it use APR which is a wonderfull API to hide Operating System
specifics.
> If I can get my name into the headlines [1] when writing Java-Stuff,
> hell, then I'll do it.
>
> Read "The Cathedral & The Bazaar" from ESR [2]. It provides a good
> insight into the social structures of the "community": motivations,
> incentives, .. It's not always the money, you know ;) Market share?
> Who cares! Customers? Who cares! I want my name to be a three letter
> acronym everyone recognizes! RMS? ESR?
>
> [1] http://apache.slashdot.org/apache/03/11/10/2057218.shtml
> http://www.theserverside.com/home/thread.jsp?thread_id=22337
>
> [2] http://www.oreilly.com/catalog/cathbazpaper/
I know well 'The Cathedral & The Bazaar' and it's not allways money,
but who speak about money here. I just speak about market shares, in
term of users base. And when more than 66% of users trust your product
you couldn't just make permanent revolution.
If you don't care about customers, in our case HTTPD users, I'm not
sure you understand that even if OSS is a software production model,
it's final goal is to produce real software, for real users, and
more users are using your software more responsability you have.
If someone want to produce software for its own use, without users
considerations, well it shouldn't works in communities and certainly not
with ASF commiters.
Re: Intermittent trouble with mod_auth_ldap in 2.0 and 1.3
Posted by Ace Suares <ac...@suares.nl>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Oh... thank you for pointing that out. I *might* have done that, I hate typing
emailaddresses :-)
I'll post the article once again, properly.
_Ace
>
> weird... the old thread has subject "the wheel of httpd-dev life is surely
> slowing down, solutions please"
>
> Your first post on the LDAP question had
>
> References: <BA...@hotmail.com>
> <3F...@apache.org> <3F...@lorch.cc>
>
> and
>
> In-Reply-To: <3F...@lorch.cc>
>
> which stuck it right in the middle of that unrelated thread.
>
> Usually this happens when somebody has a message selected and uses their
> client's "Reply" button and then changes the subject manually. If that
> isn't what you did, then I don't know why your mail client added the bogus
> headers.
- --
Ace Suares' Internet Consultancy
NIEUW ADRES: Postbus 2599, 4800 CN Breda
telefoon: 06-244 33 608
fax en voicemail: 0848-707 705
website: http://www.suares.nl * http://www.qwikzite.nl
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.2-rc1-SuSE (GNU/Linux)
iD8DBQE/sR7ey7boE8xtIjURAlepAJ4pAxHzAThXWc8GvDpr93qihN6hwgCgjXmM
nt9oXjtxpEjcw1S04iyCrNQ=
=aOXK
-----END PGP SIGNATURE-----
Re: Intermittent trouble with mod_auth_ldap in 2.0 and 1.3
Posted by Jeff Trawick <tr...@attglobal.net>.
Ace Suares wrote:
>>BTW, when you reply to a post on some topic and then change the subject, in
>>many mail clients your post will appear in the thread of that old topic.
>>That seems to be the case with this new thread.
>
>
> Hmm... very intersting.
> I was not aware that I replied to a post on this list or have changed a
> subject. Could you refer met to the 'old' thread ??
weird... the old thread has subject "the wheel of httpd-dev life is surely
slowing down, solutions please"
Your first post on the LDAP question had
References: <BA...@hotmail.com>
<3F...@apache.org> <3F...@lorch.cc>
and
In-Reply-To: <3F...@lorch.cc>
which stuck it right in the middle of that unrelated thread.
Usually this happens when somebody has a message selected and uses their
client's "Reply" button and then changes the subject manually. If that isn't
what you did, then I don't know why your mail client added the bogus headers.
Re: Intermittent trouble with mod_auth_ldap in 2.0 and 1.3
Posted by Ace Suares <ac...@suares.nl>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
> Ace Suares wrote:
> > Wether I am using Apache 1.3 with mod_auth_ldap 1.6.0 (from Rudedog) or
> > Apache 2.0 with the distributed auth_ldap module (which is, as I
> > understand, based on the rudedog module), I am experiencing the same
> > problems.
>
> BTW, when you reply to a post on some topic and then change the subject, in
> many mail clients your post will appear in the thread of that old topic.
> That seems to be the case with this new thread.
Hmm... very intersting.
I was not aware that I replied to a post on this list or have changed a
subject. Could you refer met to the 'old' thread ??
_Ace
website: http://www.suares.nl * http://www.qwikzite.nl
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.2-rc1-SuSE (GNU/Linux)
iD8DBQE/sRgry7boE8xtIjURAoFZAJ9XXh647TwDNN7xq8TAIhBkqPX4XgCgojrV
eplOx5sPdAZtaiVMLkSTV/8=
=P8Xi
-----END PGP SIGNATURE-----
Re: Intermittent trouble with mod_auth_ldap in 2.0 and 1.3
Posted by Jeff Trawick <tr...@attglobal.net>.
Ace Suares wrote:
> Wether I am using Apache 1.3 with mod_auth_ldap 1.6.0 (from Rudedog) or Apache
> 2.0 with the distributed auth_ldap module (which is, as I understand, based
> on the rudedog module), I am experiencing the same problems.
BTW, when you reply to a post on some topic and then change the subject, in
many mail clients your post will appear in the thread of that old topic. That
seems to be the case with this new thread.
Intermittent trouble with mod_auth_ldap in 2.0 and 1.3
Posted by Ace Suares <ac...@suares.nl>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Hi All,
Wether I am using Apache 1.3 with mod_auth_ldap 1.6.0 (from Rudedog) or Apache
2.0 with the distributed auth_ldap module (which is, as I understand, based
on the rudedog module), I am experiencing the same problems.
Over at the auth_ldap@rudedog.org mailinglist, we analyzed the problem with
help op Brent Putnam, who posted a patch almost 2 years ago for a certain
problem that seems related. Find the patch and a description here:
http://www.rudedog.org/pipermail/auth_ldap/2001-December/043545.html
The problem that Brent describes relates to the use of AuthLdapBindDN, but I
am binding anonymously and seem to have the same problems.
The most clever description of the problem can be found in above link, but
I'll put it down in my own words:
Whenever I login as userA, which is succesfull, and then want to login to
another URL with another .htaccess file with another Realm, as userB, I am
not allowed access. In my setup, only anonymous can see (certain attributes)
from all entires in the ldap directory; userA can not see userB and vice
versa.
When I get to the page for userB, I don't even get a pop-up that asks me for
username and password. I just get a 401 error. When I refresh the page
several times, I might get a pop-up, which I fill in with the correct
authentication information, but acces is disallowed and I get a pop-up again.
I can keep doing this several times. Meanwhile, I can go back to the page for
userA with no problems.
After a certain amount of refreshes followed by a certain amount of filling in
authentication info in the pop-up, I suddenly get access. Then, the page for
userA doens't let me in anymore. Even if they are in different realms!
I can provide you with more debugging info, but at the moment I'll just wait
for reactions.
My settings in apache for mod_auth_ldap:
in httpd.conf:
AuthLDAPOpCacheSize 0
AuthLDAPCacheSize 0
______________
in .htaccess for userA in urlA:
AuthName "Login for example.com"
AuthType Basic
ldap://localhost:389/dc=example,dc=com,qapp=qwido?uid?sub?(objectclass=qManager)
AuthLDAPRemoteUserIsDN on
require valid-user
in .htaccess for userB in urlB:
AuthName "Login for suares.com"
AuthType Basic
AuthLDAPURL
ldap://localhost:389/dc=suares,dc=com,qapp=qwido?uid?sub?(objectclass=qManager)
AuthLDAPRemoteUserIsDN on
require valid-user
Above is the config for Apache 1.3, but I am experiencing the same problems
with Apache 2.0. I also tried Opera, Mozilla and Konquerer as browsers.
I would appreciate any info on this issue.
I hope this is the riht place to contact developers for mod_auth_ldap in 2.0
Cheers,
Ace
website: http://www.suares.nl * http://www.qwikzite.nl
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.2-rc1-SuSE (GNU/Linux)
iD8DBQE/sQB6y7boE8xtIjURAu0NAKCMaOMtTbYzblRpIIxYjWv/sWxnswCeNtFd
4hWYBnoQn8qBFTiWdHEAR5w=
=n/sa
-----END PGP SIGNATURE-----
Re: the wheel of httpd-dev life is surely slowing down, solutions
please
Posted by André Malo <nd...@perlig.de>.
* Daniel Lorch <ml...@lorch.cc> wrote:
> If I can get my name into the headlines [1] when writing Java-Stuff,
> hell, then I'll do it.
>
> Read "The Cathedral & The Bazaar" from ESR [2]. It provides a good
> insight into the social structures of the "community": motivations,
> incentives, .. It's not always the money, you know ;) Market share?
> Who cares! Customers? Who cares! I want my name to be a three letter
> acronym everyone recognizes! RMS? ESR?
Sure, but that's your way. It's not mine, for example. My personal intention
is to make the software better. That's it. I don't need to read my name in
headlines.
Why did I start contributing? Because I've found errors. Why did I start
cleaning up Bugzilla-Reports? Because I searched something and didn't find
through because there was too much noise. That's all. Forget my name. Use
that great piece of open source software and learn from it. I love good and
clean code. So my most contributions clean up code and try to improve it. In
fact, most people will never know about this, because it's just behind the
scenes.
Apart from the above I don't like Java very much ;-)
nd
Re: the wheel of httpd-dev life is surely slowing down, solutions
please
Posted by Daniel Lorch <ml...@lorch.cc>.
hi,
> I'm not sure http-dev is the place to flam ASF and its commiters.
I don't think it was Peter's intention to flame anyone. The ASF has done
a great job to deliver a fantastic, widely-deployed webserver.
Consindering though that Apache 2 is mostly a refactored 1.3.x, which
doesn't provide anything spectacularly new. Yes improved X and better Y,
but hey, it still is a webserver and it doesn't make coffee for you.
If I can get my name into the headlines [1] when writing Java-Stuff,
hell, then I'll do it.
Read "The Cathedral & The Bazaar" from ESR [2]. It provides a good
insight into the social structures of the "community": motivations,
incentives, .. It's not always the money, you know ;) Market share?
Who cares! Customers? Who cares! I want my name to be a three letter
acronym everyone recognizes! RMS? ESR?
[1] http://apache.slashdot.org/apache/03/11/10/2057218.shtml
http://www.theserverside.com/home/thread.jsp?thread_id=22337
[2] http://www.oreilly.com/catalog/cathbazpaper/
-daniel
RE: the wheel of httpd-dev life is surely slowing down, solutions please
Posted by "Peter J. Cranstone" <cr...@msn.com>.
>> Well the http tuning of string handling is a known factor of
>> optimization
You're right - nothing new about optimizing string handling - just doing it
>> BTW, if you post these benchmarks on the HTTPd-dev list, should I
assume you'll give ASF your optimized & tuned algorithms ?
I wouldn't assume anything at this point - however if you remember correctly
we did give the ASF mod_gzip (last time I checked even Google was
compressing their output), however there will be an open source contribution
at some point.
>> Do you known that IBM does some nice optimization using FRCA on its
Apache 2.0 implementation on iSeries
Great - where are the benchmarks, as I said in my earlier post what's the
differentiator between 1.x with 66% of the market and 2.x with 0% of the
market?
Remember your audience - it either has to make me money or save me money. If
I'm going to implement 2.x then I should be able to see a return on the time
I've invested either through a hardware/performance improvement or
productivity.
We all know why 2.x is struggling - the bar was set with 1.x and 2.x failed
to move it more than about 1 inch.
Peter
-----Original Message-----
From: Henri Gomez [mailto:hgomez@apache.org]
Sent: Tuesday, November 11, 2003 8:25 AM
To: dev@httpd.apache.org
Subject: Re: the wheel of httpd-dev life is surely slowing down, solutions
please
Peter J. Cranstone a écrit :
> There is no flame - just a couple of points and a request for data.
>
>
>>>If you want to improve something, you should provide solutions,
>
> not critics
>
> Certainly - early next year you will see them. Here are some current
> performance stats with some new technology we're working on.
>
>
> Configuration Tool Elapsed Time
> (secs) Data Transfer Rate
> (KB/sec) Requests per Second Requests per
> Minute Performance Gain
> Factor
> Apache Apache Bench 38.735 882.92 2581.64 154,898 1.0
> Cyclone Proxy Cache
> +
> Apache Apache Bench 15.663 2387.79 6384.47 383,068 2.47
> Apache Zeus Bench 39.961 855.83 2502.44 150,146.4 1.0
> Squid
> +
> Apache Zeus Bench 28.910 1314.42 3459.01 207,540.6 1.38
> Cyclone Proxy Cache
> +
> Apache Zeus Bench 15.176 2464.42 6589.35 395,361 2.63
> Cyclone Proxy Cache
> (Tuned Parser)
> +
> Apache Zeus Bench 13.505 2769.34 7404.67 444,280.2 2.95
> Cyclone Proxy Cache
> (4 Tuned Functions)
> +
> Apache Zeus Bench 13.006 2875.6 7688.76 461,325.6 3.07
>
> These numbers were obtained using a single processor Itanium® 1.0Ghz
> (Madison) chip. By tuning certain HTTP string handling functions we have
> seen up to a factor 11 performance improvement.
>
> Our next benchmark is due by year end. Essentially we will be adding one
> more line for the stats above. The goal is very simple - transmit greater
> than 1 million requests in a single minute on a single processor Itanium
> 1Ghz machine. A factor 10 performance improvement. A single processor
> Deerfield Itanium® chip costs $744 - our solution doesn't require a
current
> OS, nor hard drive to operate - it scales to multiple chips and can
support
> a cache of up to 1 terabyte of RAM
>
>
>>>Revolution is for new players, carefully crafted evolutions are for the
>
> Mass
>
> Yep
Support for a 1TB cache, no hard drive, no current OS required, and
> the ability to pump data faster than any other platform on the planet
should
> do the trick. Only thing left is to get the Itanium® platform into a
single
> 1RU box at sub $5,000. I doubt we will have to wait long for that.
>
> Long live the revolution
>
> Regards,
Well the http tuning of string handling is a known factor of
optimization, just study tomcat 3.2, 3.3 and Coyote 1.1 and
you'll that it could still be optimized.
BTW, if you post these benchmarks on the httpd-dev list, should I
assume you'll give ASF your optimized&tuned algorythms ?
Do you known that IBM does some nice optimization using FRCA on its
Apache 2.0 implementation on iSeries ?
A proof that Apache 2.0 is a great platform for such games ;)
Re: the wheel of httpd-dev life is surely slowing down, solutions
please
Posted by Henri Gomez <hg...@apache.org>.
Peter J. Cranstone a écrit :
> There is no flame - just a couple of points and a request for data.
>
>
>>>If you want to improve something, you should provide solutions,
>
> not critics
>
> Certainly - early next year you will see them. Here are some current
> performance stats with some new technology we're working on.
>
>
> Configuration Tool Elapsed Time
> (sec’s) Data Transfer Rate
> (KB/sec) Requests per Second Requests per
> Minute Performance Gain
> Factor
> Apache Apache Bench 38.735 882.92 2581.64 154,898 1.0
> Cyclone Proxy Cache
> +
> Apache Apache Bench 15.663 2387.79 6384.47 383,068 2.47
> Apache Zeus Bench 39.961 855.83 2502.44 150,146.4 1.0
> Squid
> +
> Apache Zeus Bench 28.910 1314.42 3459.01 207,540.6 1.38
> Cyclone Proxy Cache
> +
> Apache Zeus Bench 15.176 2464.42 6589.35 395,361 2.63
> Cyclone Proxy Cache
> (Tuned Parser)
> +
> Apache Zeus Bench 13.505 2769.34 7404.67 444,280.2 2.95
> Cyclone Proxy Cache
> (4 Tuned Functions)
> +
> Apache Zeus Bench 13.006 2875.6 7688.76 461,325.6 3.07
>
> These numbers were obtained using a single processor Itanium® 1.0Ghz
> (Madison) chip. By tuning certain HTTP string handling functions we have
> seen up to a factor 11 performance improvement.
>
> Our next benchmark is due by year end. Essentially we will be adding one
> more line for the stats above. The goal is very simple - transmit greater
> than 1 million requests in a single minute on a single processor Itanium
> 1Ghz machine. A factor 10 performance improvement. A single processor
> Deerfield Itanium® chip costs $744 - our solution doesn't require a current
> OS, nor hard drive to operate - it scales to multiple chips and can support
> a cache of up to 1 terabyte of RAM
>
>
>>>Revolution is for new players, carefully crafted evolutions are for the
>
> Mass
>
> Yep… Support for a 1TB cache, no hard drive, no current OS required, and
> the ability to pump data faster than any other platform on the planet should
> do the trick. Only thing left is to get the Itanium® platform into a single
> 1RU box at sub $5,000. I doubt we will have to wait long for that.
>
> Long live the revolution
>
> Regards,
Well the http tuning of string handling is a known factor of
optimization, just study tomcat 3.2, 3.3 and Coyote 1.1 and
you'll that it could still be optimized.
BTW, if you post these benchmarks on the httpd-dev list, should I
assume you'll give ASF your optimized&tuned algorythms ?
Do you known that IBM does some nice optimization using FRCA on its
Apache 2.0 implementation on iSeries ?
A proof that Apache 2.0 is a great platform for such games ;)
RE: the wheel of httpd-dev life is surely slowing down, solutions please
Posted by "Peter J. Cranstone" <cr...@msn.com>.
There is no flame - just a couple of points and a request for data.
>> If you want to improve something, you should provide solutions,
not critics
Certainly - early next year you will see them. Here are some current
performance stats with some new technology we're working on.
Configuration Tool Elapsed Time
(secs) Data Transfer Rate
(KB/sec) Requests per Second Requests per
Minute Performance Gain
Factor
Apache Apache Bench 38.735 882.92 2581.64 154,898 1.0
Cyclone Proxy Cache
+
Apache Apache Bench 15.663 2387.79 6384.47 383,068 2.47
Apache Zeus Bench 39.961 855.83 2502.44 150,146.4 1.0
Squid
+
Apache Zeus Bench 28.910 1314.42 3459.01 207,540.6 1.38
Cyclone Proxy Cache
+
Apache Zeus Bench 15.176 2464.42 6589.35 395,361 2.63
Cyclone Proxy Cache
(Tuned Parser)
+
Apache Zeus Bench 13.505 2769.34 7404.67 444,280.2 2.95
Cyclone Proxy Cache
(4 Tuned Functions)
+
Apache Zeus Bench 13.006 2875.6 7688.76 461,325.6 3.07
These numbers were obtained using a single processor Itanium® 1.0Ghz
(Madison) chip. By tuning certain HTTP string handling functions we have
seen up to a factor 11 performance improvement.
Our next benchmark is due by year end. Essentially we will be adding one
more line for the stats above. The goal is very simple - transmit greater
than 1 million requests in a single minute on a single processor Itanium
1Ghz machine. A factor 10 performance improvement. A single processor
Deerfield Itanium® chip costs $744 - our solution doesn't require a current
OS, nor hard drive to operate - it scales to multiple chips and can support
a cache of up to 1 terabyte of RAM
>> Revolution is for new players, carefully crafted evolutions are for the
Mass
Yep
Support for a 1TB cache, no hard drive, no current OS required, and
the ability to pump data faster than any other platform on the planet should
do the trick. Only thing left is to get the Itanium® platform into a single
1RU box at sub $5,000. I doubt we will have to wait long for that.
Long live the revolution
Regards,
Peter
-----Original Message-----
From: Henri Gomez [mailto:hgomez@apache.org]
Sent: Tuesday, November 11, 2003 7:41 AM
To: dev@httpd.apache.org
Subject: Re: the wheel of httpd-dev life is surely slowing down, solutions
please
Peter J. Cranstone a écrit :
>>>It's not anymore "cool" to work on Apache.
>
>
> You nailed it - because no one knows where it's going. Where's the focus,
> what does Apache really want to be, whose leading the charge?
>
> I've been following this forum a long, long time and the change in the
last
> 2 years has been the most dramatic - the old guard has gone, there is
little
> leadership and even less reason to do anything.
>
> It takes a tremendous amount of work to build a quality software project
and
> sadly there is little enthusiasm to really improve Apache.
>
> One reason is obvious - with 66% of the market you're a monopoly (close)
and
> we've all seen what happens when competition disappears from the market
> place.
I'm not sure http-dev is the place to flam ASF and its commiters.
If you want to improve something, you should provide solutions,
not critics.
HTTPD have 66% of the market and that's great to see that
an OpenSource solution is well behind M$.
Sun, Oracle and majors corps have stopped dreaming having 50% of market
share some years ago.
At least we could say, Apache Software Foundation does it and
maintain its leading position.
How ?
- By producing solutions like HTTPD which are stable,
full featured and works on so many platforms.
Revolution is for new players, carefully crafted evolutions are for the
mass.
Re: the wheel of httpd-dev life is surely slowing down, solutions
please
Posted by Henri Gomez <hg...@apache.org>.
Peter J. Cranstone a écrit :
>>>It's not anymore "cool" to work on Apache.
>
>
> You nailed it - because no one knows where it's going. Where's the focus,
> what does Apache really want to be, whose leading the charge?
>
> I've been following this forum a long, long time and the change in the last
> 2 years has been the most dramatic - the old guard has gone, there is little
> leadership and even less reason to do anything.
>
> It takes a tremendous amount of work to build a quality software project and
> sadly there is little enthusiasm to really improve Apache.
>
> One reason is obvious - with 66% of the market you're a monopoly (close) and
> we've all seen what happens when competition disappears from the market
> place.
I'm not sure http-dev is the place to flam ASF and its commiters.
If you want to improve something, you should provide solutions,
not critics.
HTTPD have 66% of the market and that's great to see that
an OpenSource solution is well behind M$.
Sun, Oracle and majors corps have stopped dreaming having 50% of market
share some years ago.
At least we could say, Apache Software Foundation does it and
maintain its leading position.
How ?
- By producing solutions like HTTPD which are stable,
full featured and works on so many platforms.
Revolution is for new players, carefully crafted evolutions are for the
mass.
RE: the wheel of httpd-dev life is surely slowing down, solutions please
Posted by "Peter J. Cranstone" <cr...@msn.com>.
>> It's not anymore "cool" to work on Apache.
You nailed it - because no one knows where it's going. Where's the focus,
what does Apache really want to be, whose leading the charge?
I've been following this forum a long, long time and the change in the last
2 years has been the most dramatic - the old guard has gone, there is little
leadership and even less reason to do anything.
It takes a tremendous amount of work to build a quality software project and
sadly there is little enthusiasm to really improve Apache.
One reason is obvious - with 66% of the market you're a monopoly (close) and
we've all seen what happens when competition disappears from the market
place.
Regards,
Peter
-----Original Message-----
From: Daniel Lorch [mailto:ml-daniel@lorch.cc]
Sent: Tuesday, November 11, 2003 7:10 AM
To: dev@httpd.apache.org
Subject: Re: the wheel of httpd-dev life is surely slowing down, solutions
please
hi,
> 2d). CRT seemed to come as a replacement for design discussions. It's
> very easy to observe from the traffic numbers:
Please excuse the total ignorance of passive Apache-Dev readers, but
these abbreviations were new to me. I've found then im the Apache
Glossary, though, and provide them to all who didn't know them ei-
ther:
http://incubator.apache.org/learn/glossary.html#CommitThenReview
http://incubator.apache.org/learn/glossary.html#ReviewThenCommit
Not trying to start a flamewar, but could the Jakarta Project have
had an influence on the decline? Hal Flynn's [1] points out quite well
that for most server-sided applications Java (and it's clone .NET)
provide a viable platform for secure applications with negligible
impact on performance. And with all the fuss around J2EE (JBoss vs.
Geronimo) a new feature in Apache might not catch as much attention
in the community anymore. OpenSource is a development model which
bases on peer-based ego-gratification and if the incentives to work
on Apache aren't that high anymore, the Apache httpd server might not
be able to attract as many developers anymore. Or to put it in other
words: It's not anymore "cool" to work on Apache.
uuhmm .. <asbestos> or not? No intention to provoke, seriously. We're
all grownup people and our kids are going to read these mailinglists
in CS-history classes, so behave, please ;)
[1] http://www.theregister.co.uk/content/4/33859.html
-daniel
Re: the wheel of httpd-dev life is surely slowing down, solutions
please
Posted by Daniel Lorch <ml...@lorch.cc>.
hi,
> 2d). CRT seemed to come as a replacement for design discussions. It's
> very easy to observe from the traffic numbers:
Please excuse the total ignorance of passive Apache-Dev readers, but
these abbreviations were new to me. I've found then im the Apache
Glossary, though, and provide them to all who didn't know them ei-
ther:
http://incubator.apache.org/learn/glossary.html#CommitThenReview
http://incubator.apache.org/learn/glossary.html#ReviewThenCommit
Not trying to start a flamewar, but could the Jakarta Project have
had an influence on the decline? Hal Flynn's [1] points out quite well
that for most server-sided applications Java (and it's clone .NET)
provide a viable platform for secure applications with negligible
impact on performance. And with all the fuss around J2EE (JBoss vs.
Geronimo) a new feature in Apache might not catch as much attention
in the community anymore. OpenSource is a development model which
bases on peer-based ego-gratification and if the incentives to work
on Apache aren't that high anymore, the Apache httpd server might not
be able to attract as many developers anymore. Or to put it in other
words: It's not anymore "cool" to work on Apache.
uuhmm .. <asbestos> or not? No intention to provoke, seriously. We're
all grownup people and our kids are going to read these mailinglists
in CS-history classes, so behave, please ;)
[1] http://www.theregister.co.uk/content/4/33859.html
-daniel
Re: the wheel of httpd-dev life is surely slowing down, solutions please
Posted by Justin Erenkrantz <ju...@erenkrantz.com>.
On Thu, Nov 13, 2003 at 10:29:55AM -0500, Jim Jagielski wrote:
> Another point to consider... With 2.2, module writers will need to
> worry about *3* versions of Apache. Commercial entities which
> have *just* gotten around to porting their 1.3 modules for 2.0
> will likely not bother with 2.2 modules for awhile.
Well, 2.2 modules for the most part should be source-compatible with
2.0. Certain subsystems have changed dramatically (auth being the prime
example I can think of), but I wouldn't expect every module to have to
change to support 2.2. (This is, again, taking my perspective that what
we have in HEAD is roughly what should be in 2.2 modulo testing.)
Ideally, once we start producing 2.2 releases on a regular basis, we
won't be producing 2.0 releases on such a regular basis. -- justin
Re: the wheel of httpd-dev life is surely slowing down, solutions please
Posted by Jim Jagielski <ji...@jagunet.com>.
On Nov 12, 2003, at 8:51 PM, Justin Erenkrantz wrote:
>
> I think our changes that we already have in the tree is about 'right'
> for 2.2 (no big architecture changes, but lots of modules have been
> rewritten and improved). It's just that no one has time or desire to
> shepherd 2.2 to a release. And, I think we need APR 1.0 out the door
> before 2.2 can be rationally discussed.
>
> If we did any major changes from this point that require modules to
> rewrite themselves, we'd need to go to 3.0 per our versioning rules.
> And, I'd *strongly* discourage that. I don't think it's in anyone's
> best interest to revamp our architecture at this stage.
>
> We don't need to give a moving target to our poor module developers.
> Only by producing a series of high-quality releases can we ensure that
> module writers will be confident in writing against 2.0 or 2.2. If we
> come out with 3.x now, I think we'll just end up hurting ourselves in
> the long run even worse than we are now. -- justin
>
Another point to consider... With 2.2, module writers will need to
worry about *3* versions of Apache. Commercial entities which
have *just* gotten around to porting their 1.3 modules for 2.0
will likely not bother with 2.2 modules for awhile.
There's no easy answer... Maybe dev on 2.0 is "slow" simply
because it's "obvious" that it's a dead end. Once 2.1
happened, there was a real concern in putting effort into
2.0 because the gravestone was already being carved for
it.
I do think that some sort of improved communication between
bugs and developers would be good. In general, once a developer
is aware of a bug and/or patch, things get done.
Re: the wheel of httpd-dev life is surely slowing down, solutions please
Posted by Aaron Bannert <aa...@clove.org>.
On Wed, Nov 12, 2003 at 05:51:46PM -0800, Justin Erenkrantz wrote:
> >Creating roles like this is just as bad as having chunks of httpd "owned"
> >by one particular developer. (See below)
>
> I don't think you understand the role of 'patch manager.' On the projects
> I'm involved with, that person's only responsibility is for ensuring that
> the patches posted to the mailing list are entered into the right tool.
>
> I think that can easily be a volunteer, rotating job as you don't need to
> do anything more than that.
I don't know what this hypothetical tool looks like, but why wouldn't
the person submitting the patch just use the tool directly?
Seriously though, until we actually have a tool that we want to use,
this part of the discussion is moot.
> In the patch manager role, nothing would preclude others from adding
> patches to the right tool. But, having someone whose responsibility it is
> to ensure that patches get inputted into the tool on a timely basis is
> goodness. On other projects I'm involved with, that person isn't an even a
> committer. They don't have to understand what the patch does - just make
> sure it is recorded and allow for annotation.
>
> And, it doesn't have to be on a 'timely' basis - once a week, you go
> through and ensure that all [PATCH] messages are in the right patch
> management tool. This lessens the number of dropped patches that we'd
> have. (Bugzilla sucks at this, so I think we'd need a new tool.)
Bugzilla wasn't designed for this, so yes, it sucks at it.
I still maintain that the person submitting the patch should simply
enter it into the patch system directly.
Even easier would be a patch system that monitored the list for [PATCH]
emails and tracked those automatically. We might come up with some standard
syntax to help the patch tracking system know what version of httpd (or
apr or whatever) the patch applies to.
> >Woah woah woah. Are you discouraging people from thinking about big
> >changes for the 2.2 timeframe? If someone has a revolutionary idea for
> >the next generation of Apache HTTPD, who will stand in their way? Not I.
>
> Frankly, yes.
>
> I think our changes that we already have in the tree is about 'right' for
> 2.2 (no big architecture changes, but lots of modules have been rewritten
> and improved). It's just that no one has time or desire to shepherd 2.2 to
> a release. And, I think we need APR 1.0 out the door before 2.2 can be
> rationally discussed.
[tangential issue] Why does it take some much time and desire to make a
release? Shouldn't that be one of the easier things to do around here?
> If we did any major changes from this point that require modules to rewrite
> themselves, we'd need to go to 3.0 per our versioning rules. And, I'd
> *strongly* discourage that. I don't think it's in anyone's best interest
> to revamp our architecture at this stage.
>
> We don't need to give a moving target to our poor module developers. Only
> by producing a series of high-quality releases can we ensure that module
> writers will be confident in writing against 2.0 or 2.2. If we come out
> with 3.x now, I think we'll just end up hurting ourselves in the long run
> even worse than we are now.
Let me pose a different angle: If our module API is so broken that we need
to change it in an incompatible way, then don't you think the module authors
would rather have the improved interface rather than being stuck with the
broken one?
In other words, the only time we actually talk about making drastic changes
is when they are warranted. Therefore, being conservative about API changes
merely serves to discourage creative development and that is _a bad thing_.
-aaron
Re: the wheel of httpd-dev life is surely slowing down, solutions
please
Posted by Justin Erenkrantz <ju...@erenkrantz.com>.
--On Wednesday, November 12, 2003 16:38:11 -0800 Aaron Bannert
<aa...@clove.org> wrote:
>> I believe Sander's suggested that we start a patch manager role
>> (discussion at AC Hackathon!). A few other projects I'm involved with
>> do this. It'd be goodness, I think. But, the problem is finding a
>> volunteer willing to shepherd patches.
>
> -1
>
> Creating roles like this is just as bad as having chunks of httpd "owned"
> by one particular developer. (See below)
I don't think you understand the role of 'patch manager.' On the projects
I'm involved with, that person's only responsibility is for ensuring that
the patches posted to the mailing list are entered into the right tool.
I think that can easily be a volunteer, rotating job as you don't need to
do anything more than that.
> How is this different than "owners" above? You can't expect developers
> on this list to think ahead of time when they might have extra time
> to contribute to this project. Most of the time it is on a whim
> late at night when all of the important chores are taken care of.
> If you ask for people to sign up for a slotted amount of time,
> you'll only end up excluding people who don't work on Apache at
> their day job.
In the patch manager role, nothing would preclude others from adding
patches to the right tool. But, having someone whose responsibility it is
to ensure that patches get inputted into the tool on a timely basis is
goodness. On other projects I'm involved with, that person isn't an even a
committer. They don't have to understand what the patch does - just make
sure it is recorded and allow for annotation.
And, it doesn't have to be on a 'timely' basis - once a week, you go
through and ensure that all [PATCH] messages are in the right patch
management tool. This lessens the number of dropped patches that we'd
have. (Bugzilla sucks at this, so I think we'd need a new tool.)
> Woah woah woah. Are you discouraging people from thinking about big
> changes for the 2.2 timeframe? If someone has a revolutionary idea for
> the next generation of Apache HTTPD, who will stand in their way? Not I.
Frankly, yes.
I think our changes that we already have in the tree is about 'right' for
2.2 (no big architecture changes, but lots of modules have been rewritten
and improved). It's just that no one has time or desire to shepherd 2.2 to
a release. And, I think we need APR 1.0 out the door before 2.2 can be
rationally discussed.
If we did any major changes from this point that require modules to rewrite
themselves, we'd need to go to 3.0 per our versioning rules. And, I'd
*strongly* discourage that. I don't think it's in anyone's best interest
to revamp our architecture at this stage.
We don't need to give a moving target to our poor module developers. Only
by producing a series of high-quality releases can we ensure that module
writers will be confident in writing against 2.0 or 2.2. If we come out
with 3.x now, I think we'll just end up hurting ourselves in the long run
even worse than we are now. -- justin
Re: the wheel of httpd-dev life is surely slowing down, solutions please
Posted by Aaron Bannert <aa...@clove.org>.
On Mon, Nov 10, 2003 at 05:50:46PM -0800, Justin Erenkrantz wrote:
> --On Monday, November 10, 2003 17:14:35 -0800 Stas Bekman <st...@stason.org>
> wrote:
> I believe Sander's suggested that we start a patch manager role (discussion
> at AC Hackathon!). A few other projects I'm involved with do this. It'd
> be goodness, I think. But, the problem is finding a volunteer willing to
> shepherd patches.
-1
Creating roles like this is just as bad as having chunks of httpd "owned"
by one particular developer. (See below)
Having a patch tracking tool would be a much better solution, in my opinion.
>
> >a. certain (most?) chunks of httpd lack owners. If httpd was to have
>
> I really dislike this. The least maintained modules are those that had
> 'owners.' When those owners left, they didn't send a notice saying, 'we're
> leaving.' The code just rotted.
>
> I firmly believe that httpd's group ownership is the right model. Only
> recently have those modules that had 'owners' been cleaned up. (Yay for
> nd!)
+1
> >b. similar to the root rotation we have on the infrastructure, I
> >suggest to have a voluntary weekly rotation of httpd-dev list champion
>
> +1
-1
How is this different than "owners" above? You can't expect developers
on this list to think ahead of time when they might have extra time
to contribute to this project. Most of the time it is on a whim
late at night when all of the important chores are taken care of.
If you ask for people to sign up for a slotted amount of time,
you'll only end up excluding people who don't work on Apache at
their day job.
> Additionally, I'd say that we're best off *slowing* down 2.x development to
> let module authors write modules against 2.0. Changing the world in 2.2
> would be detrimental, I think.
Woah woah woah. Are you discouraging people from thinking about big changes
for the 2.2 timeframe? If someone has a revolutionary idea for the next
generation of Apache HTTPD, who will stand in their way? Not I.
-aaron
Re: the wheel of httpd-dev life is surely slowing down, solutions
please
Posted by Justin Erenkrantz <ju...@erenkrantz.com>.
--On Monday, November 10, 2003 17:14:35 -0800 Stas Bekman <st...@stason.org>
wrote:
> Suggestion: make bugzilla CC bug-reports to the dev list. Most
> developers won't just go and check the bugs database to see if there
Why can't they subscribe to bugs@httpd.apache.org? I don't think throwing
all of the bugs on dev@ is going to help matters. I'd just add a procmail
rule to toss them back into my bugs@httpd folder.
> In my personal opinion, the move to CTR from RTC had very bad
> implications on the design of httpd 2.1.
CTR is still in effect in 2.1. Only RTC is in effect for 2.0, but, IMHO,
that's to try to cover our butts that people don't break the server. I
believe that RTC is very good for maintenance branches that shouldn't break
anything (like our stated goal for 2.0), and CTR is good for development
branches (2.1).
> 2a). Design of new on-going features (and changes/fixes) of the
> existing features is not discussed before it's put to the code. When
> it's committed it's usually too late to have any incentive to give a
> design feedback, after all it's already working and we are too busy
> with other things, so whatever.
Do you have an example?
> The worst part is that it's now easy to sneak in code which otherwise
> would never be accepted (and backport it to 2.0). I don't have any
> examples, but I think the danger is there.
I'm sorry, but when has this happened?
> 2b). As a side-effect when someone asks a design question (e.g. me)
> it's not being answered because people tell: we are in the CRT mode,
> go ahead code it and commit it. But if the poster doesn't know what's
> the right design, if she did she won't have asked in first place.
We've been in CTR for the 'open' branches. You have commit access, so you
have been entrusted with doing the right thing. If you do something wrong,
someone will eventually chime in - as it is CTR. ;-)
> 2c). Future design. There is no clear roadmap of where do we go with
> Apache 2.1. People scratch their itches with new ideas which is cool,
> but if there was a plan to work on things, this may have invited new
> developers to scratch their itches.
I don't believe there was ever a plan for 2.0 either. ;-)
I think 2.0 GA happened because, well, um, a few people that would have
stopped it were out of town. Yet, it happened, so at that point, we
started to be committed to what 2.0 was. When we adopted the binary
compatibility rules, we were fixed even further as to what 2.0 was. Until
those points, I don't think any of us had a plan for 2.0...
I think the binary compatibility rules were essential and, in retrospect,
we should have had them in place before we went GA in 2.0. Yet, we know
better now. 2.2 shouldn't have those same mistakes when we do that.
> 2d). CRT seemed to come as a replacement for design discussions. It's
> very easy to observe from the traffic numbers:
Not sure why you think there's a correlation here. Statistics can be
misleading. =)
> You don't need a fancy graph to see a clear decline in the last 1.5
> years. Quite a few people suggested that this is due to the market
> decline. I won't doubt them, but it's quite possible that the market
People change jobs or whatnot. Committers are only expected to contribute
when they have time. It's not like we're all getting paid to work on this.
I'm certainly not.
I confess that I don't have the time I used to, but I'm also involved
'behind the scenes' within the ASF more than I was a few years ago. So,
perhaps my time committment to the ASF is the same, but it's spread out
amongst other projects/worthy causes. Working on the same thing for years
on end can be a tad disconcerting. We all need a break at times.
> I don't have numbers to support my clause, but I have a strong feeling
> that nowadays we see a much smaller number of posts with contributions
> from non-developers, since most contributions are simply ignored and
Perhaps.
I believe Sander's suggested that we start a patch manager role (discussion
at AC Hackathon!). A few other projects I'm involved with do this. It'd
be goodness, I think. But, the problem is finding a volunteer willing to
shepherd patches.
> a. certain (most?) chunks of httpd lack owners. If httpd was to have
I really dislike this. The least maintained modules are those that had
'owners.' When those owners left, they didn't send a notice saying, 'we're
leaving.' The code just rotted.
I firmly believe that httpd's group ownership is the right model. Only
recently have those modules that had 'owners' been cleaned up. (Yay for
nd!)
> b. similar to the root rotation we have on the infrastructure, I
> suggest to have a voluntary weekly rotation of httpd-dev list champion
+1
> c. turn the dealing with contributions and bugs into a sexy
> thing. Everybody wants to feed their ego, there is no better way to
> make your ego happy then getting a positive feedback from a person you
> have helped to resolve a bug or handhold to provide a good patch for
> the new feature, they spent so much time working on.
Sure, but that requires a time committment. That's not always possible to
expect from people with 'high list karma.'
> (even though that technically they are still committers and all) the
> those who join the project. Which obviously has clear effects on the
> productivity of the dev list.
>From my perspective, httpd-2.x does what I want it to at this point. Are
there some things that I'd like cleaned up? Sure. But, my personal
opinion is that the codebase is in better quality now than it was 2+ years
ago (or whenever I first joined).
Additionally, I'd say that we're best off *slowing* down 2.x development to
let module authors write modules against 2.0. Changing the world in 2.2
would be detrimental, I think.
> all. It's very easy to see that questions from most known httpd
> developers almost always have a long thread with resolution. Other
> posters are usually lucky to get any responses at all, and if they do
> usually the thread would die out without any resolutions.
I'd say that's because we know we must be persistent, AND there's also a
personal motivation to resolving the issues at hand ("scratch our own
itch").
> do". I'd like to suggest that it's a very own httpd problem, because
> usually you get questions from 3rd party developers when something in
> httpd doesn't cooperate with those 3rd party modules and needs to be
Sure.
> fixed. I think it's much less important to work on Apache 2.1 and much
> more important to polish 2.0 and make 3rd party developers
> happy. (wearing the 3rd party module developer hat) we don't need no
Um, 2.0 has fixed binary compat rules, so you can't make any changes that
break that.
> demands. So telling them: "hey, we gave you a commit access, just go
> and code/fix it". is not always working.
(I realize I've probably told you that line you are quoting above on
several occassions.)
In all seriousness, if that's the case then you shouldn't accept being a
committer. Scratching your own itch is how I believe it should work. I'm
sorry to hear that you dislike that, but I don't know what we can do to
help *you* specifically. You've got commit access (and on the PMC, to
boot). Ultimately, I don't think there are any roadblocks that we're
imposing on you. And, I think that's the best we can do. -- justin
Re: the wheel of httpd-dev life is surely slowing down, solutions please
Posted by Colm MacCarthaigh <co...@stdlib.net>.
On Mon, Nov 10, 2003 at 05:14:35PM -0800, Stas Bekman wrote:
> 3). Contributions
>
> I don't have numbers to support my clause, but I have a strong feeling
> that nowadays we see a much smaller number of posts with contributions
> from non-developers
More facetious than anything else, I'm going to hijack this part
to make a small suggestion; give non-committers the option of
not having their e-mail in the CHANGES/STATUS files, or at least
of having them obfuscated in some fashion.
The ammount of spam I get due to the httpd CHANGES file is simply
unreal, most of it nonsense - with the forged from headers of other
users of this list.
That's about the only thing that ever makes me think twice about
posting a patch/contrib.
PS. This is not a complaint, it's my own silly fault for never
remembering to ask someone about this.
--
Colm MacCárthaigh Public Key: colm+pgp@stdlib.net
Re: the wheel of httpd-dev life is surely slowing down, solutions
please
Posted by Daniel Lorch <ml...@lorch.cc>.
hi,
> 2d). CRT seemed to come as a replacement for design discussions. It's
> very easy to observe from the traffic numbers:
Please excuse the total ignorance of passive Apache-Dev readers, but
these abbreviations were new to me. I've found then im the Apache
Glossary, though, and provide them to all who didn't know them ei-
ther:
http://incubator.apache.org/learn/glossary.html#CommitThenReview
http://incubator.apache.org/learn/glossary.html#ReviewThenCommit
Not trying to start a flamewar, but could the Jakarta Project have
had an influence the decline? Hal Flynn's [1] points out quite well
that for most server-sided applications Java (and it's clone .NET)
provide a viable platform for secure applications with negligible
impact on performance. And with all the fuss around J2EE (JBoss vs.
Geronimo) a new feature in Apache might not catch as much attention
in the community anymore. OpenSource is a development model which
bases on peer-based ego-gratification and if the incentives to work
on Apache aren't that high anymore, the Apache httpd server might not
be able to attract as many developers anymore. Or to put it in other
words: It's not anymore "cool" to work on Apache.
uuhmm .. <asbestos> or not? No intention to provoke, seriously. We're
all grownup people and our kids are going to read these mailinglists
in CS-history classes, so behave, please ;)
[1] http://www.theregister.co.uk/content/4/33859.html
-daniel
Re: the wheel of httpd-dev life is surely slowing down, solutions
please
Posted by Jeff Trawick <tr...@attglobal.net>.
Stas Bekman wrote:
> 1) Bugs
>
> searching for NEW and REOPENED bugs in httpd-2.0 returns: 420 entries
> http://nagoya.apache.org/bugzilla/buglist.cgi?bug_status=NEW&bug_status=REOPENED&product=Apache+httpd-2.0
yes, far too many :(
> Suggestion: make bugzilla CC bug-reports to the dev list. Most
> developers won't just go and check the bugs database to see if there
> is something to fix, both because they don't have time and because
> there are too many things to fix. Posting the reports to the list
> raises a chance for the developer to have a free minute and may be to
> resolve the bug immediately or close it.
That would reduce the number of subscribers to dev@, which is the opposite of
what we need. At least we have a monthy summary posted to dev@ which reminds
folks that there is a bug db. Also, it is my interpretation of bug db traffic
that Joshua and Andre' already jump on the PRs that can be resolved
immediately. Many of the outstanding PRs are quite difficult to resolve.
Working the bug db is a noble effort, but many in our community have no time to
work it. The way to get more time being spent on the bug db is to take the
kind of actions that increase the number of httpd developers.
> 2) Lack of Design
>
> 2a). Design of new on-going features (and changes/fixes) of the
> existing features is not discussed before it's put to the code. When
> it's committed it's usually too late to have any incentive to give a
> design feedback, after all it's already working and we are too busy
> with other things, so whatever.
We leave it up to the developer to judge whether some design issue needs to be
discussed first. If something is committed prior to being discussed, then the
person with the objection must start the discussion at that point. This is a
tool against stagnation. If somebody has the time/energy to work on something,
we can't put up a roadblock just because nobody else has the time/skills to
discuss it with them. The only roadblock is the one to protect users of stable
releases.
> The worst part is that it's now easy to sneak in code which otherwise
> would never be accepted (and backport it to 2.0). I don't have any
> examples, but I think the danger is there.
There is a barrier to getting things backported to 2.0 as a protection against
possible drawbacks of C-T-R.
If three people are in favor of putting something in the stable branch,
mistakes can still happen, but I don't see how we can refer to such a backport
as sneaking in code.
> 2b). As a side-effect when someone asks a design question (e.g. me)
> it's not being answered because people tell: we are in the CRT mode,
> go ahead code it and commit it. But if the poster doesn't know what's
> the right design, if she did she won't have asked in first place.
Alternate opinion: If a question is not answered, it is because nobody has
time/skills.
As mentioned above, I see the C-T-R mode for 2.1-dev as a tool that helps
work-around a lack of energy/skills on the list.
> 2c). Future design. There is no clear roadmap of where do we go with
> Apache 2.1. People scratch their itches with new ideas which is cool,
> but if there was a plan to work on things, this may have invited new
> developers to scratch their itches.
>
> 2d). CRT seemed to come as a replacement for design discussions. It's
> very easy to observe from the traffic numbers:
As Bill said, it has been this way for some time. At some point we created a
stable branch for 2.0-dev, but 2.1-dev is handled in the same way that 2.x was
handled all along.
> 3). Contributions
>
> I don't have numbers to support my clause, but I have a strong feeling
> that nowadays we see a much smaller number of posts with contributions
> from non-developers
I can believe it.
> Solutions:
>
> a. certain (most?) chunks of httpd lack owners. If httpd was to have
> clear owners of certain code bases then it'll be easy to take care of
> bug reports and contributions/patches. Even though httpd is
> collectively owned, it doesn't preclude champions in certain areas,
> who can see that good things happen to the areas they overlook.
>
> b. similar to the root rotation we have on the infrastructure, I
> suggest to have a voluntary weekly rotation of httpd-dev list champion
> whose responsibility is to make sure that most (all?) bug-reports and
> contributions are dealts with: assigned to those who those champions
> who know how to fix/review/adopt things (See (a) above), asking for
> more input if needed, etc. You don't have to know httpd as your five
> fingers to be able to do a great rewarding job.
Personally I'm very disinterested in a system that puts certain responsibility
on a single person for an area of the code or for a period of time. Work/life
issues arise from time to time that force folks to disappear for a stretch. To
the greatest extent possible, any policies/procedures we have should have the
goal of channelling volunteer efforts instead of placing certain
responsibilities with a single person.
It would be good to have a single place other than the mailing list (2.1-dev
STATUS or bug db or something else) where patches are tracked and where
contributors can know the status. Somebody brought up the idea of "patch
manager." I'm interested in hearing more about that. Anything that focuses
the available light on these contributions would be very helpful.
> 3a) I can hardly see new developers joining the team, should we blame
> the economy or the lack of encouragement for people to contribute.
perhaps some of both... One of these we can't take action on anyway. All we
can try to do is be gracious with contributions. Sooner or later that will
result in new people devoting significant time to httpd.
> 4). Feedback Preference and List Karma
Without agreeing or disagreeing with anything you wrote here, I'm not sure what
possible actions could be. Any action should be consistent with the idea that
httpd developers are all volunteers and any effort they are able to make is to
be valued. But it doesn't bring a healthy community to finish with saying
* If an httpd developer is only interested in 2.1-dev, so what?
* If an httpd developer is only interested in a certain functional area of the
code, so what?
* If an httpd developer ignores the bug db, so what?
* If an httpd developer ignores dev@ posts that they don't think is something
for httpd to worry about, so what?
Somehow as a group we have to help new folks who are willing to do the work to
fix/enhance httpd in a way that is beneficial to others. Having a better way
to track patches and make sure that would-be contributers get feedback is a
step in the right direction.
RE: the wheel of httpd-dev life is surely slowing down, solutions please
Posted by "Peter J. Cranstone" <cr...@msn.com>.
Thanks for the stat - our environment supports throughputs >1Gbps on a
single processor. Near linear scalability is expected with additional chips
up to 256 CPU's.
I agree with your comments on IPv6 - it's already here - might as well
embrace the horror.
Regards,
Peter
-----Original Message-----
From: Colm MacCarthaigh,,, [mailto:colmmacc@stdlib.net] On Behalf Of Colm
MacCarthaigh
Sent: Tuesday, November 11, 2003 9:53 AM
To: dev@httpd.apache.org
Subject: Re: the wheel of httpd-dev life is surely slowing down, solutions
please
On Tue, Nov 11, 2003 at 06:02:36AM -0700, Peter J. Cranstone wrote:
> So, anyone got any hard data that shows Apache 2.x serving pages "factors"
> faster than 1.x?
Yes, plenty :) ftp.heanet.ie serves about 1 million requests, well over
a terabyte of data per day and maintains an average of about 220Mbps for
http. It's roughly 10 times less latent, and serves about 5 times more per
second under 2.0 than 1.3, and I benchmarked that in the early 2.0 days.
Performance under 2.x has improved since.
If I could use sendfile (my hardware is still broken in IPv6) I'd see
a much bigger increase in performance. 2.x knocks the socks off of 1.x.
I've benchmarked many other transitions, and though the improvement
is smaller for dynamic content I've never seen the numbers get worse.
Of course this is all going from 1.x prefork to a 2.0 worker mpm.
More importantly though; 2.x has IPv6 support. And whilst many people
reading this mail may think IPv6 is an obscure requirement, many of us
are in parts of internet where it's de facto. We couldn't even consider
rolling out an app which didn't have reliable IPv6 support. Many people
are finding themselves in such parts of the internet, at an increasing
rate.
As an asside; It's been my experience that most of industry doesn't
rate application performance all that highly in evaluation criteria
(mainly because buying beefier hardware is an easier solution).
--
Colm MacCárthaigh Public Key: colm+pgp@stdlib.net
Re: the wheel of httpd-dev life is surely slowing down, solutions please
Posted by Colm MacCarthaigh <co...@stdlib.net>.
On Tue, Nov 11, 2003 at 06:02:36AM -0700, Peter J. Cranstone wrote:
> So, anyone got any hard data that shows Apache 2.x serving pages "factors"
> faster than 1.x?
Yes, plenty :) ftp.heanet.ie serves about 1 million requests, well over
a terabyte of data per day and maintains an average of about 220Mbps for
http. It's roughly 10 times less latent, and serves about 5 times more per
second under 2.0 than 1.3, and I benchmarked that in the early 2.0 days.
Performance under 2.x has improved since.
If I could use sendfile (my hardware is still broken in IPv6) I'd see
a much bigger increase in performance. 2.x knocks the socks off of 1.x.
I've benchmarked many other transitions, and though the improvement
is smaller for dynamic content I've never seen the numbers get worse.
Of course this is all going from 1.x prefork to a 2.0 worker mpm.
More importantly though; 2.x has IPv6 support. And whilst many people
reading this mail may think IPv6 is an obscure requirement, many of us
are in parts of internet where it's de facto. We couldn't even consider
rolling out an app which didn't have reliable IPv6 support. Many people
are finding themselves in such parts of the internet, at an increasing
rate.
As an asside; It's been my experience that most of industry doesn't
rate application performance all that highly in evaluation criteria
(mainly because buying beefier hardware is an easier solution).
--
Colm MacCárthaigh Public Key: colm+pgp@stdlib.net
RE: the wheel of httpd-dev life is surely slowing down, solutions please
Posted by "Peter J. Cranstone" <cr...@msn.com>.
Something else to think about...
What's the differentiator "in the market place" between 1.x and 2.x?
(hint: it's not a feature list)
If I was to go out and buy Apache (Covalent) apart from some management
tools (features) what's the biggest differentiator between the old version
(public domain 1.x) and the new version (2.x.)
Invariably three things come to mind - performance, stability,
compatibility?
So is Apache 2.x faster or slower than 1.x? Has anyone shown definitively
that 2.x can serve pages faster than 1.x? and if so by what percentage or
factor?
Looks like 2.1 is going to be more stable than the old 2.x versions but is
it more stable than 1.x?
Finally compatibility? Can I run my 1.x modules unmodified in a 2.x
environment?
HTTPD dev life is slowing down - simply fixing bugs and adding new features
is not the way to differentiate, there's probably 50,000 users of 2.x while
there's millions and millions of 1.x users.
What's it going to take to shift them to 2.x?
Not features (only the minority of users now care) - not compatibility
(although it helps) - my bet is raw performance.
So, anyone got any hard data that shows Apache 2.x serving pages "factors"
faster than 1.x?
Regards,
Peter
-----Original Message-----
From: William A. Rowe, Jr. [mailto:wrowe@apache.org]
Sent: Tuesday, November 11, 2003 2:00 AM
To: dev@httpd.apache.org
Cc: dev@httpd.apache.org
Subject: Re: the wheel of httpd-dev life is surely slowing down, solutions
please
At 07:14 PM 11/10/2003, Stas Bekman wrote:
>I have several reasons to believe that the wheel of httpd-dev life is
>slowing down and something has to be done to get this wheel up to the
>speed like in the good old days. The following observation are listed
>in no particular order. I've also tried to offer suggestions how to
>resolve problems.
Always good to discuss, but httpd is about folks contributing and the team
reviewing patches. When there are lots of activity, things move quickly.
Summer is a slow period for alot of projects. Stability slows things down
too (and yes, 2.0.48 is far more stable than anything released before on
the 2.0 branch.)
>1) Bugs
>
>searching for NEW and REOPENED bugs in httpd-2.0 returns: 420 entries
>http://nagoya.apache.org/bugzilla/buglist.cgi?bug_status=NEW&bug_status=REO
PENED&product=Apache+httpd-2.0
Yes - these are important, I'm the first to admit I have a number I track
that
I haven't resolved. And more eyes are needed :) However...
>Suggestion: make bugzilla CC bug-reports to the dev list. Most
>developers won't just go and check the bugs database to see if there
>is something to fix, both because they don't have time and because
>there are too many things to fix. Posting the reports to the list
>raises a chance for the developer to have a free minute and may be to
>resolve the bug immediately or close it.
Nope, it assures a good number of interested contributors SHUT DOWN
their subscriptions to httpd-dev. Trust me, I subscribe to bug traffic as
well.
Guess which of the two I read daily? If I didn't split them, I would never
be caught up on the developer discussions. (Even those I don't understand
completely I do read.)
>2) Lack of Design
>
>In my personal opinion, the move to CTR from RTC had very bad
>implications on the design of httpd 2.1.
You seem to be misinformed here... Apache 2.0 was ALWAYS CTR
until we decided to put a roadblock in the way of breaking the stable
version, and encourage that experimentation to move to an unstable
branch. RTC was only introduced after the last ApacheCon. And very
very resisted, at that.
>2a). Design of new on-going features (and changes/fixes) of the
>existing features is not discussed before it's put to the code. When
>it's committed it's usually too late to have any incentive to give a
>design feedback, after all it's already working and we are too busy
>with other things, so whatever.
However, "Show me the code" remains the mantra of httpd. So continuing
to use CTR, the code is in place. With httpd-test/perl-framework the coder
can ever verify that their new change doesn't break any tests that are
already defined.
>The worst part is that it's now easy to sneak in code which otherwise
>would never be accepted (and backport it to 2.0). I don't have any
>examples, but I think the danger is there.
That's always been true in httpd. We didn't lock 1.3 until 2.0 was nearing
completion. We didn't lock 2.0 until 2.1 was available for folks to hack
on.
It is VERY hard to get code into 2.0. That code is RTC. Gratuitous changes
and new features aren't encouraged. Time to move on to 2.1 for all of the
great and wonderful things our web server aught to do. In fact, it's why
I've
suggested point blank that the 'which pass at the startup file' patch you
offered
will never belong in 2.0 - anyone designing for an arbitrary 2.0 can't even
count
on such a feature. But anyone using the future 2.2 release should trust
that
it is present.
>2b). As a side-effect when someone asks a design question (e.g. me)
>it's not being answered because people tell: we are in the CRT mode,
>go ahead code it and commit it. But if the poster doesn't know what's
>the right design, if she did she won't have asked in first place.
Nothing stops anyone from posting up code first. Many of us post up
code when we just aren't sure if it's the one best answer - and although
it takes folks a while sometimes to review, the dialog is useful. But that
discussion shouldn't be an obstacle to writing code.
>2c). Future design. There is no clear roadmap of where do we go with
>Apache 2.1. People scratch their itches with new ideas which is cool,
>but if there was a plan to work on things, this may have invited new
>developers to scratch their itches.
The roadmap is this. When httpd-2.1 solves enough issues that 2.0 cannot
(such as a major auth reorganization, perhaps auth credential chains, and
hopefully some improvements to the filtering schema and file system
provider)
AND it can pass the regressions as gracefully as 2.0 - we are ready to start
with some alphas. All someone has to do is speak up and say it's time.
No fixed lists of features set in stone, no. If that *must have* feature
isn't
done, then it moves on to 2.3 to await inclusion in 2.4. It all depends on
what folks can dedicate their time to.
>2d). CRT seemed to come as a replacement for design discussions. It's
>very easy to observe from the traffic numbers:
>[snip]
>Quiz: based on this input, tell which date CRT was introduced at.
CRT was always there.
The discrepancy was that 2.0 was a RADICAL overhaul :)
So early in development, folks raised many, many invaluable questions
about a whole new design. httpd-2.0 (and 2.1) are not being changed
nearly as radically.
Remember httpd is also the sum of httpd-dev and apr-dev. And that the
presence (at last) of a users list has pulled some burden off of httpd-dev.
>3). Contributions
>
>I don't have numbers to support my clause, but I have a strong feeling
>that nowadays we see a much smaller number of posts with contributions
>from non-developers, since most contributions are simply ignored and
>it's a known thing among Apache community that posting fixes and
>suggestions to httpd-dev list is usually a waste of time. (This is
>based on my personal experience and discussions with other developers
>who aren't httpd-core coders). I realize that I'm not entirely correct
>saying that, since some things are picked up, so I apologize to those
>folks who do try hard to pick those things.
I've heard this complaint too, and felt guilty about all of the interesting
and
worthwhile patches that I personally did not have time to address. Many
still sit in my dev@httpd box awaiting free cycles (marked priority-urgent.)
>The obvious problem is that most of those things are unsexy, usually
>don't fit someones itch and they sometimes require a lot of
>communication overhead with the reporter/contributor just to
>understand what exactly is going on.
A problem in every open source project, and certainly one that we don't have
any obvious answer to.
>Solutions:
>
>a. certain (most?) chunks of httpd lack owners. If httpd was to have
>clear owners of certain code bases then it'll be easy to take care of
>bug reports and contributions/patches. Even though httpd is
>collectively owned, it doesn't preclude champions in certain areas,
>who can see that good things happen to the areas they overlook.
We do have champions, but many are preoccupied (I have mod_isapi
patches just sitting there waiting to be reviewed and finally collected into
a proof set that the replacement patches work for the existing cases plus
all the edge cases folks have discovered.)
>b. similar to the root rotation we have on the infrastructure, I
>suggest to have a voluntary weekly rotation of httpd-dev list champion
>whose responsibility is to make sure that most (all?) bug-reports and
>contributions are dealts with: assigned to those who those champions
>who know how to fix/review/adopt things (See (a) above), asking for
>more input if needed, etc. You don't have to know httpd as your five
>fingers to be able to do a great rewarding job.
We do, folks pick up the ball and bust through those reports for a while,
then lose time/interest/energy to keep up. It's up to others to step up,
or have patience.
>c. turn the dealing with contributions and bugs into a sexy
>thing. Everybody wants to feed their ego, there is no better way to
>make your ego happy then getting a positive feedback from a person you
>have helped to resolve a bug or handhold to provide a good patch for
>the new feature, they spent so much time working on.
Of course, I recognize this, I hope others do too. I'm proud to have
shepherded
many new contributor's patches into the code.
>3a) I can hardly see new developers joining the team, should we blame
>the economy or the lack of encouragement for people to contribute. I
>think we now have a higher rate of developers who leave the project
>(even though that technically they are still committers and all) the
>those who join the project. Which obviously has clear effects on the
>productivity of the dev list.
We have a high barrier to entry, because we have so many users who
count on the project. We are always open to new suggestions, and these
belong on the pmc list.
>4). Feedback Preference and List Karma
>
>List Karma is a known issue in all Open Source projects. It's obvious
>that known and respected members of the httpd-dev list come to a
>higher preference when it comes to posting feedback and any answers at
>all. It's very easy to see that questions from most known httpd
>developers almost always have a long thread with resolution. Other
>posters are usually lucky to get any responses at all, and if they do
>usually the thread would die out without any resolutions.
That's unfair :-) My comments are just as ignored as yours, or Joe
Newauthor
who dropped into the list once, I believe. The relationship between posts
and
answers has more to do with the number of literate developers who understand
the area being discussed, and understand the contributor's comments.
>I'm not questioning the validity of the list karma, this is something
>that I happen to stick to myself on the lists where I can give useful
>feedback: I usually answer first questions from people whom I know for
>good and bad. What I want to talk about is about the karma of the
>posters who don't ask to solve their "personal" problems, but act on
>behalf sub-projects built upon httpd. These posters try to solve a
>problem for hundreds, thousands and sometimes millions of people, who
>won't use Apache at all if they all they needed is to serve static
>pages -- there are several other web servers which outperform Apache
>on this front. They use Apache because of the 3rd party modules that
>build upon Apache. It's those 3rd party modules that Apache should be
>thankful to to its current world domination in the web server sector.
Undoubtedly.
>Therefore I think httpd developers should stop thinking, "this
>poster's question is not httpd's problem, I have better things to
>do". I'd like to suggest that it's a very own httpd problem, because
>usually you get questions from 3rd party developers when something in
>httpd doesn't cooperate with those 3rd party modules and needs to be
>fixed. I think it's much less important to work on Apache 2.1 and much
>more important to polish 2.0 and make 3rd party developers
>happy. (wearing the 3rd party module developer hat) we don't need no
>Apache 2.1 in the sky, we need Apache 2.0 on steady on the ground, so
>we can run our modules on.
We have 2.0 - we bolted it to the wall so that mod_newfeature.so would
load on 2.0.43, or 2.0.99. We are diverting the creative (read - not yet
fully
though out) energies to an unstable branch that assures 2.0 is almost always
ready for release when someone has the impetus to cut another release.
>and if I may add that giving those 3rd party developers a commit
>access is not a solution to the problem, because they hardly survive
>coping with their own projects' bugs, support issues and feature
>demands. So telling them: "hey, we gave you a commit access, just go
>and code/fix it". is not always working.
Commit access isn't lightly granted either, so this certainly isn't an
option
that is likely to be entertained.
Let me reiterate about httpd-2.1. It will never be released, it is the
sandbox
that leads us to 2.2 - when it's close. That means we must beg our third
party module authors to hack in the workaround for 2.0 - and look forward
and propose those changes they want that will make 2.2 the answer to all
of their problems.
In reality, we will see a handful of patches to 2.0 that won't ever be
applied
to that tree, but should be considered for immediate inclusion in 2.1 so
that
we can hand it back and ask, "Here - a new version - does your module work
better with our next release?" We hope, of course the answer is yes.
And when we make changes to 2.1 - we need to be the community to try
the code, and offer up "No!!! That messes up my beautiful design for
mod_foo!"
Tell us early so we offer the best platform for supporting third party
modules.
Stas, thanks for raising these issues, and I hope your comments do generate
some additional activity, around making 2.0.49 even more stable, and around
improving the software so that the 2.2.0 release solves hosts of issues.
Bill
Re: the wheel of httpd-dev life is surely slowing down,
solutions please
Posted by "William A. Rowe, Jr." <wr...@apache.org>.
At 07:14 PM 11/10/2003, Stas Bekman wrote:
>I have several reasons to believe that the wheel of httpd-dev life is
>slowing down and something has to be done to get this wheel up to the
>speed like in the good old days. The following observation are listed
>in no particular order. I've also tried to offer suggestions how to
>resolve problems.
Always good to discuss, but httpd is about folks contributing and the team
reviewing patches. When there are lots of activity, things move quickly.
Summer is a slow period for alot of projects. Stability slows things down
too (and yes, 2.0.48 is far more stable than anything released before on
the 2.0 branch.)
>1) Bugs
>
>searching for NEW and REOPENED bugs in httpd-2.0 returns: 420 entries
>http://nagoya.apache.org/bugzilla/buglist.cgi?bug_status=NEW&bug_status=REOPENED&product=Apache+httpd-2.0
Yes - these are important, I'm the first to admit I have a number I track that
I haven't resolved. And more eyes are needed :) However...
>Suggestion: make bugzilla CC bug-reports to the dev list. Most
>developers won't just go and check the bugs database to see if there
>is something to fix, both because they don't have time and because
>there are too many things to fix. Posting the reports to the list
>raises a chance for the developer to have a free minute and may be to
>resolve the bug immediately or close it.
Nope, it assures a good number of interested contributors SHUT DOWN
their subscriptions to httpd-dev. Trust me, I subscribe to bug traffic as well.
Guess which of the two I read daily? If I didn't split them, I would never
be caught up on the developer discussions. (Even those I don't understand
completely I do read.)
>2) Lack of Design
>
>In my personal opinion, the move to CTR from RTC had very bad
>implications on the design of httpd 2.1.
You seem to be misinformed here... Apache 2.0 was ALWAYS CTR
until we decided to put a roadblock in the way of breaking the stable
version, and encourage that experimentation to move to an unstable
branch. RTC was only introduced after the last ApacheCon. And very
very resisted, at that.
>2a). Design of new on-going features (and changes/fixes) of the
>existing features is not discussed before it's put to the code. When
>it's committed it's usually too late to have any incentive to give a
>design feedback, after all it's already working and we are too busy
>with other things, so whatever.
However, "Show me the code" remains the mantra of httpd. So continuing
to use CTR, the code is in place. With httpd-test/perl-framework the coder
can ever verify that their new change doesn't break any tests that are
already defined.
>The worst part is that it's now easy to sneak in code which otherwise
>would never be accepted (and backport it to 2.0). I don't have any
>examples, but I think the danger is there.
That's always been true in httpd. We didn't lock 1.3 until 2.0 was nearing
completion. We didn't lock 2.0 until 2.1 was available for folks to hack on.
It is VERY hard to get code into 2.0. That code is RTC. Gratuitous changes
and new features aren't encouraged. Time to move on to 2.1 for all of the
great and wonderful things our web server aught to do. In fact, it's why I've
suggested point blank that the 'which pass at the startup file' patch you offered
will never belong in 2.0 - anyone designing for an arbitrary 2.0 can't even count
on such a feature. But anyone using the future 2.2 release should trust that
it is present.
>2b). As a side-effect when someone asks a design question (e.g. me)
>it's not being answered because people tell: we are in the CRT mode,
>go ahead code it and commit it. But if the poster doesn't know what's
>the right design, if she did she won't have asked in first place.
Nothing stops anyone from posting up code first. Many of us post up
code when we just aren't sure if it's the one best answer - and although
it takes folks a while sometimes to review, the dialog is useful. But that
discussion shouldn't be an obstacle to writing code.
>2c). Future design. There is no clear roadmap of where do we go with
>Apache 2.1. People scratch their itches with new ideas which is cool,
>but if there was a plan to work on things, this may have invited new
>developers to scratch their itches.
The roadmap is this. When httpd-2.1 solves enough issues that 2.0 cannot
(such as a major auth reorganization, perhaps auth credential chains, and
hopefully some improvements to the filtering schema and file system provider)
AND it can pass the regressions as gracefully as 2.0 - we are ready to start
with some alphas. All someone has to do is speak up and say it's time.
No fixed lists of features set in stone, no. If that *must have* feature isn't
done, then it moves on to 2.3 to await inclusion in 2.4. It all depends on
what folks can dedicate their time to.
>2d). CRT seemed to come as a replacement for design discussions. It's
>very easy to observe from the traffic numbers:
>[snip]
>Quiz: based on this input, tell which date CRT was introduced at.
CRT was always there.
The discrepancy was that 2.0 was a RADICAL overhaul :)
So early in development, folks raised many, many invaluable questions
about a whole new design. httpd-2.0 (and 2.1) are not being changed
nearly as radically.
Remember httpd is also the sum of httpd-dev and apr-dev. And that the
presence (at last) of a users list has pulled some burden off of httpd-dev.
>3). Contributions
>
>I don't have numbers to support my clause, but I have a strong feeling
>that nowadays we see a much smaller number of posts with contributions
>from non-developers, since most contributions are simply ignored and
>it's a known thing among Apache community that posting fixes and
>suggestions to httpd-dev list is usually a waste of time. (This is
>based on my personal experience and discussions with other developers
>who aren't httpd-core coders). I realize that I'm not entirely correct
>saying that, since some things are picked up, so I apologize to those
>folks who do try hard to pick those things.
I've heard this complaint too, and felt guilty about all of the interesting and
worthwhile patches that I personally did not have time to address. Many
still sit in my dev@httpd box awaiting free cycles (marked priority-urgent.)
>The obvious problem is that most of those things are unsexy, usually
>don't fit someones itch and they sometimes require a lot of
>communication overhead with the reporter/contributor just to
>understand what exactly is going on.
A problem in every open source project, and certainly one that we don't have
any obvious answer to.
>Solutions:
>
>a. certain (most?) chunks of httpd lack owners. If httpd was to have
>clear owners of certain code bases then it'll be easy to take care of
>bug reports and contributions/patches. Even though httpd is
>collectively owned, it doesn't preclude champions in certain areas,
>who can see that good things happen to the areas they overlook.
We do have champions, but many are preoccupied (I have mod_isapi
patches just sitting there waiting to be reviewed and finally collected into
a proof set that the replacement patches work for the existing cases plus
all the edge cases folks have discovered.)
>b. similar to the root rotation we have on the infrastructure, I
>suggest to have a voluntary weekly rotation of httpd-dev list champion
>whose responsibility is to make sure that most (all?) bug-reports and
>contributions are dealts with: assigned to those who those champions
>who know how to fix/review/adopt things (See (a) above), asking for
>more input if needed, etc. You don't have to know httpd as your five
>fingers to be able to do a great rewarding job.
We do, folks pick up the ball and bust through those reports for a while,
then lose time/interest/energy to keep up. It's up to others to step up,
or have patience.
>c. turn the dealing with contributions and bugs into a sexy
>thing. Everybody wants to feed their ego, there is no better way to
>make your ego happy then getting a positive feedback from a person you
>have helped to resolve a bug or handhold to provide a good patch for
>the new feature, they spent so much time working on.
Of course, I recognize this, I hope others do too. I'm proud to have shepherded
many new contributor's patches into the code.
>3a) I can hardly see new developers joining the team, should we blame
>the economy or the lack of encouragement for people to contribute. I
>think we now have a higher rate of developers who leave the project
>(even though that technically they are still committers and all) the
>those who join the project. Which obviously has clear effects on the
>productivity of the dev list.
We have a high barrier to entry, because we have so many users who
count on the project. We are always open to new suggestions, and these
belong on the pmc list.
>4). Feedback Preference and List Karma
>
>List Karma is a known issue in all Open Source projects. It's obvious
>that known and respected members of the httpd-dev list come to a
>higher preference when it comes to posting feedback and any answers at
>all. It's very easy to see that questions from most known httpd
>developers almost always have a long thread with resolution. Other
>posters are usually lucky to get any responses at all, and if they do
>usually the thread would die out without any resolutions.
That's unfair :-) My comments are just as ignored as yours, or Joe Newauthor
who dropped into the list once, I believe. The relationship between posts and
answers has more to do with the number of literate developers who understand
the area being discussed, and understand the contributor's comments.
>I'm not questioning the validity of the list karma, this is something
>that I happen to stick to myself on the lists where I can give useful
>feedback: I usually answer first questions from people whom I know for
>good and bad. What I want to talk about is about the karma of the
>posters who don't ask to solve their "personal" problems, but act on
>behalf sub-projects built upon httpd. These posters try to solve a
>problem for hundreds, thousands and sometimes millions of people, who
>won't use Apache at all if they all they needed is to serve static
>pages -- there are several other web servers which outperform Apache
>on this front. They use Apache because of the 3rd party modules that
>build upon Apache. It's those 3rd party modules that Apache should be
>thankful to to its current world domination in the web server sector.
Undoubtedly.
>Therefore I think httpd developers should stop thinking, "this
>poster's question is not httpd's problem, I have better things to
>do". I'd like to suggest that it's a very own httpd problem, because
>usually you get questions from 3rd party developers when something in
>httpd doesn't cooperate with those 3rd party modules and needs to be
>fixed. I think it's much less important to work on Apache 2.1 and much
>more important to polish 2.0 and make 3rd party developers
>happy. (wearing the 3rd party module developer hat) we don't need no
>Apache 2.1 in the sky, we need Apache 2.0 on steady on the ground, so
>we can run our modules on.
We have 2.0 - we bolted it to the wall so that mod_newfeature.so would
load on 2.0.43, or 2.0.99. We are diverting the creative (read - not yet fully
though out) energies to an unstable branch that assures 2.0 is almost always
ready for release when someone has the impetus to cut another release.
>and if I may add that giving those 3rd party developers a commit
>access is not a solution to the problem, because they hardly survive
>coping with their own projects' bugs, support issues and feature
>demands. So telling them: "hey, we gave you a commit access, just go
>and code/fix it". is not always working.
Commit access isn't lightly granted either, so this certainly isn't an option
that is likely to be entertained.
Let me reiterate about httpd-2.1. It will never be released, it is the sandbox
that leads us to 2.2 - when it's close. That means we must beg our third
party module authors to hack in the workaround for 2.0 - and look forward
and propose those changes they want that will make 2.2 the answer to all
of their problems.
In reality, we will see a handful of patches to 2.0 that won't ever be applied
to that tree, but should be considered for immediate inclusion in 2.1 so that
we can hand it back and ask, "Here - a new version - does your module work
better with our next release?" We hope, of course the answer is yes.
And when we make changes to 2.1 - we need to be the community to try
the code, and offer up "No!!! That messes up my beautiful design for mod_foo!"
Tell us early so we offer the best platform for supporting third party modules.
Stas, thanks for raising these issues, and I hope your comments do generate
some additional activity, around making 2.0.49 even more stable, and around
improving the software so that the 2.2.0 release solves hosts of issues.
Bill
Re: the wheel of httpd-dev life is surely slowing down,
solutions please
Posted by "William A. Rowe, Jr." <wr...@apache.org>.
At 07:14 PM 11/10/2003, Stas Bekman wrote:
>I have several reasons to believe that the wheel of httpd-dev life is
>slowing down and something has to be done to get this wheel up to the
>speed like in the good old days. The following observation are listed
>in no particular order. I've also tried to offer suggestions how to
>resolve problems.
>
>======================================================================
>1) Bugs
>
>searching for NEW and REOPENED bugs in httpd-2.0 returns: 420 entries
>http://nagoya.apache.org/bugzilla/buglist.cgi?bug_status=NEW&bug_status=REOPENED&product=Apache+httpd-2.0
>
>Suggestion: make bugzilla CC bug-reports to the dev list. Most
>developers won't just go and check the bugs database to see if there
>is something to fix, both because they don't have time and because
>there are too many things to fix. Posting the reports to the list
>raises a chance for the developer to have a free minute and may be to
>resolve the bug immediately or close it.
>
>======================================================================
>2) Lack of Design
>
>In my personal opinion, the move to CTR from RTC had very bad
>implications on the design of httpd 2.1.
>
>2a). Design of new on-going features (and changes/fixes) of the
>existing features is not discussed before it's put to the code. When
>it's committed it's usually too late to have any incentive to give a
>design feedback, after all it's already working and we are too busy
>with other things, so whatever.
>
>The worst part is that it's now easy to sneak in code which otherwise
>would never be accepted (and backport it to 2.0). I don't have any
>examples, but I think the danger is there.
>
>2b). As a side-effect when someone asks a design question (e.g. me)
>it's not being answered because people tell: we are in the CRT mode,
>go ahead code it and commit it. But if the poster doesn't know what's
>the right design, if she did she won't have asked in first place.
>
>2c). Future design. There is no clear roadmap of where do we go with
>Apache 2.1. People scratch their itches with new ideas which is cool,
>but if there was a plan to work on things, this may have invited new
>developers to scratch their itches.
>
>2d). CRT seemed to come as a replacement for design discussions. It's
>very easy to observe from the traffic numbers:
>
>http://marc.theaimsgroup.com/?l=apache-httpd-dev
> 2003-11-01 - 2003-12-01 (118 messages)
> 2003-10-01 - 2003-11-01 (336 messages)
> 2003-09-01 - 2003-10-01 (275 messages)
> 2003-08-01 - 2003-09-01 (264 messages)
> 2003-07-01 - 2003-08-01 (321 messages)
> 2003-06-01 - 2003-07-01 (430 messages)
> 2003-05-01 - 2003-06-01 (450 messages)
> 2003-04-01 - 2003-05-01 (359 messages)
> 2003-03-01 - 2003-04-01 (696 messages)
> 2003-02-01 - 2003-03-01 (573 messages)
> 2003-01-01 - 2003-02-01 (546 messages)
> 2002-12-01 - 2003-01-01 (436 messages)
> 2002-11-01 - 2002-12-01 (538 messages)
> 2002-10-01 - 2002-11-01 (763 messages)
> 2002-09-01 - 2002-10-01 (894 messages)
> 2002-08-01 - 2002-09-01 (815 messages)
> 2002-07-01 - 2002-08-01 (721 messages)
> 2002-06-01 - 2002-07-01 (916 messages)
> 2002-05-01 - 2002-06-01 (1028 messages)
> 2002-04-01 - 2002-05-01 (1380 messages)
> 2002-03-01 - 2002-04-01 (1094 messages)
> 2002-02-01 - 2002-03-01 (1155 messages)
>
>Quiz: based on this input, tell which date CRT was introduced at.
>
>You don't need a fancy graph to see a clear decline in the last 1.5
>years. Quite a few people suggested that this is due to the market
>decline. I won't doubt them, but it's quite possible that the market
>hasn't changed to the worst in the last 1.5 years (it is more likely
>that we should have seen this dip in 2001-2002, which I can't see from
>the numbers at the above URL).
>
>The cvs commit average rate hasn't changed much, which shows that
>development continues though mainly behind the scenes.
>http://marc.theaimsgroup.com/?l=apache-cvs&r=1&w=2
>
>
>
>======================================================================
>3). Contributions
>
>I don't have numbers to support my clause, but I have a strong feeling
>that nowadays we see a much smaller number of posts with contributions
>from non-developers, since most contributions are simply ignored and
>it's a known thing among Apache community that posting fixes and
>suggestions to httpd-dev list is usually a waste of time. (This is
>based on my personal experience and discussions with other developers
>who aren't httpd-core coders). I realize that I'm not entirely correct
>saying that, since some things are picked up, so I apologize to those
>folks who do try hard to pick those things.
>
>The obvious problem is that most of those things are unsexy, usually
>don't fit someones itch and they sometimes require a lot of
>communication overhead with the reporter/contributor just to
>understand what exactly is going on.
>
>Solutions:
>
>a. certain (most?) chunks of httpd lack owners. If httpd was to have
>clear owners of certain code bases then it'll be easy to take care of
>bug reports and contributions/patches. Even though httpd is
>collectively owned, it doesn't preclude champions in certain areas,
>who can see that good things happen to the areas they overlook.
>
>b. similar to the root rotation we have on the infrastructure, I
>suggest to have a voluntary weekly rotation of httpd-dev list champion
>whose responsibility is to make sure that most (all?) bug-reports and
>contributions are dealts with: assigned to those who those champions
>who know how to fix/review/adopt things (See (a) above), asking for
>more input if needed, etc. You don't have to know httpd as your five
>fingers to be able to do a great rewarding job.
>
>c. turn the dealing with contributions and bugs into a sexy
>thing. Everybody wants to feed their ego, there is no better way to
>make your ego happy then getting a positive feedback from a person you
>have helped to resolve a bug or handhold to provide a good patch for
>the new feature, they spent so much time working on.
>
>
>3a) I can hardly see new developers joining the team, should we blame
>the economy or the lack of encouragement for people to contribute. I
>think we now have a higher rate of developers who leave the project
>(even though that technically they are still committers and all) the
>those who join the project. Which obviously has clear effects on the
>productivity of the dev list.
>
>======================================================================
>4). Feedback Preference and List Karma
>
>List Karma is a known issue in all Open Source projects. It's obvious
>that known and respected members of the httpd-dev list come to a
>higher preference when it comes to posting feedback and any answers at
>all. It's very easy to see that questions from most known httpd
>developers almost always have a long thread with resolution. Other
>posters are usually lucky to get any responses at all, and if they do
>usually the thread would die out without any resolutions.
>
>I'm not questioning the validity of the list karma, this is something
>that I happen to stick to myself on the lists where I can give useful
>feedback: I usually answer first questions from people whom I know for
>good and bad. What I want to talk about is about the karma of the
>posters who don't ask to solve their "personal" problems, but act on
>behalf sub-projects built upon httpd. These posters try to solve a
>problem for hundreds, thousands and sometimes millions of people, who
>won't use Apache at all if they all they needed is to serve static
>pages -- there are several other web servers which outperform Apache
>on this front. They use Apache because of the 3rd party modules that
>build upon Apache. It's those 3rd party modules that Apache should be
>thankful to to its current world domination in the web server sector.
>
>Therefore I think httpd developers should stop thinking, "this
>poster's question is not httpd's problem, I have better things to
>do". I'd like to suggest that it's a very own httpd problem, because
>usually you get questions from 3rd party developers when something in
>httpd doesn't cooperate with those 3rd party modules and needs to be
>fixed. I think it's much less important to work on Apache 2.1 and much
>more important to polish 2.0 and make 3rd party developers
>happy. (wearing the 3rd party module developer hat) we don't need no
>Apache 2.1 in the sky, we need Apache 2.0 on steady on the ground, so
>we can run our modules on.
>
>and if I may add that giving those 3rd party developers a commit
>access is not a solution to the problem, because they hardly survive
>coping with their own projects' bugs, support issues and feature
>demands. So telling them: "hey, we gave you a commit access, just go
>and code/fix it". is not always working.
>
>
>
>
>
>__________________________________________________________________
>Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
>http://stason.org/ mod_perl Guide ---> http://perl.apache.org
>mailto:stas@stason.org http://use.perl.org http://apacheweek.com
>http://modperlbook.org http://apache.org http://ticketmaster.com
>
RE: the wheel of httpd-dev life is surely slowing down, solutions please
Posted by "Peter J. Cranstone" <cr...@msn.com>.
Good response - ask the customer what he wants and then help him achieve it.
It all starts with stability - compatibility - performance. The ASF has a
tough job ahead of it, getting millions of users to change.
Not an easy task in today's environment
Peter
-----Original Message-----
From: Ben Hyde [mailto:bhyde@pobox.com]
Sent: Thursday, November 13, 2003 7:24 AM
To: dev@httpd.apache.org
Subject: Re: the wheel of httpd-dev life is surely slowing down, solutions
please
I've quite a few ideas and opinions about why things might be quiet
these days. I'd recommend against taking any of these ideas too
seriously. Here is an idea: we have gotten out in front of the users.
Products have features and overtime they get more and more features.
Users have some ability to absorb features and overtime their ability
to do so rises. For young products the users are ahead of the product
("What do you mean it doesn't to CGI scripting!?"). For older products
the mismatch gets smaller and smaller ("I need massive virtual
hosting!").
My joke about this is that the process is a perfect behaviorist
training loop. Young products get strong feedback which trains their
sponsors to listen to customer demand and add features. Overtime that
feedback peter's out, which trains them to listen more and more
carefully. Finally they get to the point where they listen to the wind
and they hear feature demands in it's ghostly moans. That's call
market research. :-)
Commercial products tend to overshoot the users. Consider Microsoft's
office products!
Open source is less given to overshoot. If features only go in because
a user volunteered to do the work that acts; to some degree to temper
the chance of overshoot.
Open source can still overshoot. Exceptional users may add features
that are rarely needed. Firms, lead by market research or other
in-house demands, may volunteer.
So, one theory is that we have overshot the user's ability to absorb
new features. Some amount of overshoot is to be expected on any major
release.
Solutions? Well if you buy this model - and like I said it's only one
- then the trick is to aid users in climbing the learning curve.
Figuring out where the user demand is and then helping to bridge the
gap using the new features. Helping all the complementary products
absorb the new stuff.
This may seem like an argument that we have filled out our ecological
niche; but it's slightly different than that. The niche isn't fixed,
it is a free variable as well.
- ben
Re: the wheel of httpd-dev life is surely slowing down, solutions please
Posted by Ben Hyde <bh...@pobox.com>.
I've quite a few ideas and opinions about why things might be quiet
these days. I'd recommend against taking any of these ideas too
seriously. Here is an idea: we have gotten out in front of the users.
Products have features and overtime they get more and more features.
Users have some ability to absorb features and overtime their ability
to do so rises. For young products the users are ahead of the product
("What do you mean it doesn't to CGI scripting!?"). For older products
the mismatch gets smaller and smaller ("I need massive virtual
hosting!").
My joke about this is that the process is a perfect behaviorist
training loop. Young products get strong feedback which trains their
sponsors to listen to customer demand and add features. Overtime that
feedback peter's out, which trains them to listen more and more
carefully. Finally they get to the point where they listen to the wind
and they hear feature demands in it's ghostly moans. That's call
market research. :-)
Commercial products tend to overshoot the users. Consider Microsoft's
office products!
Open source is less given to overshoot. If features only go in because
a user volunteered to do the work that acts; to some degree to temper
the chance of overshoot.
Open source can still overshoot. Exceptional users may add features
that are rarely needed. Firms, lead by market research or other
in-house demands, may volunteer.
So, one theory is that we have overshot the user's ability to absorb
new features. Some amount of overshoot is to be expected on any major
release.
Solutions? Well if you buy this model - and like I said it's only one
- then the trick is to aid users in climbing the learning curve.
Figuring out where the user demand is and then helping to bridge the
gap using the new features. Helping all the complementary products
absorb the new stuff.
This may seem like an argument that we have filled out our ecological
niche; but it's slightly different than that. The niche isn't fixed,
it is a free variable as well.
- ben