You are viewing a plain text version of this content. The canonical link for it is here.
Posted to server-dev@james.apache.org by "Peter M. Goldstein" <pe...@yahoo.com> on 2002/10/25 20:17:12 UTC
Current changes (was RE: Code change the Apache way)
> Some suggestions to facilitate changes:
> - Cleanup/redesign few things at a time not 5 things together
We would have. Except you keep blocking submission with -1 votes. And
Noel keeps trying to figure out creative ways to appease you while
keeping the momentum to improve the product going
> - Keep scope as close to what is being attempted. We have been making
more
> design changes than a bug fix/cleanup needs.
I'm baffled by your definition of "design changes". Basically you seem
to think that any change that
i) Removes a file
ii) Reorganizes the methods of a file
constitutes a design change. It doesn't.
Of the changes discussed to date, the only one that is arguably a
"design change" is removing the dependency on the scheduler. And that
one was discussed ad nauseum over the summer. Everyone agreed, the
scheduler had to go. Everyone but you still does. We have successful
demonstrated tests of a scheduler-free James that runs like a bat out of
hell. You haven't put the time and energy in to bolster your position -
you haven't provided a .sar upon request. But we're still stuck on your
-1.
This behavior is exactly the sort of plodding thinking that let this
trivial bug
http://issues.apache.org/bugzilla/show_bug.cgi?id=5824
sit around for nine months. In short, if something doesn't work it
doesn't constitute a design. It's only when
i) An architectural issue has been discussed and documented
ii) The basic functionality expected of that feature is provided (bugs
are allowed, critical ones are not)
that you have anything that can possibly constitute a design. The above
criteria ensure that the proposed feature/arrangement/etc. works, and
that understanding of the ideas involved is passed to the community.
Otherwise, you've got nothing.
I have a real problem with this sort of paralysis. It basically
guarantees that the product goes nowhere.
In the last three months the James community has, by group effort:
i) Closed James as an open SMTP relay
ii) Added a new service for fetching POP3 mail to a single mailbox
iii) Improved the performance of the spooler by 50%+, depending on the
configuration.
iv) Removed multiple critical memory leaks
v) Made the NNTP service come into line with spec
vi) Made NNTP SSL and auth functional
vii) Improved SMTP performance by 900%. Other handlers should see a
substantial improvement as well.
viii) Reduced heap size under load dramatically.
ix) Fixed god knows how many other bugs
At no time save this one did we debate whether these were "design
changes". We examined the code base and modified it as necessary to
make James work properly. In some cases files were reorganized or
deleted, but this was almost never a real consideration as to whether
the fix would be accepted. There was discussion in almost all cases of
whether the proposed fixes were the right fixes, but the community came
to a consensus and the fixes were made in short order. The current
discussion, forced entirely by you, is easily the longest discussion
we've had on implementing a bug fix during my time at James. And easily
the least productive.
As software developers, we are bound by our own sense of professionalism
to maintain support for our published APIs until and unless we announce
that the next version will not support them. This is a responsibility
to our user base, and it is one I have championed vigorously.
The same logic doesn't hold true for the details of our internal
classes. Frankly, I don't care how the system currently works
internally - that's not a criterion that we use to evaluate change.
There is nothing sacrosanct about the current code. In most of the
above cases the current "design" was flawed, in either a major
(AuthService) or a minor (SMTPHandler) way. So they were fixed. That
applies whether the original author of the code is Serge, you, me, or
anyone else. That's the way successful software projects work.
--Peter
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: Current changes (was RE: Code change the Apache way)
Posted by Harmeet Bedi <ha...@kodemuse.com>.
Sorry last email was a bit confusing, did cut and paste in the wrong email.
I try to reduce time on emails, and that sometimes reduces the quality of
response:
Please ignore the last email and read this one.
----- Original Message -----
From: "Noel J. Bergman" <no...@devtech.com>
> > I don't think much is to be gained by continuing this thread.
> > I really would like us to get over 2.1 and focus on 3.0.
>
> The only thing I'd like to gain from it is to ensure that we don't have
> similar issues for v3.
me too. :-)
One thing that would help a lot is a formal release manager. It could be an
active developer or even someone outside the group who feels strongly about
mail,
schedule/planning and can herd cats..
Would be good to have someone who has managed distributed or open source
projects in the past where developers are not always available. Any takers ?
I think this is an extremely critical role and would help make us all more
successful with James. We should also decide on what are we going to do in
the next major release.
>
> > I haven't seen any test added to verify the
> > functionality. We should augment our testing.
>
> Sounds like an opportunity. Volunteering? :-)
I hope to be more active in at least testing.
>
> > Hopefully NNTP will be a focus for 3.0 along with more testing.
>
> Thanks for reminding me! Check out this thread:
>
> http://www.mail-archive.com/general@jakarta.apache.org/msg05985.html
>
> TestMaker sounds similar to what we talked about building with BSF. But
> neither TestMaker nor JMeter seem to handle SMTP, POP3, NNTP or IMAP.
>
Started looking at it. It seems very python oriented.
One problem is that this approach seems to require significant coding
component in scripts. Scripts are not strongly typed and at least one level
of checks are lost.
It would be good to have some template based test capability. Preferably,
tied into a standard too.
For example it should be trivial to pick from RFC 821 section
-------------------------
3.2. FORWARDING
Example of Forwarding
Either
S: RCPT TO:<Po...@USC-ISI.ARPA>
R: 251 User not local; will forward to <Po...@USC-ISIF.ARPA>
Or
S: RCPT TO:<Pa...@USC-ISIB.ARPA>
R: 551 User not local; please try <Mo...@USC-ISIF.ARPA>
-------------------------
And have script like
# forwarding test from RFC 821, Section 3.2
# user local
S: RCPT TO:test@localhost
C: 250.*
# user is not local
S: RCPT TO:test@yahoo.com
C: 551.*
With a TestMaker it looks like a lot more work.
Harmeet
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: Current changes (was RE: Code change the Apache way)
Posted by "Noel J. Bergman" <no...@devtech.com>.
> I don't think much is to be gained by continuing this thread.
> I really would like us to get over 2.1 and focus on 3.0.
The only thing I'd like to gain from it is to ensure that we don't have
similar issues for v3.
> I haven't seen any test added to verify the
> functionality. We should augment our testing.
Sounds like an opportunity. Volunteering? :-)
> Hopefully NNTP will be a focus for 3.0 along with more testing.
Thanks for reminding me! Check out this thread:
http://www.mail-archive.com/general@jakarta.apache.org/msg05985.html
TestMaker sounds similar to what we talked about building with BSF. But
neither TestMaker nor JMeter seem to handle SMTP, POP3, NNTP or IMAP.
--- Noel
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: Current changes (was RE: Code change the Apache way)
Posted by Harmeet Bedi <ha...@kodemuse.com>.
From: "Noel J. Bergman" <no...@devtech.com>
> The NNTP code was broken for a long time with no one working on it. Peter
> fixed it and posted changes for comment. Obviously, he didn't ignore the
> discussion, because you and he got into an argument over the changes. You
> two couldn't agree
Yes, Peter found exact problem. I offerred to fix it and he ignored and
checked it in.
The question I had was - Why did he invite the discussion and if he had he
should have let the author make a stab at it before changing the design.
This is esp. important as the author had disagreed with the changes.
Anyway ignored that but it that was followed by some more design changes
that seemed to be only driven by cleanup instincts and caused at least one
significant regression.
Btw. for all the excitement about bug fixing and pointing to
implementation/design flaws, I haven't seen any test added to verify the
functionality. We should augment our testing.
I don't think much is to be gained by continuing this thread. I really would
like us to get over 2.1 and focus on 3.0.
Hopefully NNTP will be a focus for 3.0 along with more testing.
Harmeet
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: Current changes (was RE: Code change the Apache way)
Posted by "Noel J. Bergman" <no...@devtech.com>.
> From: Nicola Ken Barozzi <ni...@apache.org>
> > A note on this subject:
> > Developers that are not involved in a specific part of the codebase
> > should abstain from vetoing components that other developers work on.
> That is the exact thing that got me ticked off.
The NNTP code was broken for a long time with no one working on it. Peter
fixed it and posted changes for comment. Obviously, he didn't ignore the
discussion, because you and he got into an argument over the changes. You
two couldn't agree, but that is not the same as if Peter were vetoing
changes that you were making to NNTP, without his having spent time working
on it. I doubt that he'd have done either, if someone had been actively
maintaining it. You could have vetoed his changes, and submitted your own
for acceptance.
As I understand it from the messages between you two, Peter said on multiple
occassions that future versions should have a "real" Authentication Service,
but he removed AuthService for v2.1 because it was seriously flawed, and his
change was the more expedient one for this release.
Honestly, I haven't looked at the code that was in question, and personally,
I'd have just moved NNTP out of the release (e.g., into proposal/) until
someone(s) agreed on how to fix it, and did so.
> This had been going on for some time
I've only been here since April or so, but you've got me with that one.
What has been going on for some time? I have not seen any substantial
change checked in that wasn't discussed first, or at least posted for
discussion. Changes are lazy consensus. If no one speaks up, they are
accepted. No one should be faulted for not hearing what's not said. Every
time an issue has come up, it has been discussed, so far as I'm aware.
--- Noel
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: Current changes (was RE: Code change the Apache way)
Posted by Harmeet Bedi <ha...@kodemuse.com>.
----- Original Message -----
From: Nicola Ken Barozzi <ni...@apache.org>
> A note on this subject:
> Developers that are not involved in a specific part of the codebase
> should abstain from vetoing components that other developers work on.
> For example, say that I have worked for weeks on IMAP, and someone that
> has never committed anything to it vetos one of my commits...
That is the exact thing that got me ticked off.
This had been going on for some time, but invite to discussion and ignore
and silent NNTP design changes forced me to react more than I normally do.
In case you want to dig up look before '[PATCH] Removing Scheduler
dependency, refactoring service code'.
Harmeet
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: Current changes (was RE: Code change the Apache way)
Posted by Harmeet Bedi <ha...@kodemuse.com>.
[Harmeet]
> > > - Keep scope as close to what is being attempted. We have been
> making
> > more
> > > design changes than a bug fix/cleanup needs.
> >
[Serge]
> > Yes which are fixing show-stopper bugs and improving performace by 30%
> +.
> >
> > Whats an email server if you can't count on it to do its job? How many
> > people keep asking "is it production ready?"
>
[Peter]
> Agreed. One minor correction. Performance improvement isn't 30%, it's
> 900%. Prior to these changes James crashed under a load of ~6
> mails/second (360 mails/minute). With the current code base (Watchdog,
> object pooling, deleteonexit removed) we are getting a performance of
> ~60 mails/second (3600 mails/minute).
This is not a minor correction. I think you have done an excellent job.
While I may not agree with you all the time but what you have done is very
creditable.
What I was suggesting neither discourages or slows. It may do the opposite -
facilitate and make things easier.
Harmeet
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: Current changes (was RE: Code change the Apache way)
Posted by "Noel J. Bergman" <no...@devtech.com>.
> > > we should get into the habit of tagging... we
> > > haven't done this since 2.0a3.
> > Ahem, we've (I've) tagged everything thats invloved
> > incrementing the version numbers.
> Where are you tagging? I graphed build.xml,
> a src/java/.java file, and a couple other
> files and didn't see any tags or branches.
While you guys are hashing this out, perhaps we can use this discussion as
fodder for putting together good guidelines / instructions for Committers
with respect to the CVS is in order? :-)
--- Noel
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: Current changes (was RE: Code change the Apache way)
Posted by Serge Knystautas <se...@lokitech.com>.
Danny Angus wrote:
>>Also, with the release, we should get into the habit of tagging... we
>>haven't done this since 2.0a3.
>
>
> Ahem, we've (I've) tagged everything thats invloved incrementing the version numbers.
Where are you tagging? I graphed build.xml, a src/java/.java file, and
a couple other files and didn't see any tags or branches.
--
Serge Knystautas
Loki Technologies - Unstoppable Websites
http://www.lokitech.com
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: Current changes (was RE: Code change the Apache way)
Posted by Danny Angus <da...@apache.org>.
> Also, with the release, we should get into the habit of tagging... we
> haven't done this since 2.0a3.
Ahem, we've (I've) tagged everything thats invloved incrementing the version numbers.
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: Current changes (was RE: Code change the Apache way)
Posted by Serge Knystautas <se...@lokitech.com>.
Noel J. Bergman wrote:
> I took a look at the change log that Serge put out for each prior release;
> boy, do we have a lot of work to do pulling that together for 2.1!
Yeah, that's a real pain, but really necessary (and part of the Apache
Way if I'm not mistaken (although I already have been once today)). I
usually use a cvs changelog to make sure I don't miss any changes. Also
generally makes it easy to give the right person/people credit too.
Also, with the release, we should get into the habit of tagging... we
haven't done this since 2.0a3. I really like this bit from Maven's
site...
http://jakarta.apache.org/turbine/maven/development-process.html.
Documentation on our release process and naming conventions and anything
else related would be great to capture as we do this next build.
--
Serge Knystautas
Loki Technologies - Unstoppable Websites
http://www.lokitech.com
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: Current changes (was RE: Code change the Apache way)
Posted by "Noel J. Bergman" <no...@devtech.com>.
> A design is not a design until it works.
Well, it could just be a bad design :-), but that issue aside, I think that
an implementation can validate a design, but that a bad implementation
doesn't invalidate a design.
Yes, a validation ought to provide the basic functionality.
Not having spent any time looking at AuthService, I'd have to correct that
lack before involving myself in that technical discussion.
> I've starting coordinating a list by examining the
> commit messages for the last three months.
Good place to start.
--- Noel
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: Current changes (was RE: Code change the Apache way)
Posted by "Peter M. Goldstein" <pe...@yahoo.com>.
Noel,
> > In short, if something doesn't work it doesn't
> > constitute a design.
>
> I don't agree with that as a general statement, and I don't believe
that
> you
> do, either. Bugs do not invalidate design. Architecture, design and
> implementation each has a role, and a defect in one does NOT imply a
> defect
> in another. Bugs exist and can be fixed. For that matter, proper
> operation
> does not imply that the design or architecture is correct, but as
Thomas
> Kuhn would likely argue, we often require failure before we are
motivated
> to
> undertake structural change.
I actually do believe in this as a general statement. A design is not a
design until it works. That doesn't mean that the implementation can't
have bugs, but it does mean that the basic functionality has to be
provided and work as expected. Anything else is virtually an invitation
for developers to spend their time doing what most of them enjoy most
(designing components) and not doing what most of them enjoy least
(testing functionality). BTW, I put myself into this group.
A great example of this is AuthService. AuthService was a singleton
component that stored authentication credentials in a global fashion.
This was a critical bug from day one. Even the simplest negative test
case would've exposed the problem. But none was done. Thus I don't
consider that design to have any merit - since it never worked, I can't
really worry about breaking it. Fixing it in any sense would've
required changing the "design", as the auth credentials would've had to
have been moved from the global context.
I do agree that basic proper function is not a sufficient condition for
optimal design, but I will argue that it's a necessary condition. An
architecture that precludes proper function is not an optimal design -
it's not even a valid one.
> Again, as for the rest, I think that we're moving forward, should
leave it
> at that, and do what we can to keep James established as an attractive
> community for developers interested in a high quality, flexible, mail
> server.
>
> I took a look at the change log that Serge put out for each prior
release;
> boy, do we have a lot of work to do pulling that together for 2.1!
Yeah, I've starting coordinating a list by examining the commit messages
for the last three months. Lots to do...
--Peter
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: Current changes (was RE: Code change the Apache way)
Posted by "Noel J. Bergman" <no...@devtech.com>.
> Except you keep blocking submission with -1 votes.
Did he? Or were they simply implied? It has been so long, I honestly don't
recall. And at this point, since we are moving forward, I DO NOT CARE. I
do, however, believe that should all relax and move forward with as much
harmony as we can muster.
As for "design changes", hopefully we can all agree with something I posted
a few minutes ago, to wit:
- As a general rule, micro-change is less stressful than macro.
But not necessarily more correct.
- Sometimes a change of design is good, or even necessary.
Those things should be discussed.
- Not every change that moves code is a design change,
and not every class is part of the design.
Now, in the context of this thread, as opposed to the other one where I
really want to keep the discussion on the Apache philosophy and not our
current issues, I will say that I consider BaseConnectionHandler to fall
into that last category. It is a class that clearly was part of the
implementation, but not a design element.
> Of the changes discussed to date, the only one that is arguably a
> "design change" is removing the dependency on the scheduler.
Actually, I would disagree with that argument. The Watchdog fills exactly
the same role in the design that the Scheduler was used to fill. There is
no design change. If anything, tying handlers more closely to their server,
rather than having a handler as an pseudo-independent construct, could be
viewed as a design change, although a minor design change with significant
positive implementation benefits. We still have the same design elements,
all filling the same roles. All that has changed is a bit of linkage.
> In short, if something doesn't work it doesn't
> constitute a design.
I don't agree with that as a general statement, and I don't believe that you
do, either. Bugs do not invalidate design. Architecture, design and
implementation each has a role, and a defect in one does NOT imply a defect
in another. Bugs exist and can be fixed. For that matter, proper operation
does not imply that the design or architecture is correct, but as Thomas
Kuhn would likely argue, we often require failure before we are motivated to
undertake structural change.
Again, as for the rest, I think that we're moving forward, should leave it
at that, and do what we can to keep James established as an attractive
community for developers interested in a high quality, flexible, mail
server.
I took a look at the change log that Serge put out for each prior release;
boy, do we have a lot of work to do pulling that together for 2.1!
--- Noel
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>