You are viewing a plain text version of this content. The canonical link for it is here.
Posted to server-dev@james.apache.org by "Peter M. Goldstein" <pe...@yahoo.com> on 2002/10/15 04:10:08 UTC
Scheduler Interface
All,
I guess I just don't get it. Someone please explain it to me.
We have an interface, written and implemented in a variety of forms by
Peter Donald. Peter D. is the primary author of Phoenix, one of the
continuing primary motivators behind Avalon, and a pretty smart guy.
When he is confronted with our use of the Scheduler he replied:
SS: Here is why we raised this as an issue. The Apache James mail
server
SS: project is using this timer mechanism to implement a connection
timeout on
SS: the SMTP port. When it receives data on the the port, it calls
SS: resetTrigger to resets the timer.
PD: ouch it is not really designed for that!
If the primary designer of the interface recognizes that the use to
which we are putting the Scheduler is not within its original design
parameters, why is there such unswerving loyalty to this interface?
When we examine the analogous interface and implementation from the Java
libraries - the Timer - we see exactly the same result. Why is this?
It could, one supposes, be simple incompetence on the part of the Sun
developers. I'm certainly not going to suggest that they don't make
mistakes. But I think we can conclude, especially with the Avalon case
to back us up, that the reason applying that class to this problem
results in OutOfMemoryErrors is that the two models don't fit.
Schedulers and Timers are fundamentally designed for situations where:
i) In general, events will be triggered
ii) Events are not continually added and removed extremely frequently
iii) The event target and the reset condition are not tightly coupled -
that is, we expect the time to trigger (and hence the time the
monitoring thread sleeps) to be independent of changes to the target
during the sleep time. That's simply not true in this case.
Let me repeat. There are extremely good technical reasons why the
Scheduler model is inappropriate. Those have been laid out in seemingly
exhaustive detail. References have been cited. The designer of the
interface thinks this is a bad idea.
On the other hand we have a new, simple, easy to understand interface
that would allow you to place a Scheduler behind it with no trouble
whatsoever.
I'm sorry. This is drop dead simple. I don't even see grounds for a
disagreement.
--Peter
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: Scheduler Interface
Posted by "Noel J. Bergman" <no...@devtech.com>.
> Because a vote hasn't been asked for.
Actually, Harmeet asked that it be voted on, and he is a committer. I just
posted the message.
--- Noel
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: Scheduler Interface
Posted by Danny Angus <da...@apache.org>.
> Why has this issue turned into a my code your code battle when
> clearly there
> is a CORRECT approach?
Because I'm *not* prepared to back a change which peforms less well in a simple test. "CORRECT" or not, I'll back any change that can be seen to work and resist anything that can be seen to degrade performance.
> Peter, Noel ... make those changes and lets get on with it.
Unfortunately, Serge, this cannot really be done without lazy concensus approval.
> Why haven't the other committers voted?
Because a vote hasn't been asked for. (Noel isnt a commiter) or they are not as interested in this issue as you are.
> I am just venting frustration here, sorry guys.
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: Scheduler Interface
Posted by Serge Sozonoff <se...@globalbeach.com>.
Hi Guys,
I know I can't vote, but here is my +1 for making Peter's watchdog changes.
How many more programmers is it going to take to explain what Peter, Noel et
al have explained till everyone "gets it".
Why has this issue turned into a my code your code battle when clearly there
is a CORRECT approach?
Peter, Noel ... make those changes and lets get on with it.
Why haven't the other committers voted?
I am just venting frustration here, sorry guys.
Thanks, Serge
----- Original Message -----
From: "Peter M. Goldstein" <pe...@yahoo.com>
To: "'James Developers List'" <ja...@jakarta.apache.org>
Sent: Tuesday, October 15, 2002 4:10 AM
Subject: Scheduler Interface
>
> All,
>
> I guess I just don't get it. Someone please explain it to me.
>
> We have an interface, written and implemented in a variety of forms by
> Peter Donald. Peter D. is the primary author of Phoenix, one of the
> continuing primary motivators behind Avalon, and a pretty smart guy.
> When he is confronted with our use of the Scheduler he replied:
>
> SS: Here is why we raised this as an issue. The Apache James mail
> server
> SS: project is using this timer mechanism to implement a connection
> timeout on
> SS: the SMTP port. When it receives data on the the port, it calls
> SS: resetTrigger to resets the timer.
>
> PD: ouch it is not really designed for that!
>
> If the primary designer of the interface recognizes that the use to
> which we are putting the Scheduler is not within its original design
> parameters, why is there such unswerving loyalty to this interface?
>
> When we examine the analogous interface and implementation from the Java
> libraries - the Timer - we see exactly the same result. Why is this?
> It could, one supposes, be simple incompetence on the part of the Sun
> developers. I'm certainly not going to suggest that they don't make
> mistakes. But I think we can conclude, especially with the Avalon case
> to back us up, that the reason applying that class to this problem
> results in OutOfMemoryErrors is that the two models don't fit.
> Schedulers and Timers are fundamentally designed for situations where:
>
> i) In general, events will be triggered
> ii) Events are not continually added and removed extremely frequently
> iii) The event target and the reset condition are not tightly coupled -
> that is, we expect the time to trigger (and hence the time the
> monitoring thread sleeps) to be independent of changes to the target
> during the sleep time. That's simply not true in this case.
>
> Let me repeat. There are extremely good technical reasons why the
> Scheduler model is inappropriate. Those have been laid out in seemingly
> exhaustive detail. References have been cited. The designer of the
> interface thinks this is a bad idea.
>
> On the other hand we have a new, simple, easy to understand interface
> that would allow you to place a Scheduler behind it with no trouble
> whatsoever.
>
> I'm sorry. This is drop dead simple. I don't even see grounds for a
> disagreement.
>
> --Peter
>
>
>
>
> --
> To unsubscribe, e-mail:
<ma...@jakarta.apache.org>
> For additional commands, e-mail:
<ma...@jakarta.apache.org>
>
>
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: Scheduler Interface
Posted by Harmeet Bedi <ha...@kodemuse.com>.
Good point about scheduler interface.
2 things to chat about:
1st thing
======
What would be an appropriate scheduler interface ?
Here is a candidate:
public interface DelayedEventScheduler
{
/**
* Schedule an offset based trigger.
* Note that if a TimeTrigger already has same name then it is removed.
*
* @param id trigger identifier
* @param trigger the trigger
* @param target the target
*/
void addTrigger( Object id, long offset, Target target );
/**
* Remove a scheduled trigger by id.
*
* @param id trigger identifier.
* @exception NoSuchElementException if no trigger exists with that name
*/
void removeTrigger( Object id )
throws NoSuchElementException;
/**
* Force a trigger time to be recalculated.
*
* @param id trigger identifier
* @exception NoSuchElementException if no trigger exists with that name
*/
void resetTrigger( Object id )
throws NoSuchElementException;
}
/** calback interface when an event is triggered */
public interface Target
{
void targetTriggered();
}
2nd thing
======
There has been some degree to reluctance to change abstraction. Maybe this
is due to the late release stage. Can the handler keep the same abstraction
but implementation ignore the inappropriate part of the interface and map to
simpler interface. This provides a simple swapping strategy till there is
consensus on 1st point and something to look forward for the next release.
Regarding:
> When we examine the analogous interface and implementation from the Java
> libraries - the Timer - we see exactly the same result. Why is this?
Agreed JDK Timer implemenation had the same problems. It was an unpleasant
surprise to me. But there is a 3rd implementation that was posted that did
not exhibit the same bad behaviour. Try that one. My testing and Noel's
testing indicates significant improvement. There is now an improved version
that addresses issues that were posted about 3rd implementation.
Harmeet
PS: sorry last email got sent a bit early.
----- Original Message -----
From: "Peter M. Goldstein" <pe...@yahoo.com>
To: "'James Developers List'" <ja...@jakarta.apache.org>
Sent: Monday, October 14, 2002 7:10 PM
Subject: Scheduler Interface
>
> All,
>
> I guess I just don't get it. Someone please explain it to me.
>
> We have an interface, written and implemented in a variety of forms by
> Peter Donald. Peter D. is the primary author of Phoenix, one of the
> continuing primary motivators behind Avalon, and a pretty smart guy.
> When he is confronted with our use of the Scheduler he replied:
>
> SS: Here is why we raised this as an issue. The Apache James mail
> server
> SS: project is using this timer mechanism to implement a connection
> timeout on
> SS: the SMTP port. When it receives data on the the port, it calls
> SS: resetTrigger to resets the timer.
>
> PD: ouch it is not really designed for that!
>
> If the primary designer of the interface recognizes that the use to
> which we are putting the Scheduler is not within its original design
> parameters, why is there such unswerving loyalty to this interface?
>
> When we examine the analogous interface and implementation from the Java
> libraries - the Timer - we see exactly the same result. Why is this?
> It could, one supposes, be simple incompetence on the part of the Sun
> developers. I'm certainly not going to suggest that they don't make
> mistakes. But I think we can conclude, especially with the Avalon case
> to back us up, that the reason applying that class to this problem
> results in OutOfMemoryErrors is that the two models don't fit.
> Schedulers and Timers are fundamentally designed for situations where:
>
> i) In general, events will be triggered
> ii) Events are not continually added and removed extremely frequently
> iii) The event target and the reset condition are not tightly coupled -
> that is, we expect the time to trigger (and hence the time the
> monitoring thread sleeps) to be independent of changes to the target
> during the sleep time. That's simply not true in this case.
>
> Let me repeat. There are extremely good technical reasons why the
> Scheduler model is inappropriate. Those have been laid out in seemingly
> exhaustive detail. References have been cited. The designer of the
> interface thinks this is a bad idea.
>
> On the other hand we have a new, simple, easy to understand interface
> that would allow you to place a Scheduler behind it with no trouble
> whatsoever.
>
> I'm sorry. This is drop dead simple. I don't even see grounds for a
> disagreement.
>
> --Peter
>
>
>
>
> --
> To unsubscribe, e-mail:
<ma...@jakarta.apache.org>
> For additional commands, e-mail:
<ma...@jakarta.apache.org>
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: Scheduler Interface
Posted by "Noel J. Bergman" <no...@devtech.com>.
> Peter, the proposal for watchdog change has been put out an hour or so
back
> and I sent some questions on it.
That is simply not so. The proposal first made months ago, the code was
revised, and submitted more formally a week ago. Not an hour ago.
In any event, I've been answering your questions. I don't know where Andrei
is, these days, but I do thank him in absentia for his efforts.
> Expected freeze was on 10th and you still haven't posted a bug fixed
patch.
This debate has delayed that submission, Harmeet. Else we could have spent
this time testing and working instead of debating.
> I prefer patching the server with better scheduler for now.
I don't, but I'm perfectly happy to offer the choice of implementation.
> I agree the interface to manage timouts/(or scheduler for timeouts)
> needs changes but I don't want to cross the hill today.
It was crossed in June, July, August and October by a whole group of
developers. It isn't a new issue.
--- Noel
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: Scheduler Interface
Posted by Harmeet Bedi <ha...@kodemuse.com>.
Peter, the proposal for watchdog change has been put out an hour or so back
and I sent some questions on it.
Your patch was sent on 8th, Danny had issues with it and Noel told me over
IM that your were fixing bugs in it.
Expected freeze was on 10th and you still haven't posted a bug fixed patch.
What do you want me to say ?
I prefer patching the server with better scheduler for now. I agree the
interface to manage timouts/(or scheduler for timeouts) needs changes but I
don't want to cross the hill today. That is my preference but I have said if
there is significant measurable gain I will not have an issue.
Also if other committers agree to the change, I'll go along.
Don't know what else to say...
Harmeet
----- Original Message -----
From: "Peter M. Goldstein" <pe...@yahoo.com>
>
> Harmeet,
>
> > And I agree that we should talk about change but not for this release.
> >
> > Here are the dates that were sent out.(by Peter)
> > ----------------------
> > Feature Freeze: 10/3/02
> > Code Freeze: 10/11/02
> > Doc Freeze: 10/11/02
> > Branch Date: 10/20/02
> > Formal Test Period: 10/12/02 - 10/20/02
> > Release Date: 10/21/02
> > ----------------------
>
> Tut, tut. Misrepresentation is getting to be a habit with you. First
> Noel's email and now mine.
>
> Like all good developers, I would never post a list of dates sans a list
> of features. You will note that in the list of features for the release
> plan that was proposed, there is an explicit line item called something
> like "Andrei's Changes". As Noel's timeline (and my extensive comments
> on this list) make clear, this line item specifically refers to the
> changes we are currently discussing.
>
> Moreover, there was no discussion of this schedule or any potential
> impact on it when you asked for a few days to review my patch. It seems
> in questionable taste to pull out the release schedule now. Even more
> so since you didn't bother to +1 it when it was proposed.
>
> Please don't misrepresent my statements.
>
> --Peter
>
>
>
>
>
> --
> To unsubscribe, e-mail:
<ma...@jakarta.apache.org>
> For additional commands, e-mail:
<ma...@jakarta.apache.org>
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: Peter, Harmeet
Posted by "Peter M. Goldstein" <pe...@yahoo.com>.
All,
Please ignore this email. I'll be posting the latest and greatest
version of the Watchdog, including the re-factoring to allow use of
multiple factories shortly. Noel and I have been working on and testing
it for much of today.
--Peter
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: Peter, Harmeet
Posted by "Peter M. Goldstein" <pe...@yahoo.com>.
Noel,
> - I don't believe there will be an improvement with watchdog. I don't
> think
> all of Peter's patch has been posted. Danny you tried the posted patch
and
> it made things worse for you. If the patch did improve
> performance/scalability significantly and I could verify it there
would
> not
> have been as much of an issue. We may be exchanging one set of
problems
> with
> other.
You are wrong. See Noel's test results.
Attached is the SMTPHandler.java (with the sendMail line commented out)
and SMTPServer.java I've been using. Also attached are the Watchdog
classes used in our tests over the last few days. Works like a charm.
Note the single theWatchdogThread.notify() call that was removed from
the dispose method of TimeoutWatchdog.
I've explained over and over why Danny encountered the issue he did. I
don't see why the point needs to be belabored again.
> - I don't think refactoring is needed. There are ways of addressing
> configuration overhead with current structure.
You haven't proposed one. I gave reasons why your suggestions were
erroneous, and you haven't addressed any of them.
Moreover, as I made clear in my earlier posts, this is not all about
performance. This is about centralized logging, failure on startup,
etc. All the things that a server should do. The changes to the server
are minimal and extremely low impact.
> Here is what I would like. Have Peter or Noel put out proposals for
vote.
>
> Please assume my +1.
> - If there are two +1 (non Peter)committers votes,
> or
> - only one +1(non Peter) and no committer vote for 2 days after a
single
> (non Peter) committer +1.
>
> I don't know of a more accomodating way to compromise on this. Does
this
> sound reasonable ?
Vote +1 on Noel's proposal, which let's you keep a centralized scheduler
and be done with it.
> Will you guys please try to sort your problem out, I have no idea why
you
> are at each others throats on *every* topic, but its getting a bit
> wearisome. We're not going to get any progress if you can't *both*
> compromise.
A compromise has been offered. Noel made a compromise suggestion and
I've even offered to implement it. It's not what I want, and it's not
optimal, but it will allow the code to run in either mode.
He's made a proposal. If you don't like the fact that he isn't a
committer and made a proposal and called for a VOTE, then take the
proposal as coming from me. We've wasted more than enough time on this,
and I'm willing to settle on a suboptimal compromise just to be done
with it and move on.
--Peter
Re: Peter, Harmeet
Posted by Harmeet Bedi <ha...@kodemuse.com>.
The reason for my concern has been 2 fold.
- I don't believe there will be an improvement with watchdog. I don't think
all of Peter's patch has been posted. Danny you tried the posted patch and
it made things worse for you. If the patch did improve
performance/scalability significantly and I could verify it there would not
have been as much of an issue. We may be exchanging one set of problems with
other.
- I don't think refactoring is needed. There are ways of addressing
configuration overhead with current structure.
Here is what I would like. Have Peter or Noel put out proposals for vote.
Please assume my +1.
- If there are two +1 (non Peter)committers votes,
or
- only one +1(non Peter) and no committer vote for 2 days after a single
(non Peter) committer +1.
I don't know of a more accomodating way to compromise on this. Does this
sound reasonable ?
Harmeet
----- Original Message -----
From: "Danny Angus" <da...@apache.org>
To: "James Developers List" <ja...@jakarta.apache.org>
Sent: Tuesday, October 15, 2002 4:22 AM
Subject: Peter, Harmeet
Will you guys please try to sort your problem out, I have no idea why you
are at each others throats on *every* topic, but its getting a bit
wearisome. We're not going to get any progress if you can't *both*
compromise.
d.
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Peter, Harmeet
Posted by Danny Angus <da...@apache.org>.
Will you guys please try to sort your problem out, I have no idea why you are at each others throats on *every* topic, but its getting a bit wearisome. We're not going to get any progress if you can't *both* compromise.
d.
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: Scheduler Interface
Posted by "Peter M. Goldstein" <pe...@yahoo.com>.
Harmeet,
> And I agree that we should talk about change but not for this release.
>
> Here are the dates that were sent out.(by Peter)
> ----------------------
> Feature Freeze: 10/3/02
> Code Freeze: 10/11/02
> Doc Freeze: 10/11/02
> Branch Date: 10/20/02
> Formal Test Period: 10/12/02 - 10/20/02
> Release Date: 10/21/02
> ----------------------
Tut, tut. Misrepresentation is getting to be a habit with you. First
Noel's email and now mine.
Like all good developers, I would never post a list of dates sans a list
of features. You will note that in the list of features for the release
plan that was proposed, there is an explicit line item called something
like "Andrei's Changes". As Noel's timeline (and my extensive comments
on this list) make clear, this line item specifically refers to the
changes we are currently discussing.
Moreover, there was no discussion of this schedule or any potential
impact on it when you asked for a few days to review my patch. It seems
in questionable taste to pull out the release schedule now. Even more
so since you didn't bother to +1 it when it was proposed.
Please don't misrepresent my statements.
--Peter
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: Scheduler Interface
Posted by "Noel J. Bergman" <no...@devtech.com>.
I've got it. Heading to bed. Start testing in AM. OK, *later* in AM.
--- Noel
-----Original Message-----
From: Harmeet Bedi [mailto:harmeet@kodemuse.com]
Sent: Tuesday, October 15, 2002 7:33
To: James Developers List
Subject: Re: Scheduler Interface
> Please make sure to zip up the entire binary dist, so that I don't have to
> go through hoops to install it.
here it is.
http://www.kodemuse.com/pub/harmeet/james.zip
If anyone else wants go get it but connection is slow. Noel is downloading
so try after half hour or so.
Harmeet
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: Scheduler Interface
Posted by Harmeet Bedi <ha...@kodemuse.com>.
> Please make sure to zip up the entire binary dist, so that I don't have to
> go through hoops to install it.
here it is.
http://www.kodemuse.com/pub/harmeet/james.zip
If anyone else wants go get it but connection is slow. Noel is downloading
so try after half hour or so.
Harmeet
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: Scheduler Interface
Posted by "Noel J. Bergman" <no...@devtech.com>.
> I am trying to help get the interface solid. See emails and help in
> solidfying proposal you have sent out (minus abstractions)
And I've been answering your questions. :-) Even if it is 3:45 AM.
> haven't voted -1
> Said that if significant number of commiters vote on this, I'll be happy
> to go along.
I've noticed. :-)
> I you read any email I sent, and conversation over IM, you will notice I
do
> not object to change but the timing of this. This was based on release
> schedule that was posted.
That is not consistently clear to me from your e-mails, but I'll take your
word for it.
As for the schedule, Peter (who proposed the scheduler) based it upon his
expectation of finishing this long awaited set of changes done sooner; it
was late because he diverted to fix NNTP.
> I have just finished testing a scheduler impl with cleanup and javadocs
etc.
> Will sent it around if you want to test it out.
Please make sure to zip up the entire binary dist, so that I don't have to
go through hoops to install it.
> There were 2 issues Service Refactoring and watchdog. The combination does
> mean significant change in the handler interaction. This should be done as
a
> proposal and put for voting. That is the standard way.
I do agree that changes should be approved by committers, but I've looked
over the submission, and I disagree with your "significant change" issue.
--- Noel
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: Scheduler Interface
Posted by Harmeet Bedi <ha...@kodemuse.com>.
From: "Noel J. Bergman" <no...@devtech.com>
> On the one hand, you say that you are willing to accept the group's
> decisions, on the other hand you are doing everything that you can to
reject
> them.
Actually I am not,
- I am trying to help get the interface solid. See emails and help in
solidfying proposal you have sent out (minus abstractions)
- haven't voted -1
- Said that if significant number of commiters vote on this, I'll be happy
to go along.
I you read any email I sent, and conversation over IM, you will notice I do
not object to change but the timing of this. This was based on release
schedule that was posted.
I have just finished testing a scheduler impl with cleanup and javadocs etc.
Will sent it around if you want to test it out. I really can't test Peter's
patch because he has not posted the fixed code. Danny had problems with
Peter's patch. My patch works better than current release. So should we not
look at alternatives because of a patch sent by Andrei 1 month back.
There were 2 issues Service Refactoring and watchdog. The combination does
mean significant change in the handler interaction. This should be done as a
proposal and put for voting. That is the standard way.
Harmeet
----- Original Message -----
From: "Noel J. Bergman" <no...@devtech.com>
To: "James Developers List" <ja...@jakarta.apache.org>
Sent: Monday, October 14, 2002 11:43 PM
Subject: RE: Scheduler Interface
> Harmeet,
>
> You, yourself, have held up this issue for over a week, distracting
everyone
> from the work of getting out the release. Instead of integrating code,
> testing, and working on resolving any defects, we've been going round and
> round on this issue with you.
>
> The code was first contributed months ago, and is specifically to fix bugs
> in the code. And since your code still needs work, and wasn't submitted
"in
> time", I am not inclined to accept your tactic of using the schedule as an
> excuse to take your code by default.
>
> > This is not the time to redesign. Please do impact analysis
>
> There is NO RE-DESIGN OF THE HANDLERS, the code was contributed months
ago,
> the impact analysis has already been included in many messages from Peter
> and myself. I've also mentioned the impact of your code. You have not
> contributed one constructive objection to the change, other than that you
> don't want a change and you are concerned about threads. I have suggested
> both a means to support your implementation AND discussed threading with
> you.
>
> On the one hand, you say that you are willing to accept the group's
> decisions, on the other hand you are doing everything that you can to
reject
> them.
>
> You've asked for a vote. I've called it.
>
> --- Noel
>
>
> --
> To unsubscribe, e-mail:
<ma...@jakarta.apache.org>
> For additional commands, e-mail:
<ma...@jakarta.apache.org>
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: Scheduler Interface
Posted by "Noel J. Bergman" <no...@devtech.com>.
Harmeet,
You, yourself, have held up this issue for over a week, distracting everyone
from the work of getting out the release. Instead of integrating code,
testing, and working on resolving any defects, we've been going round and
round on this issue with you.
The code was first contributed months ago, and is specifically to fix bugs
in the code. And since your code still needs work, and wasn't submitted "in
time", I am not inclined to accept your tactic of using the schedule as an
excuse to take your code by default.
> This is not the time to redesign. Please do impact analysis
There is NO RE-DESIGN OF THE HANDLERS, the code was contributed months ago,
the impact analysis has already been included in many messages from Peter
and myself. I've also mentioned the impact of your code. You have not
contributed one constructive objection to the change, other than that you
don't want a change and you are concerned about threads. I have suggested
both a means to support your implementation AND discussed threading with
you.
On the one hand, you say that you are willing to accept the group's
decisions, on the other hand you are doing everything that you can to reject
them.
You've asked for a vote. I've called it.
--- Noel
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: Scheduler Interface
Posted by Harmeet Bedi <ha...@kodemuse.com>.
----- Original Message -----
From: "Noel J. Bergman" <no...@devtech.com>
Subject: RE: Scheduler Interface
> > So propose an alternate interface.
>
> I did. And one that supports BOTH implementations.
Make a formal proposal and ask for Vote for next release.
And I agree that we should talk about change but not for this release.
Here are the dates that were sent out.(by Peter)
----------------------
Feature Freeze: 10/3/02
Code Freeze: 10/11/02
Doc Freeze: 10/11/02
Branch Date: 10/20/02
Formal Test Period: 10/12/02 - 10/20/02
Release Date: 10/21/02
----------------------
>
> > Propose and implementation and measure the benefit.
> > Why is that problematic with this approach ?
>
> I believe in proper design. As the IBM paper mentions, proper DESIGN is
the
> most important issue with multi-threading. It is hard and time consuming
to
> load test a multithreaded implementation, and even though I am willing to
> spend some time doing so, I really have my own work to do.
This is not the time to redesign. Please do impact analysis
Harmeet
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: Scheduler Interface
Posted by "Noel J. Bergman" <no...@devtech.com>.
> So propose an alternate interface.
I did. And one that supports BOTH implementations.
> Propose and implementation and measure the benefit.
> Why is that problematic with this approach ?
I believe in proper design. As the IBM paper mentions, proper DESIGN is the
most important issue with multi-threading. It is hard and time consuming to
load test a multithreaded implementation, and even though I am willing to
spend some time doing so, I really have my own work to do.
You tried using Timer, and I had to explain why that fails. You refered to
Doug Lea, and I pointed out that his code uses two threads per timed
operation. In fact, if you look at the CommandMap implementation that I
posted months ago, you will find very similar code for handling time guarded
operations. I hadn't looked at Doug's code until tonight, but although it
is gratifying to see the similarities, it isn't surprising. There are
certain known techniques, and this one is a classic within our field.
> Believe it or not, I have a fix mitigate synchronization issue.
> It is really simple and implicit in your messsage. :-)
I look forward to seeing, it. HOWEVER ...
You have said that you wonder why Avalon and Sun have such inefficient
implementations. They are not inefficient. They are designed for a
particular task. There are certain assumptions about how schedulers are
used. They implemented the code so that they could minimize the contention
issues for multiple users, thus allowing their classes to be more scalable.
The tradeoff there is that they simply mark removed items as reset, and
clean up later. In normal operation (for a scheduler), both of those
implementations use far less memory and far less CPU than your class. As I
have said multiple times, there is a difference between a Scheduler and a
Watchdog.
Earlier you indicated that you would accept being wrong. Now you are
holding onto an idea for which no one else has expressed support. I admire
sticking to your beliefs when everyone else says that you're wrong, and I've
been willing to help you investigate a shared thread solution, but at this
point I do not see that this is contributing to improving James. It is just
making everyone frustrated, and I, for one, am not happy with the idea of
maintaining the status quo because one person won't allow the project to
move forward.
I have suggested a compromise. Peter says that he'll do the work to
implement the compromise. Both implementations will be in the code. The
work is already done. What is your issue with accepting this solution?
--- Noel
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: Scheduler Interface
Posted by Harmeet Bedi <ha...@kodemuse.com>.
----- Original Message -----
From: "Noel J. Bergman" <no...@devtech.com>
To: "James Developers List" <ja...@jakarta.apache.org>
Sent: Monday, October 14, 2002 9:31 PM
Subject: RE: Scheduler Interface
> > What would be an appropriate scheduler interface ?
>
> The one you outline hear has the same problem. You are putting a lookup
> into the INTERFACE instead of burying it in the IMPLEMENTATION.
So propose an alternate interface. Propose and implementation and measure
the benefit. Why is that problematic with this approach ?
I don't want us to change things at this point unless there is demonstrable
gain.
>
> > There has been some degree to reluctance to change abstraction.
>
> No. Peter, myself, and on occassion YOU, are willing to use the Watchdog
> interface with the handlers. Danny seems willing. From what I
understand,
> others are willing. The work has already been done. Your code can be put
> under it to provide an alternative. Peter is willing to make the changes,
> if you won't (I'm really pressed for time right now).
It is not as much about work. I am looking for this:
> a) Measurable system gain
> b) Not swap strategies in this release esp. if there is a improved
Scheduler
> implementation. We can take this up in next release.
> c) I want a backoff strategy if there are issues.
Harmeet
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: Scheduler Interface
Posted by "Noel J. Bergman" <no...@devtech.com>.
> What would be an appropriate scheduler interface ?
The one you outline hear has the same problem. You are putting a lookup
into the INTERFACE instead of burying it in the IMPLEMENTATION.
> There has been some degree to reluctance to change abstraction.
No. Peter, myself, and on occassion YOU, are willing to use the Watchdog
interface with the handlers. Danny seems willing. From what I understand,
others are willing. The work has already been done. Your code can be put
under it to provide an alternative. Peter is willing to make the changes,
if you won't (I'm really pressed for time right now).
--- Noel
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: Scheduler Interface
Posted by Harmeet Bedi <ha...@kodemuse.com>.
----- Original Message -----
From: "Peter M. Goldstein" <pe...@yahoo.com>
>
> Harmeet,
>
> Do you actually read my emails? Because it sure doesn't sound like it.
>
> > 1st thing
> > ======
> >
> > What would be an appropriate scheduler interface ?
>
> The Scheduler interface is an appropriate interface. For a scheduler.
> This is not a scheduler. This is a timeout.
All right here is a more precise question.
What would be an appropriate scheduler interface for James timeout
monitoring ?
Please put forward a proposal as set of interfaces.
> Your implementation almost certainly has the exact same problems, for
> the exact same reasons. The fault lies in the thread contention on the
> lookup. Are we clear? You can't avoid that without violating thread
> safety.
Here are some questions to consider.
- Have you bothered to measure performace and scalability of alternate
approaches ?
- For actual mail messages how much does it slow ?
- Can that be addressed by using concurrent maps from doug lea's concurrent
library ? - Is there a alternative that has measurable better performance
and scalability characteristics ? The last email from Danny indicated
a -ive. Has something changed since. If it has why don't you post rather
than pose.
Let us talk code and results rather than take up mutually exclusive
positions.
There is a way to resolve conflict - measurable results.
Suggest we measure:
- Scalability
- Performance
- Impact on the rest of system
>
> The reason the Timer was a surprise to you was precisely because you
> fail to appreciate this point. I didn't bother replying to your Timer
> implementation because I knew it would crash. I let Noel test it, and
> it crashed. Surprise.
Good thinking Peter.
>
> I'll also note that you claimed it wouldn't crash before he ran his test
> (having run tests of your own) and he managed to bring it down in a
> minute or two each time.
I did think JDK folks would have a better implementation then they had, but
you may also note that after that I came up with something better, tested it
and it did improve on uptime significantly. Do you have something that
improves the system measurable ? If you do, once more post it.
> The swapping strategy is not simple. It is simply wrong.
Peter, I have never said that we cannot change the interface, however I want
a) Measurable system gain
b) Not swap strategies in this release esp. if there is a improved Scheduler
implementation. We can take this up in next release. Said this earlier too.
c) I want a backoff strategy if there are issues. If we had release with
your patch and Danny had not tested, there would have been a significant
regression and no easy implementation swap.
Harmeet
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: Scheduler Interface
Posted by "Peter M. Goldstein" <pe...@yahoo.com>.
Harmeet,
Do you actually read my emails? Because it sure doesn't sound like it.
> 1st thing
> ======
>
> What would be an appropriate scheduler interface ?
The Scheduler interface is an appropriate interface. For a scheduler.
This is not a scheduler. This is a timeout. Pick one of my previous
emails and read it.
> 2nd thing
> ======
>
> There has been some degree to reluctance to change abstraction. Maybe
this
> is due to the late release stage. Can the handler keep the same
> abstraction
> but implementation ignore the inappropriate part of the interface and
map
> to
> simpler interface. This provides a simple swapping strategy till there
is
> consensus on 1st point and something to look forward for the next
release.
>
> Regarding:
> > When we examine the analogous interface and implementation from the
Java
> > libraries - the Timer - we see exactly the same result. Why is
this?
>
> Agreed JDK Timer implemenation had the same problems. It was an
unpleasant
> surprise to me. But there is a 3rd implementation that was posted that
did
> not exhibit the same bad behaviour. Try that one. My testing and
Noel's
> testing indicates significant improvement. There is now an improved
> version
> that addresses issues that were posted in 3rd implementation.
Your implementation almost certainly has the exact same problems, for
the exact same reasons. The fault lies in the thread contention on the
lookup. Are we clear? You can't avoid that without violating thread
safety. Period. It doesn't matter what you do in your implementation -
it will either:
i) Violate thread safety
ii) Slow down asymptotically as the number of contending threads
increases. For reset times that are a few percent of the thread
execution time, this is disastrous.
The reason the Timer was a surprise to you was precisely because you
fail to appreciate this point. I didn't bother replying to your Timer
implementation because I knew it would crash. I let Noel test it, and
it crashed. Surprise.
I'll also note that you claimed it wouldn't crash before he ran his test
(having run tests of your own) and he managed to bring it down in a
minute or two each time. With 5 active connections (as you're on a
Win2k box). Try ramping that up to 20. Or 80, when you load all four
services. Crash. Thread contention scales non-linearly.
I'll also point out that you failed to address any of the comments about
the interface designer's comments and why the Java Timer manifests the
behavior it does. Again, that's because you don't understand this
point.
The swapping strategy is not simple. It is simply wrong.
--Peter
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: Scheduler Interface
Posted by Harmeet Bedi <ha...@kodemuse.com>.
Good point about scheduler interface ?
2 things to chat about:
1st thing
======
What would be an appropriate scheduler interface ?
public interface DelayedEventScheduler
{
/**
* Schedule an offset based trigger.
* Note that if a TimeTrigger already has same name then it is removed.
*
* @param id trigger identifier
* @param trigger the trigger
* @param target the target
*/
void addTrigger( Object id, long offset, Target target );
/**
* Remove a scheduled trigger by id.
*
* @param id trigger identifier.
* @exception NoSuchElementException if no trigger exists with that name
*/
void removeTrigger( Object id )
throws NoSuchElementException;
/**
* Force a trigger time to be recalculated.
*
* @param id trigger identifier
* @exception NoSuchElementException if no trigger exists with that name
*/
void resetTrigger( Object id )
throws NoSuchElementException;
}
/** calback interface when an event is triggered */
public interface Target
{
void targetTriggered();
}
2nd thing
======
There has been some degree to reluctance to change abstraction. Maybe this
is due to the late release stage. Can the handler keep the same abstraction
but implementation ignore the inappropriate part of the interface and map to
simpler interface. This provides a simple swapping strategy till there is
consensus on 1st point and something to look forward for the next release.
Regarding:
> When we examine the analogous interface and implementation from the Java
> libraries - the Timer - we see exactly the same result. Why is this?
Agreed JDK Timer implemenation had the same problems. It was an unpleasant
surprise to me. But there is a 3rd implementation that was posted that did
not exhibit the same bad behaviour. Try that one. My testing and Noel's
testing indicates significant improvement. There is now an improved version
that addresses issues that were posted in 3rd implementation.
Harmeet
----- Original Message -----
From: "Peter M. Goldstein" <pe...@yahoo.com>
To: "'James Developers List'" <ja...@jakarta.apache.org>
Sent: Monday, October 14, 2002 7:10 PM
Subject: Scheduler Interface
>
> All,
>
> I guess I just don't get it. Someone please explain it to me.
>
> We have an interface, written and implemented in a variety of forms by
> Peter Donald. Peter D. is the primary author of Phoenix, one of the
> continuing primary motivators behind Avalon, and a pretty smart guy.
> When he is confronted with our use of the Scheduler he replied:
>
> SS: Here is why we raised this as an issue. The Apache James mail
> server
> SS: project is using this timer mechanism to implement a connection
> timeout on
> SS: the SMTP port. When it receives data on the the port, it calls
> SS: resetTrigger to resets the timer.
>
> PD: ouch it is not really designed for that!
>
> If the primary designer of the interface recognizes that the use to
> which we are putting the Scheduler is not within its original design
> parameters, why is there such unswerving loyalty to this interface?
>
> When we examine the analogous interface and implementation from the Java
> libraries - the Timer - we see exactly the same result. Why is this?
> It could, one supposes, be simple incompetence on the part of the Sun
> developers. I'm certainly not going to suggest that they don't make
> mistakes. But I think we can conclude, especially with the Avalon case
> to back us up, that the reason applying that class to this problem
> results in OutOfMemoryErrors is that the two models don't fit.
> Schedulers and Timers are fundamentally designed for situations where:
>
> i) In general, events will be triggered
> ii) Events are not continually added and removed extremely frequently
> iii) The event target and the reset condition are not tightly coupled -
> that is, we expect the time to trigger (and hence the time the
> monitoring thread sleeps) to be independent of changes to the target
> during the sleep time. That's simply not true in this case.
>
> Let me repeat. There are extremely good technical reasons why the
> Scheduler model is inappropriate. Those have been laid out in seemingly
> exhaustive detail. References have been cited. The designer of the
> interface thinks this is a bad idea.
>
> On the other hand we have a new, simple, easy to understand interface
> that would allow you to place a Scheduler behind it with no trouble
> whatsoever.
>
> I'm sorry. This is drop dead simple. I don't even see grounds for a
> disagreement.
>
> --Peter
>
>
>
>
> --
> To unsubscribe, e-mail:
<ma...@jakarta.apache.org>
> For additional commands, e-mail:
<ma...@jakarta.apache.org>
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>