You are viewing a plain text version of this content. The canonical link for it is here.
Posted to server-dev@james.apache.org by "Peter M. Goldstein" <pe...@yahoo.com> on 2002/10/18 01:18:45 UTC
Memory Leak...found and fixed
All,
Ok folks, we've found the big non-scheduler memory leak. The source can
be found in the MimeMessageInputStreamSource. The use of the
deleteOnExit() method either caused a JVM internal list to grow without
bound or prevented cleanup of the temporary file objects. Whatever the
exact details, it's clear that this is the source of the problem. The
dependence on the finalizer method for file cleanup is another severe
problem in this class. Removing the deleteOnExit and adding an
explicit dispose() chain to force deletion of the file resolves this
memory issue.
In the process of resolving this, we discovered a serious bug in the
Avalon Excalibur DefaultPool and DefaultThreadPool implementations.
Basically, if an interrupted thread attempts to return an object to the
pool, it corrupts the mutex of the pool and causes an eventual
catastrophic failure. This issue is currently under discussion on the
Avalon-dev list.
Noel and I have been running tests and revamping code for much of the
last two or three days. As of today, we've got a code base that runs
consistently with 20 connections, 2 messages per connection, 1k
messages, at a rate of ~3000 messages per minute. Heap size remains
stable at about 4.5 MB. Total thread count is stable at 112 threads.
This is on a PII 400MHz Celeron Linux box.
We had some issues that resulted from the above Avalon Excalibur bugs,
but we think we've got a relatively bullet proof version of the code.
We're running a test now (Noel is gone for the next four hours, but the
test is running in his absence) but the first 30 minutes of testing
showed no triggering of the Avalon Excalibur problem. Previous runs
(before bulletproofing) resulted in an error within the first 15 minutes
and a unresponsive server within 1.5 hours.
Once we've got a successful test, and we've taken a little time to
neaten up the code, we'll be ready to post the diffs. I suspect we'll
reach that point sometime tomorrow, assuming all goes well with the
latest tests.
--Peter
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: [VOTE] New Commiter Noel J Bergman
Posted by Stephen McConnell <mc...@apache.org>.
Serge Sozonoff wrote:
>All,
>
>I know I can't vote, but...... +1
>
>I don't understand how anyone can vote otherwise, all you have to do is read
>the last several months of this list where From = Noel Bergman to realise
>that Noel knows his stuff and is able to contribute to *any* portion of
>JAMES including IMAP.
>
Serge you said it all - I can't vote either but here is my +1 for Noel.
Harmeet - *please* reconsider your position.
Cheers, Steve.
>Serge
>
>----- Original Message -----
>From: "Danny Angus" <da...@apache.org>
>To: "James Developers List" <ja...@jakarta.apache.org>
>Sent: Saturday, October 19, 2002 11:46 PM
>Subject: [VOTE] New Commiter Noel J Bergman
>
>
>Guys,
>
>I'd like to propose Noel as a commiter, If I have to detail his
>accomplishments you've been too inactive.. Over the last months Noel has
>contributed code and opinion, offered his help on the Users list and
>mediated in Robust Discussions.
>
>Failing to elect Noel would rob James of one of our most enthusiastic
>participants.
>
>d.
>
>start the count with my +1
>
>
>--
>To unsubscribe, e-mail: <ma...@jakarta.apache.org>
>For additional commands, e-mail: <ma...@jakarta.apache.org>
>
>
>
>
>--
>To unsubscribe, e-mail: <ma...@jakarta.apache.org>
>For additional commands, e-mail: <ma...@jakarta.apache.org>
>
>
>
>
>
--
Stephen J. McConnell
OSM SARL
digital products for a global economy
mailto:mcconnell@osm.net
http://www.osm.net
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: [VOTE] New Commiter Noel J Bergman
Posted by Serge Sozonoff <se...@globalbeach.com>.
All,
I know I can't vote, but...... +1
I don't understand how anyone can vote otherwise, all you have to do is read
the last several months of this list where From = Noel Bergman to realise
that Noel knows his stuff and is able to contribute to *any* portion of
JAMES including IMAP.
Serge
----- Original Message -----
From: "Danny Angus" <da...@apache.org>
To: "James Developers List" <ja...@jakarta.apache.org>
Sent: Saturday, October 19, 2002 11:46 PM
Subject: [VOTE] New Commiter Noel J Bergman
Guys,
I'd like to propose Noel as a commiter, If I have to detail his
accomplishments you've been too inactive.. Over the last months Noel has
contributed code and opinion, offered his help on the Users list and
mediated in Robust Discussions.
Failing to elect Noel would rob James of one of our most enthusiastic
participants.
d.
start the count with my +1
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: [VOTE] New Commiter Noel J Bergman
Posted by Serge Knystautas <se...@lokitech.com>.
+1
----- Original Message -----
From: "Danny Angus" <da...@apache.org>
To: "James Developers List" <ja...@jakarta.apache.org>
Sent: Saturday, October 19, 2002 5:46 PM
Subject: [VOTE] New Commiter Noel J Bergman
Guys,
I'd like to propose Noel as a commiter, If I have to detail his
accomplishments you've been too inactive.. Over the last months Noel has
contributed code and opinion, offered his help on the Users list and
mediated in Robust Discussions.
Failing to elect Noel would rob James of one of our most enthusiastic
participants.
d.
start the count with my +1
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: [VOTE] New Commiter Noel J Bergman
Posted by Harmeet Bedi <ha...@kodemuse.com>.
----- Original Message -----
From: "Charles Benett" <ch...@apache.org>
> You have disagreed
> about one (or two) but not the other five or six.
Actually, I haven't disagreed with any of Noel's patches. I think they are
all very good esp. the HabeasWarrantMark stuff.
Harmeet
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: [VOTE] New Commiter Noel J Bergman
Posted by Charles Benett <ch...@apache.org>.
Harmeet Bedi wrote:
>-1.
>
>I think Noel is great and I would love to work with him, but I'd like to
>defer on this. I have enjoyed talking to him and consider him a friend
>however I would really like to see more contributions or complete modules.
>
>Noel everybody holds you in high esteem. Please contribute and send patches
>in, and if there is a lag in commit time, will vote for you.
>
>We already have too many pulls and we should prob. add more IMAP
>contributors.
>
>I realize that I have been absent for long, but I also think it is best for
>James to defer on this for now.
>
>Harmeet
>
>
Harmeet,
I don't think the reasons you give are sufficient to -1 Noel.
Looking through the archives I spotted 7 patches in the last 3 weeks
(listed below), I think that is very good going. You have disagreed
about one (or two) but not the other five or six. Disagreeing about a
patch does not seem to warrant a -1, either.
Yes we need more IMAP contributors - but that is surely in addition to
contributors to SMTP/POP3 etc.
Please reconsider your vote. You can -0 if you still have reservations.
(-0 is non-blocking)
Charles
Noel's Patch submissions since 30 Sept 2002:
* Subject: [PATCH] SMTPHandler logging
* Date: Fri, 04 Oct 2002 18:39:57 -0700
* Subject: [PATCH] GenericListserv
* Date: Fri, 04 Oct 2002 18:33
* Subject: [PATCH] JDBCVirtualUserTable
* Date: Wed, 02 Oct 2002 13:44:08 -0700
* Subject: [PATCH] HasHeader matcher
* Date: Wed, 02 Oct 2002 13:42:35 -0700
* Subject: [PATCH] make DEBUG configurable
* Date: Wed, 02 Oct 2002 13:23:40 -0700
* Subject: [PATCH] Logging changes
* Date: Mon, 30 Sep 2002 10:39:09 -0700
* Subject: [PATCH] HasHabeasWarrantMark / AddHabeasWarrantMark
* Date: Mon, 30 Sep 2002 09:17:44 -0700
>
>--
>To unsubscribe, e-mail: <ma...@jakarta.apache.org>
>For additional commands, e-mail: <ma...@jakarta.apache.org>
>
>
>
>
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: phoenix-bin\bin\phoenix.sh in crlf
Posted by "Peter M. Goldstein" <pe...@yahoo.com>.
Serge,
Interesting. This was recently noted on the IMAP build. I suspect this
happened with the Phoenix 4.0 release upgrade. I'll fix it, as well as
adding an appropriate entry into build.xml to ensure it remains LF.
--Peter
> -----Original Message-----
> From: Serge Sozonoff [mailto:serge@globalbeach.com]
> Sent: Sunday, October 20, 2002 11:25 AM
> To: James Developers List
> Subject: phoenix-bin\bin\phoenix.sh in crlf
>
> Hi All,
>
> It seem that the phoenix.sh script is checked in with crlf
end-of-lines
> instead of lf so it won't run out-of-the-box on unix like it should.
>
> Serge
>
>
>
> --
> To unsubscribe, e-mail: <mailto:james-dev-
> unsubscribe@jakarta.apache.org>
> For additional commands, e-mail: <mailto:james-dev-
> help@jakarta.apache.org>
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: phoenix-bin\bin\phoenix.sh in crlf
Posted by "Peter M. Goldstein" <pe...@yahoo.com>.
Serge,
Didn't see the CRLF problem, but I did see an issue with phoenix.sh not
being set to executable. I changed the build so it forces the file to
LF and I added a line to make phoenix.sh executable.
--Peter
> -----Original Message-----
> From: Serge Sozonoff [mailto:serge@globalbeach.com]
> Sent: Sunday, October 20, 2002 11:25 AM
> To: James Developers List
> Subject: phoenix-bin\bin\phoenix.sh in crlf
>
> Hi All,
>
> It seem that the phoenix.sh script is checked in with crlf
end-of-lines
> instead of lf so it won't run out-of-the-box on unix like it should.
>
> Serge
>
>
>
> --
> To unsubscribe, e-mail: <mailto:james-dev-
> unsubscribe@jakarta.apache.org>
> For additional commands, e-mail: <mailto:james-dev-
> help@jakarta.apache.org>
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: Proposal .. error messages
Posted by Serge Knystautas <se...@lokitech.com>.
I think that might be useful as an attachment, but really needs to be
buried. From my experience, most users won't read past the second
paragraph in the email (if that), so this would be useful only to the
admin who the bounce gets forwarded to.
--
Serge Knystautas
Loki Technologies - Unstoppable Websites
http://www.lokitech.com
Danny Angus wrote:
> I'd like to see what everyone thinks of this idea..
> to construct more meaningful error trails for bounce and other failed messages we should append lines to the error message of a Mail, rather than replace the message with a new one, so an error might end up reading..
>
> "sent for transport
> attempt failed because '400 service unavailable'
> retrying 1
> attempt failed because '400 service unavailable'
> given up after 2 trys"
>
>
> I did this to a version I was hacking, but never got round to comitting it.
> is it a good idea, or is there a good reason not to do this?
>
> d.
>
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: Proposal .. error messages
Posted by Danny Angus <da...@apache.org>.
> I do think such a feature would be useful, although I'm far more
> interested in seeing the bounce logic being un-hard coded from the
> server.
Yeah me too.
> IMHO, a bounce should result in a mail being directed to
> another processor, allowing full custom handling of the bounce
> condition. This would allow the logic you propose to be implemented as
> a mailet. As long as the bounce behavior is flexible, either the
> behavior you specify or the current behavior could be valuable,
> depending on the needs and desires of the administrator. Personally,
> I'd like to see our bounce handling in general discussed in the rev++
> discussion.
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: Proposal .. error messages
Posted by "Noel J. Bergman" <no...@devtech.com>.
> interested in seeing the bounce logic being un-hard coded from the
> server
Agreed. There is some prior discussion of this in the archives, and it
would be a good v3 discussion.
--- Noel
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: Proposal .. error messages
Posted by "Peter M. Goldstein" <pe...@yahoo.com>.
Danny,
I do think such a feature would be useful, although I'm far more
interested in seeing the bounce logic being un-hard coded from the
server. IMHO, a bounce should result in a mail being directed to
another processor, allowing full custom handling of the bounce
condition. This would allow the logic you propose to be implemented as
a mailet. As long as the bounce behavior is flexible, either the
behavior you specify or the current behavior could be valuable,
depending on the needs and desires of the administrator. Personally,
I'd like to see our bounce handling in general discussed in the rev++
discussion.
--Peter
> -----Original Message-----
> From: Danny Angus [mailto:danny@apache.org]
> Sent: Monday, October 21, 2002 3:41 AM
> To: James Developers List
> Subject: Proposal .. error messages
>
> I'd like to see what everyone thinks of this idea..
> to construct more meaningful error trails for bounce and other failed
> messages we should append lines to the error message of a Mail, rather
> than replace the message with a new one, so an error might end up
> reading..
>
> "sent for transport
> attempt failed because '400 service unavailable'
> retrying 1
> attempt failed because '400 service unavailable'
> given up after 2 trys"
>
>
> I did this to a version I was hacking, but never got round to
comitting
> it.
> is it a good idea, or is there a good reason not to do this?
>
> d.
>
>
> --
> To unsubscribe, e-mail: <mailto:james-dev-
> unsubscribe@jakarta.apache.org>
> For additional commands, e-mail: <mailto:james-dev-
> help@jakarta.apache.org>
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: Proposal .. error messages
Posted by Danny Angus <da...@apache.org>.
> Were your messages in short hand, or did you mean them literally?
literally, I get messages like that from time to time from other MTA's
> If you do
> it, wouldn't it make sense to add the timestamp for each attempt? What
> about the mail server(s) tried, e.g., for a service like Hotmail,
> which has
> multiple servers?
hostnames would appear where relevant.
timestamp is a nice idea, james' usp ;-)
d.
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: Proposal .. error messages
Posted by "Noel J. Bergman" <no...@devtech.com>.
I'm ambivalent. Maybe because I don't tend to get bounces like that from
other mail servers. Generally, when there is a transient error I would one
that says there is a temporary failure, and if it didn't clear up within a
week or so, I'd get one saying that it still couldn't deliver the e-mail, so
it was giving up. I don't recall getting a log of each attempt. The more I
think about it, the more I think it might be nice.
Were your messages in short hand, or did you mean them literally? If you do
it, wouldn't it make sense to add the timestamp for each attempt? What
about the mail server(s) tried, e.g., for a service like Hotmail, which has
multiple servers?
--- Noel
-----Original Message-----
From: Danny Angus [mailto:danny@apache.org]
Sent: Monday, October 21, 2002 6:41
To: James Developers List
Subject: Proposal .. error messages
I'd like to see what everyone thinks of this idea..
to construct more meaningful error trails for bounce and other failed
messages we should append lines to the error message of a Mail, rather than
replace the message with a new one, so an error might end up reading..
"sent for transport
attempt failed because '400 service unavailable'
retrying 1
attempt failed because '400 service unavailable'
given up after 2 trys"
I did this to a version I was hacking, but never got round to comitting it.
is it a good idea, or is there a good reason not to do this?
d.
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Proposal .. error messages
Posted by Danny Angus <da...@apache.org>.
I'd like to see what everyone thinks of this idea..
to construct more meaningful error trails for bounce and other failed messages we should append lines to the error message of a Mail, rather than replace the message with a new one, so an error might end up reading..
"sent for transport
attempt failed because '400 service unavailable'
retrying 1
attempt failed because '400 service unavailable'
given up after 2 trys"
I did this to a version I was hacking, but never got round to comitting it.
is it a good idea, or is there a good reason not to do this?
d.
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: phoenix-bin\bin\phoenix.sh in crlf
Posted by Danny Angus <da...@apache.org>.
I think it is the permission on the file that is not being stored correctly in cvs, as its r--r--r-- I've had to ask root to chmod it to r-xr-xr-x
d.
> -----Original Message-----
> From: Serge Sozonoff [mailto:serge@globalbeach.com]
> Sent: 20 October 2002 19:25
> To: James Developers List
> Subject: phoenix-bin\bin\phoenix.sh in crlf
>
>
> Hi All,
>
> It seem that the phoenix.sh script is checked in with crlf end-of-lines
> instead of lf so it won't run out-of-the-box on unix like it should.
>
> Serge
>
>
>
> --
> To unsubscribe, e-mail:
> <ma...@jakarta.apache.org>
> For additional commands, e-mail:
> <ma...@jakarta.apache.org>
>
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
phoenix-bin\bin\phoenix.sh in crlf
Posted by Serge Sozonoff <se...@globalbeach.com>.
Hi All,
It seem that the phoenix.sh script is checked in with crlf end-of-lines
instead of lf so it won't run out-of-the-box on unix like it should.
Serge
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: [VOTE] New Commiter Noel J Bergman
Posted by Harmeet Bedi <ha...@kodemuse.com>.
+1
Sorry about this. I was being unfair to Noel.
I think he is really good and it would be an honor/pleasure to work with
him.
Apologies to all esp. to Noel for that.
Harmeet
PS: To explain
I was unhappy about
- Unnecessary refactoring leading to sometimes silent changes in design.
- Affecting my work as I have embedded portions of James code elsewhere. I
wouldn't mind the impact, but don't want to have the impact because of
personal preferences and little gain.
- File removal based on what I thought was little justification.
- A degree of personal to and fro over mail.
I was nervous about potentially having another person repeat this based on
'right'.
Having said this, all has been done to prob. improve things. It may or may
not be right or I am may not consider it right, but either way it does not
justify holding Noel back.
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: [VOTE] New Commiter Noel J Bergman
Posted by Harmeet Bedi <ha...@kodemuse.com>.
----- Original Message -----
From: "Danny Angus" <da...@apache.org>
> I think Noel is great and I would love to work with him, but I'd like to
> defer on this.
Abstain then, please convert your -1 to a +0 or -0
[hb]
There is a point in collecting votes.
I have given mine and have not put a -1 on anything like this before. It
has either been abstain or +1.
I agree that Noel is an excellent addition, but I would like to defer. There
is very little point in voting on day x and reconsider on day x+1. Will be
happy to reconsider later.
It was painful for me to take the decision but I did it and don't want to
revert decisions in such a flip flop manner.
> I have enjoyed talking to him and consider him a friend
> however I would really like to see more contributions or complete modules.
IMHO thats not what James needs, James is functionally pretty mature, what
we need is people who will continue to improve the delivery of the
functionality we have.
[hb]
Docs and delivery are different from development. There have been cases
where docs have been opened up separatly. If James is more or less done,
maybe repository should be split up to seperate docs/delivery from codebase.
Open source needs even more consensus than usu. projects and that is what
voting scheme is designed to do.
Harmeet
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: [VOTE] New Commiter Noel J Bergman
Posted by Danny Angus <da...@apache.org>.
Harmeet,
> I think Noel is great and I would love to work with him, but I'd like to
> defer on this.
Abstain then, please convert your -1 to a +0 or -0
> I have enjoyed talking to him and consider him a friend
> however I would really like to see more contributions or complete modules.
IMHO thats not what James needs, James is functionally pretty mature, what we need is people who will continue to improve the delivery of the functionality we have.
>
> Noel everybody holds you in high esteem. Please contribute and
> send patches
> in, and if there is a lag in commit time, will vote for you.
Noel has been doing this for many months.
>
> We already have too many pulls and we should prob. add more IMAP
> contributors.
Not a reason to -1 Noel.
>
> I realize that I have been absent for long, but I also think it
> is best for
> James to defer on this for now.
I can't agree.
d.
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: [VOTE] New Commiter Noel J Bergman
Posted by Harmeet Bedi <ha...@kodemuse.com>.
-1.
I think Noel is great and I would love to work with him, but I'd like to
defer on this. I have enjoyed talking to him and consider him a friend
however I would really like to see more contributions or complete modules.
Noel everybody holds you in high esteem. Please contribute and send patches
in, and if there is a lag in commit time, will vote for you.
We already have too many pulls and we should prob. add more IMAP
contributors.
I realize that I have been absent for long, but I also think it is best for
James to defer on this for now.
Harmeet
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: [VOTE] New Commiter Noel J Bergman
Posted by Charles Benett <ch...@apache.org>.
+1
Danny Angus wrote:
>Guys,
>
>I'd like to propose Noel as a commiter, If I have to detail his accomplishments you've been too inactive.. Over the last months Noel has contributed code and opinion, offered his help on the Users list and mediated in Robust Discussions.
>
>Failing to elect Noel would rob James of one of our most enthusiastic participants.
>
>d.
>
>start the count with my +1
>
>
>--
>To unsubscribe, e-mail: <ma...@jakarta.apache.org>
>For additional commands, e-mail: <ma...@jakarta.apache.org>
>
>
>
>
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: [VOTE] New Commiter Noel J Bergman
Posted by "Peter M. Goldstein" <pe...@yahoo.com>.
Danny,
> I'd like to propose Noel as a commiter, If I have to detail his
> accomplishments you've been too inactive.. Over the last months Noel
has
> contributed code and opinion, offered his help on the Users list and
> mediated in Robust Discussions.
>
> Failing to elect Noel would rob James of one of our most enthusiastic
> participants.
Concur with a +1.
--Peter
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
[VOTE] New Commiter Noel J Bergman
Posted by Danny Angus <da...@apache.org>.
Guys,
I'd like to propose Noel as a commiter, If I have to detail his accomplishments you've been too inactive.. Over the last months Noel has contributed code and opinion, offered his help on the Users list and mediated in Robust Discussions.
Failing to elect Noel would rob James of one of our most enthusiastic participants.
d.
start the count with my +1
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: Memory Leak...found and fixed
Posted by Danny Angus <da...@apache.org>.
Well done guys,
> I don't really think it's feasible in the short term,
> especially when we have the restriction of not changing the Mailet API.
Well as its slated for changes in the next cycle, we should defer it till then anyway.
Look forward to seeing what you've done.
d.
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: Memory Leak...found and fixed
Posted by "Peter M. Goldstein" <pe...@yahoo.com>.
Serge,
> Awesome work!!
>
> This sounds suspiciously like it could be related to the extra file
> (unpaired) hanging around in the file repository implementation...
would
> be great if that albatross could be solved finally as well.
>
Thank you. :)
It is indeed related to the extra file. Unfortunately, getting rid of
that albatross is a touch more complicated. Basically, we'd need to
smarten up the Spool implementations and create an engine that allowed a
shared buffer between the InputStream from the wire and the OutputStream
into the spool. I don't really think it's feasible in the short term,
especially when we have the restriction of not changing the Mailet API.
I do think it's very feasible in the longer term, in rev++.
--Peter
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: Memory Leak...found and fixed
Posted by Serge Knystautas <se...@lokitech.com>.
Awesome work!!
This sounds suspiciously like it could be related to the extra file
(unpaired) hanging around in the file repository implementation... would
be great if that albatross could be solved finally as well.
--
Serge Knystautas
Loki Technologies
http://www.lokitech.com/
Peter M. Goldstein wrote:
> All,
>
> Ok folks, we've found the big non-scheduler memory leak. The source can
> be found in the MimeMessageInputStreamSource. The use of the
> deleteOnExit() method either caused a JVM internal list to grow without
> bound or prevented cleanup of the temporary file objects. Whatever the
> exact details, it's clear that this is the source of the problem. The
> dependence on the finalizer method for file cleanup is another severe
> problem in this class. Removing the deleteOnExit and adding an
> explicit dispose() chain to force deletion of the file resolves this
> memory issue.
>
> In the process of resolving this, we discovered a serious bug in the
> Avalon Excalibur DefaultPool and DefaultThreadPool implementations.
> Basically, if an interrupted thread attempts to return an object to the
> pool, it corrupts the mutex of the pool and causes an eventual
> catastrophic failure. This issue is currently under discussion on the
> Avalon-dev list.
>
> Noel and I have been running tests and revamping code for much of the
> last two or three days. As of today, we've got a code base that runs
> consistently with 20 connections, 2 messages per connection, 1k
> messages, at a rate of ~3000 messages per minute. Heap size remains
> stable at about 4.5 MB. Total thread count is stable at 112 threads.
> This is on a PII 400MHz Celeron Linux box.
>
> We had some issues that resulted from the above Avalon Excalibur bugs,
> but we think we've got a relatively bullet proof version of the code.
> We're running a test now (Noel is gone for the next four hours, but the
> test is running in his absence) but the first 30 minutes of testing
> showed no triggering of the Avalon Excalibur problem. Previous runs
> (before bulletproofing) resulted in an error within the first 15 minutes
> and a unresponsive server within 1.5 hours.
>
> Once we've got a successful test, and we've taken a little time to
> neaten up the code, we'll be ready to post the diffs. I suspect we'll
> reach that point sometime tomorrow, assuming all goes well with the
> latest tests.
>
> --Peter
>
>
>
>
> --
> To unsubscribe, e-mail: <ma...@jakarta.apache.org>
> For additional commands, e-mail: <ma...@jakarta.apache.org>
>
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: Memory Leak...found and fixed
Posted by Harmeet Bedi <ha...@kodemuse.com>.
----- Original Message -----
From: "Peter M. Goldstein" <pe...@yahoo.com>
>
> All,
>
> Ok folks, we've found the big non-scheduler memory leak. The source can
> be found in the MimeMessageInputStreamSource. The use of the
> deleteOnExit() method either caused a JVM internal list to grow without
> bound or prevented cleanup of the temporary file objects. Whatever the
> exact details, it's clear that this is the source of the problem. The
> dependence on the finalizer method for file cleanup is another severe
> problem in this class. Removing the deleteOnExit and adding an
> explicit dispose() chain to force deletion of the file resolves this
> memory issue.
Very good stuff. :-)
Trully awesome.
>
> In the process of resolving this, we discovered a serious bug in the
> Avalon Excalibur DefaultPool and DefaultThreadPool implementations.
> Basically, if an interrupted thread attempts to return an object to the
> pool, it corrupts the mutex of the pool and causes an eventual
> catastrophic failure. This issue is currently under discussion on the
> Avalon-dev list.
I have always wondered about avalon folks creating threading library. I
wonder why they didn't use Doug Lea's util concurrent. At one point there
was talk using it and but not sure why they invented their own.
Harmeet
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>