You are viewing a plain text version of this content. The canonical link for it is here.
Posted to server-user@james.apache.org by Diego Castillo <di...@inexbee.com> on 2002/10/11 15:28:48 UTC
James memory leak
Hi,
I am using James 2.0a3 with JDK 1.3.1 on W2K machine.
I am experiencing a memory leak in my application. During my
troubleshooting I have finally removed all my mailets in order to test
James alone. The memory leak persists!
This is my scenario:
- I have a load generator that sends 10 mail/second. Mails have
different sizes, with an average of 11Kb.
- On the James size, I have a mailet that prints the JVM total and free
memory (in Mb) every 100 messages. Here you have my spoolmanager
configuration and the results that I get:
<spoolmanager>
<threads>1</threads>
<mailetpackages>
<mailetpackage>org.apache.james.transport.mailets</mailetpackage>
<mailetpackage>test</mailetpackage>
</mailetpackages>
<matcherpackages>
<matcherpackage>org.apache.james.transport.matchers</matcherpackage>
</matcherpackages>
<processor name="root">
<mailet match="All" class="MemoryPrinter"/>
<mailet match="All" class="Null"/>
</processor>
</spoolmanager>
Mail # Total memory Free memory
0 2.5820312 0.5600738
100 5.6132812 1.6391296
200 9.0234375 2.3683395
300 15.128906 5.4931946
400 15.128906 3.0394592
500 25.488281 10.329994
600 25.488281 7.8969116
700 25.488281 3.9112778
800 25.488281 1.5132217
900 42.125 16.21582
1000 42.125 12.281708
1100 42.125 10.111511
1200 42.125 7.9458237
1300 42.125 3.9470978
1400 42.125 1.725914
1500 63.75 23.18071
1600 63.75 19.276215
1700 63.75 17.146133
1800 63.75 14.976295
1900 63.75 11.164001
2000 63.75 9.04203
2100 63.75 7.0878906
2200 63.75 4.38665
James crashes in less than 4min with an OutOfMemoryError. JVM options
that increase heap size would only make this happen a little bit later
:-(
Has anyone experienced this before? Any suggestions about a better JVM
or a different James version? May this come from Avalon?
Regards,
Diego
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: James memory leak
Posted by Danny Angus <da...@apache.org>.
PS thats 10,000 1k mails delivered in 690 seconds over 100 simultaneous connections. For what its worth.
> -----Original Message-----
> From: Noel J. Bergman [mailto:noel@devtech.com]
> Sent: 11 October 2002 20:29
> To: James Users List
> Subject: RE: James memory leak
>
>
> Cool. Looking forward to it. Also, there are classes that might be good
> candidates for object pooling, and we may see some impact from that, too.
>
> --- Noel
>
> -----Original Message-----
> From: Danny Angus [mailto:danny@apache.org]
> Sent: Friday, October 11, 2002 15:05
> To: James Users List
> Subject: RE: James memory leak
>
>
>
> > I believe THAT particular symptom comes from the
> scheduler/watchdog issue,
> > which has been documented since Spring. The flood of messages creates a
> > huge number of scheduler entries, which persist for several
> minutes before
> > being released by the scheduler.
>
> I figured. Whats more I can test your theory.. watch this space. :-)
>
>
> --
> To unsubscribe, e-mail:
> <ma...@jakarta.apache.org>
> For additional commands, e-mail:
> <ma...@jakarta.apache.org>
>
>
>
> --
> To unsubscribe, e-mail:
> <ma...@jakarta.apache.org>
> For additional commands, e-mail:
<ma...@jakarta.apache.org>
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: James memory leak
Posted by "Noel J. Bergman" <no...@devtech.com>.
Cool. Looking forward to it. Also, there are classes that might be good
candidates for object pooling, and we may see some impact from that, too.
--- Noel
-----Original Message-----
From: Danny Angus [mailto:danny@apache.org]
Sent: Friday, October 11, 2002 15:05
To: James Users List
Subject: RE: James memory leak
> I believe THAT particular symptom comes from the scheduler/watchdog issue,
> which has been documented since Spring. The flood of messages creates a
> huge number of scheduler entries, which persist for several minutes before
> being released by the scheduler.
I figured. Whats more I can test your theory.. watch this space. :-)
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: James memory leak
Posted by Danny Angus <da...@apache.org>.
> I figured. Whats more I can test your theory..
and you're spot on, using only one connection per thread eliminates the symptom.
d.
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: James memory leak
Posted by Danny Angus <da...@apache.org>.
> I believe THAT particular symptom comes from the scheduler/watchdog issue,
> which has been documented since Spring. The flood of messages creates a
> huge number of scheduler entries, which persist for several minutes before
> being released by the scheduler.
I figured. Whats more I can test your theory.. watch this space. :-)
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: James memory leak
Posted by "Noel J. Bergman" <no...@devtech.com>.
Danny,
Sorry, I should have been clearer. I said it was a known issue, I didn't
say that everything was fixed. :-)
I believe THAT particular symptom comes from the scheduler/watchdog issue,
which has been documented since Spring. The flood of messages creates a
huge number of scheduler entries, which persist for several minutes before
being released by the scheduler. The message traffic outstrips the rate at
which obsolete scheduler entries are released. This is why the new watchdog
code was proposed, designed and implemented.
Other memory leaks from 2.0a3 were fixed in 2.1-CVS, which is why I jad
asked if Diego would try the current code, so that we can see the impact of
those other changes. And when we get the watchdog change into the code, we
can get a third set of data points. I also believe that there are certain
areas where we can improve memory performance with object pooling.
--- Noel
-----Original Message-----
From: Danny Angus [mailto:danny@apache.org]
Sent: Friday, October 11, 2002 13:46
To: James Users List
Subject: RE: James memory leak
I get this too from recent cvs, 100 threads delivering 100 1k mails, Jmeas
chokes at around mail 1200, after a couple of minutes.
d.
> -----Original Message-----
> From: Noel J. Bergman [mailto:noel@devtech.com]
> Sent: 11 October 2002 16:46
> To: James Users List; diego.castillo@inexbee.com
> Subject: RE: James memory leak
>
>
> Known issue. Please try the 2.1 code, and let us know what you see.
>
> --- Noel
>
> -----Original Message-----
> From: Diego Castillo [mailto:diego.castillo@inexbee.com]
> Sent: Friday, October 11, 2002 9:29
> To: james-user@jakarta.apache.org
> Cc: Diego Castillo, INEXBEE
> Subject: James memory leak
>
>
> Hi,
>
> I am using James 2.0a3 with JDK 1.3.1 on W2K machine.
>
> I am experiencing a memory leak in my application. During my
> troubleshooting I have finally removed all my mailets in order to test
> James alone. The memory leak persists!
>
> This is my scenario:
> - I have a load generator that sends 10 mail/second. Mails have
> different sizes, with an average of 11Kb.
> - On the James size, I have a mailet that prints the JVM total and free
> memory (in Mb) every 100 messages. Here you have my spoolmanager
> configuration and the results that I get:
>
> <spoolmanager>
> <threads>1</threads>
> <mailetpackages>
> <mailetpackage>org.apache.james.transport.mailets</mailetpackage>
> <mailetpackage>test</mailetpackage>
> </mailetpackages>
> <matcherpackages>
> <matcherpackage>org.apache.james.transport.matchers</matcherpackage>
> </matcherpackages>
> <processor name="root">
> <mailet match="All" class="MemoryPrinter"/>
> <mailet match="All" class="Null"/>
> </processor>
> </spoolmanager>
>
> Mail # Total memory Free memory
> 0 2.5820312 0.5600738
> 100 5.6132812 1.6391296
> 200 9.0234375 2.3683395
> 300 15.128906 5.4931946
> 400 15.128906 3.0394592
> 500 25.488281 10.329994
> 600 25.488281 7.8969116
> 700 25.488281 3.9112778
> 800 25.488281 1.5132217
> 900 42.125 16.21582
> 1000 42.125 12.281708
> 1100 42.125 10.111511
> 1200 42.125 7.9458237
> 1300 42.125 3.9470978
> 1400 42.125 1.725914
> 1500 63.75 23.18071
> 1600 63.75 19.276215
> 1700 63.75 17.146133
> 1800 63.75 14.976295
> 1900 63.75 11.164001
> 2000 63.75 9.04203
> 2100 63.75 7.0878906
> 2200 63.75 4.38665
>
> James crashes in less than 4min with an OutOfMemoryError. JVM options
> that increase heap size would only make this happen a little bit later
> :-(
>
> Has anyone experienced this before? Any suggestions about a better JVM
> or a different James version? May this come from Avalon?
>
> Regards,
>
>
> Diego
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: James memory leak
Posted by Danny Angus <da...@apache.org>.
I get this too from recent cvs, 100 threads delivering 100 1k mails, Jmeas chokes at around mail 1200, after a couple of minutes.
d.
> -----Original Message-----
> From: Noel J. Bergman [mailto:noel@devtech.com]
> Sent: 11 October 2002 16:46
> To: James Users List; diego.castillo@inexbee.com
> Subject: RE: James memory leak
>
>
> Known issue. Please try the 2.1 code, and let us know what you see.
>
> --- Noel
>
> -----Original Message-----
> From: Diego Castillo [mailto:diego.castillo@inexbee.com]
> Sent: Friday, October 11, 2002 9:29
> To: james-user@jakarta.apache.org
> Cc: Diego Castillo, INEXBEE
> Subject: James memory leak
>
>
> Hi,
>
> I am using James 2.0a3 with JDK 1.3.1 on W2K machine.
>
> I am experiencing a memory leak in my application. During my
> troubleshooting I have finally removed all my mailets in order to test
> James alone. The memory leak persists!
>
> This is my scenario:
> - I have a load generator that sends 10 mail/second. Mails have
> different sizes, with an average of 11Kb.
> - On the James size, I have a mailet that prints the JVM total and free
> memory (in Mb) every 100 messages. Here you have my spoolmanager
> configuration and the results that I get:
>
> <spoolmanager>
> <threads>1</threads>
> <mailetpackages>
> <mailetpackage>org.apache.james.transport.mailets</mailetpackage>
> <mailetpackage>test</mailetpackage>
> </mailetpackages>
> <matcherpackages>
> <matcherpackage>org.apache.james.transport.matchers</matcherpackage>
> </matcherpackages>
> <processor name="root">
> <mailet match="All" class="MemoryPrinter"/>
> <mailet match="All" class="Null"/>
> </processor>
> </spoolmanager>
>
> Mail # Total memory Free memory
> 0 2.5820312 0.5600738
> 100 5.6132812 1.6391296
> 200 9.0234375 2.3683395
> 300 15.128906 5.4931946
> 400 15.128906 3.0394592
> 500 25.488281 10.329994
> 600 25.488281 7.8969116
> 700 25.488281 3.9112778
> 800 25.488281 1.5132217
> 900 42.125 16.21582
> 1000 42.125 12.281708
> 1100 42.125 10.111511
> 1200 42.125 7.9458237
> 1300 42.125 3.9470978
> 1400 42.125 1.725914
> 1500 63.75 23.18071
> 1600 63.75 19.276215
> 1700 63.75 17.146133
> 1800 63.75 14.976295
> 1900 63.75 11.164001
> 2000 63.75 9.04203
> 2100 63.75 7.0878906
> 2200 63.75 4.38665
>
> James crashes in less than 4min with an OutOfMemoryError. JVM options
> that increase heap size would only make this happen a little bit later
> :-(
>
> Has anyone experienced this before? Any suggestions about a better JVM
> or a different James version? May this come from Avalon?
>
> Regards,
>
>
> Diego
>
>
> --
> To unsubscribe, e-mail:
> <ma...@jakarta.apache.org>
> For additional commands, e-mail:
> <ma...@jakarta.apache.org>
>
>
> --
> To unsubscribe, e-mail:
> <ma...@jakarta.apache.org>
> For additional commands, e-mail:
> <ma...@jakarta.apache.org>
>
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: RE : James memory leak
Posted by "Noel J. Bergman" <no...@devtech.com>.
> Has the java.util.Timer class been considered for scheduling ?
The Timer class has the same problem as the Avalon Scheduler implementation.
Neither one is designed for high duty cycle resets. They are schedulers,
not watchdog timers. Harmeet didn't believe me when I explained this, so he
tried it anyway. If you look on the james-dev list, you'll see considerable
discussion on the subject.
> Why doesn't the handler cancel (and delete) itself after execution ?
The interface appears to allow you to remove/reset an entry, but basically
that just marks the entry as removable. I won't go into all of the whys and
wherefores; you can read them on james-dev.
--- Noel
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: RE : James memory leak
Posted by Maxime Levesque <ma...@webhelp.com>.
My question probably belongs to the DEV list, but here it goes anyways :
Has the java.util.Timer class been considered for scheduling ?
I've had good result using it and it would seem hard to do better
(or at least not worth the effort)...
Another question (maybe a dumb one) :
Why doesn't the handler cancel (and delete) itself after execution ?
>----- Original Message -----
>From: "Diego Castillo" <di...@inexbee.com>
>Here go my results.
>
>James 2.1a1 resists 3 mail per second but crashes at 4. Unfortunately,
>the memory leak looks bigger in 2.1a1 than 2.0a3.
>
>Memory is freed periodically, more or less every 6 minutes.
>
>
>
>
>That may be because scheduler thread fires off and removes entries when they
>have been sitting around for 6 mins (360000 in config.xml). If you sed
>config.xml to replace 360000 with say 10000 I would expect the 6 min to get
>reduced.
>
>Please note: This is a theory. I haven't really tested this, but based on
>the code I feel it may be the cause. If you do test this, please post the
>results. Would love to know more about this 6 min recovery cycle.
>Here is what may be happening: Scheduled events (one per handler) go into
>the scheduler. Scheduled events are marked as cancelled after the handler is
>done but not removed when they are marked as cancelled. A background thread,
>based on the timeout goes and removes the entries when they were scheduled
>for execution.
>
>Harmeet
>
>
>--
>To unsubscribe, e-mail: <ma...@jakarta.apache.org>
>For additional commands, e-mail: <ma...@jakarta.apache.org>
>
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: RE : James memory leak
Posted by Harmeet Bedi <ha...@kodemuse.com>.
----- Original Message -----
From: "Diego Castillo" <di...@inexbee.com>
Here go my results.
James 2.1a1 resists 3 mail per second but crashes at 4. Unfortunately,
the memory leak looks bigger in 2.1a1 than 2.0a3.
Memory is freed periodically, more or less every 6 minutes.
That may be because scheduler thread fires off and removes entries when they
have been sitting around for 6 mins (360000 in config.xml). If you sed
config.xml to replace 360000 with say 10000 I would expect the 6 min to get
reduced.
Please note: This is a theory. I haven't really tested this, but based on
the code I feel it may be the cause. If you do test this, please post the
results. Would love to know more about this 6 min recovery cycle.
Here is what may be happening: Scheduled events (one per handler) go into
the scheduler. Scheduled events are marked as cancelled after the handler is
done but not removed when they are marked as cancelled. A background thread,
based on the timeout goes and removes the entries when they were scheduled
for execution.
Harmeet
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: James memory leak
Posted by "Noel J. Bergman" <no...@devtech.com>.
Diego,
The particular memory "leak" you are seeing is the effect of the Avalon
Scheduler. It isn't so much a leak as uncontrolled growth in the
scheduler's priority queue. The next milestone build will eliminate that
issue. There are also other memory leaks. My first goal is to get the
SMTPHandler such that its own internal processing is clean. That has not
been the case until some time late last night / early this morning.
As of now, I have a private test build that runs between 30 and 50 messages
per second without any apparent memory leak in the handler. I can't say
what will happen once the spool is re-engaged, but one step at a time.
Still some work to do in the handler area.
--- Noel
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE : RE : James memory leak
Posted by Diego Castillo <di...@inexbee.com>.
Here go my results.
James 2.1a1 resists 3 mail per second but crashes at 4. Unfortunately,
the memory leak looks bigger in 2.1a1 than 2.0a3.
Memory is freed periodically, more or less every 6 minutes. I guess that
I could increase the maximum JVM heap size so that I do not get the
OutOfMemoryError before the first garbage collector round in order to
increase the maximum mail throughput.
Diego
-----Message d'origine-----
De : Noel J. Bergman [mailto:noel@devtech.com]
Envoyé : mercredi 16 octobre 2002 22:00
À : James Users List
Objet : RE: RE : James memory leak
> Does the leak occur at all with low load (like on mail per second) ?
Depends upon which one. Don't know the answer to the one I've
identified
because it hasn't been isolated yet.
--- Noel
--
To unsubscribe, e-mail:
<ma...@jakarta.apache.org>
For additional commands, e-mail:
<ma...@jakarta.apache.org>
RE: RE : James memory leak
Posted by "Noel J. Bergman" <no...@devtech.com>.
> Does the leak occur at all with low load (like on mail per second) ?
Depends upon which one. Don't know the answer to the one I've identified
because it hasn't been isolated yet.
--- Noel
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: RE : James memory leak
Posted by Maxime Levesque <ma...@webhelp.com>.
Does the leak occur at all with low load (like on mail per second) ?
Diego Castillo wrote:
>Hi Noel,
>
>I upgraded to 2.1a1-2002-09-24 and repeated my test scenario:
> - W2K SP2 + JDK 1.3.1 + James 2.1a1
> - Continuous load of 10 mails/second
>
>JVM heap size increases and causes James to hang up with an
>OutOfMemoryError in less than 2 minutes: same behaviour as in 2.0a3.
>Here go my figures:
>
>Mail # Total memory Free memory
>0 4.6484375 1.7053909
>100 11.8828125 5.2133255
>200 19.972656 7.305504
>300 19.972656 2.5455475
>400 33.347656 9.997406
>500 33.347656 5.209381
>600 54.476562 21.393051
>700 54.476562 15.805206
>800 54.476562 10.835602
>900 54.476562 5.9031982
>1000 54.726562 1.1683655
>1100 63.75 7.1124954
>1200 63.75 0.77523804
>
>I have followed up the thread, but I have not completely understood your
>point on the scheduler issue. Do you mean that the memory leak may go
>off if I reduce the mail flow? Is it possible to estimate a maximum
>throughput for the scheduler issue to disappear?
>
>Thanks,
>
>
>Diego
>
>-----Message d'origine-----
>De : Noel J. Bergman [mailto:noel@devtech.com]
>Envoyé : vendredi 11 octobre 2002 17:46
>À : James Users List; diego.castillo@inexbee.com
>Objet : RE: James memory leak
>
>Known issue. Please try the 2.1 code, and let us know what you see.
>
> --- Noel
>
>-----Original Message-----
>From: Diego Castillo [mailto:diego.castillo@inexbee.com]
>Sent: Friday, October 11, 2002 9:29
>To: james-user@jakarta.apache.org
>Cc: Diego Castillo, INEXBEE
>Subject: James memory leak
>
>
>Hi,
>
>I am using James 2.0a3 with JDK 1.3.1 on W2K machine.
>
>I am experiencing a memory leak in my application. During my
>troubleshooting I have finally removed all my mailets in order to test
>James alone. The memory leak persists!
>
>This is my scenario:
> - I have a load generator that sends 10 mail/second. Mails have
>different sizes, with an average of 11Kb.
> - On the James size, I have a mailet that prints the JVM total and free
>memory (in Mb) every 100 messages. Here you have my spoolmanager
>configuration and the results that I get:
>
><spoolmanager>
> <threads>1</threads>
> <mailetpackages>
> <mailetpackage>org.apache.james.transport.mailets</mailetpackage>
> <mailetpackage>test</mailetpackage>
> </mailetpackages>
> <matcherpackages>
> <matcherpackage>org.apache.james.transport.matchers</matcherpackage>
> </matcherpackages>
> <processor name="root">
> <mailet match="All" class="MemoryPrinter"/>
> <mailet match="All" class="Null"/>
> </processor>
></spoolmanager>
>
>Mail # Total memory Free memory
>0 2.5820312 0.5600738
>100 5.6132812 1.6391296
>200 9.0234375 2.3683395
>300 15.128906 5.4931946
>400 15.128906 3.0394592
>500 25.488281 10.329994
>600 25.488281 7.8969116
>700 25.488281 3.9112778
>800 25.488281 1.5132217
>900 42.125 16.21582
>1000 42.125 12.281708
>1100 42.125 10.111511
>1200 42.125 7.9458237
>1300 42.125 3.9470978
>1400 42.125 1.725914
>1500 63.75 23.18071
>1600 63.75 19.276215
>1700 63.75 17.146133
>1800 63.75 14.976295
>1900 63.75 11.164001
>2000 63.75 9.04203
>2100 63.75 7.0878906
>2200 63.75 4.38665
>
>James crashes in less than 4min with an OutOfMemoryError. JVM options
>that increase heap size would only make this happen a little bit later
>:-(
>
>Has anyone experienced this before? Any suggestions about a better JVM
>or a different James version? May this come from Avalon?
>
>Regards,
>
>
>Diego
>
>
>--
>To unsubscribe, e-mail:
><ma...@jakarta.apache.org>
>For additional commands, e-mail:
><ma...@jakarta.apache.org>
>
>
>--
>To unsubscribe, e-mail:
><ma...@jakarta.apache.org>
>For additional commands, e-mail:
><ma...@jakarta.apache.org>
>
>
>--
>To unsubscribe, e-mail: <ma...@jakarta.apache.org>
>For additional commands, e-mail: <ma...@jakarta.apache.org>
>
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: RE : James memory leak
Posted by "Noel J. Bergman" <no...@devtech.com>.
Maxime,
I am running load tests against James on an internal network. As of now, I
can push between 30 and 50 messages per second, without any apparent leak in
the SMTP handler.
For testing, I use Russell Coker's postal program:
http://www.coker.com.au/postal/. The James project is also to building a
test suite.
--- Noel
-----Original Message-----
From: Maxime Levesque [mailto:maxime.levesque@webhelp.com]
Sent: Thursday, October 17, 2002 11:57
To: James Users List
Subject: Re: RE : James memory leak
I'd like to run your load test (James would be very usefull for our
product but only without the leak !)
I'll give a shot at trying to solve it...
I'd like to use your test suite, can you send it to me (or post it to
the list) ?
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE : RE : James memory leak
Posted by Diego Castillo <di...@inexbee.com>.
Hi,
Sorry for the delay, I have been very busy lately. Here goes my test
suite.
I have a load generator program that sends all the mails stored in a
directory. I also have an Ant target to deploy my mailets to James
2.0a3:
- Serializer permits saving to a file a mail that you create with
Outlook and send via James.
- MemoryPrinter prints memory every 100 mails (based on the mail
subject set by the load generator).
It is very simple, but I hope that it meets your expectations.
Diego
-----Message d'origine-----
De : Maxime Levesque [mailto:maxime.levesque@webhelp.com]
Envoyé : jeudi 17 octobre 2002 17:57
À : James Users List
Objet : Re: RE : James memory leak
I'd like to run your load test (James would be very usefull for our
product but only without the leak !)
I'll give a shot at trying to solve it...
I'd like to use your test suite, can you send it to me (or post it to
the list) ?
>Hi Noel,
>
>I upgraded to 2.1a1-2002-09-24 and repeated my test scenario:
> - W2K SP2 + JDK 1.3.1 + James 2.1a1
> - Continuous load of 10 mails/second
>
>JVM heap size increases and causes James to hang up with an
>OutOfMemoryError in less than 2 minutes: same behaviour as in 2.0a3.
>Here go my figures:
>
>Mail # Total memory Free memory
>0 4.6484375 1.7053909
>100 11.8828125 5.2133255
>200 19.972656 7.305504
>300 19.972656 2.5455475
>400 33.347656 9.997406
>500 33.347656 5.209381
>600 54.476562 21.393051
>700 54.476562 15.805206
>800 54.476562 10.835602
>900 54.476562 5.9031982
>1000 54.726562 1.1683655
>1100 63.75 7.1124954
>1200 63.75 0.77523804
>
>I have followed up the thread, but I have not completely understood
your
>point on the scheduler issue. Do you mean that the memory leak may go
>off if I reduce the mail flow? Is it possible to estimate a maximum
>throughput for the scheduler issue to disappear?
>
>Thanks,
>
>
>Diego
>
>-----Message d'origine-----
>De : Noel J. Bergman [mailto:noel@devtech.com]
>Envoyé : vendredi 11 octobre 2002 17:46
>À : James Users List; diego.castillo@inexbee.com
>Objet : RE: James memory leak
>
>Known issue. Please try the 2.1 code, and let us know what you see.
>
> --- Noel
>
>-----Original Message-----
>From: Diego Castillo [mailto:diego.castillo@inexbee.com]
>Sent: Friday, October 11, 2002 9:29
>To: james-user@jakarta.apache.org
>Cc: Diego Castillo, INEXBEE
>Subject: James memory leak
>
>
>Hi,
>
>I am using James 2.0a3 with JDK 1.3.1 on W2K machine.
>
>I am experiencing a memory leak in my application. During my
>troubleshooting I have finally removed all my mailets in order to test
>James alone. The memory leak persists!
>
>This is my scenario:
> - I have a load generator that sends 10 mail/second. Mails have
>different sizes, with an average of 11Kb.
> - On the James size, I have a mailet that prints the JVM total and
free
>memory (in Mb) every 100 messages. Here you have my spoolmanager
>configuration and the results that I get:
>
><spoolmanager>
> <threads>1</threads>
> <mailetpackages>
> <mailetpackage>org.apache.james.transport.mailets</mailetpackage>
> <mailetpackage>test</mailetpackage>
> </mailetpackages>
> <matcherpackages>
>
<matcherpackage>org.apache.james.transport.matchers</matcherpackage>
> </matcherpackages>
> <processor name="root">
> <mailet match="All" class="MemoryPrinter"/>
> <mailet match="All" class="Null"/>
> </processor>
></spoolmanager>
>
>Mail # Total memory Free memory
>0 2.5820312 0.5600738
>100 5.6132812 1.6391296
>200 9.0234375 2.3683395
>300 15.128906 5.4931946
>400 15.128906 3.0394592
>500 25.488281 10.329994
>600 25.488281 7.8969116
>700 25.488281 3.9112778
>800 25.488281 1.5132217
>900 42.125 16.21582
>1000 42.125 12.281708
>1100 42.125 10.111511
>1200 42.125 7.9458237
>1300 42.125 3.9470978
>1400 42.125 1.725914
>1500 63.75 23.18071
>1600 63.75 19.276215
>1700 63.75 17.146133
>1800 63.75 14.976295
>1900 63.75 11.164001
>2000 63.75 9.04203
>2100 63.75 7.0878906
>2200 63.75 4.38665
>
>James crashes in less than 4min with an OutOfMemoryError. JVM options
>that increase heap size would only make this happen a little bit later
>:-(
>
>Has anyone experienced this before? Any suggestions about a better JVM
>or a different James version? May this come from Avalon?
>
>Regards,
>
>
>Diego
>
>
>--
>To unsubscribe, e-mail:
><ma...@jakarta.apache.org>
>For additional commands, e-mail:
><ma...@jakarta.apache.org>
>
>
>--
>To unsubscribe, e-mail:
><ma...@jakarta.apache.org>
>For additional commands, e-mail:
><ma...@jakarta.apache.org>
>
>
>--
>To unsubscribe, e-mail:
<ma...@jakarta.apache.org>
>For additional commands, e-mail:
<ma...@jakarta.apache.org>
>
--
To unsubscribe, e-mail:
<ma...@jakarta.apache.org>
For additional commands, e-mail:
<ma...@jakarta.apache.org>
Re: RE : James memory leak
Posted by Maxime Levesque <ma...@webhelp.com>.
I'd like to run your load test (James would be very usefull for our
product but only without the leak !)
I'll give a shot at trying to solve it...
I'd like to use your test suite, can you send it to me (or post it to
the list) ?
>Hi Noel,
>
>I upgraded to 2.1a1-2002-09-24 and repeated my test scenario:
> - W2K SP2 + JDK 1.3.1 + James 2.1a1
> - Continuous load of 10 mails/second
>
>JVM heap size increases and causes James to hang up with an
>OutOfMemoryError in less than 2 minutes: same behaviour as in 2.0a3.
>Here go my figures:
>
>Mail # Total memory Free memory
>0 4.6484375 1.7053909
>100 11.8828125 5.2133255
>200 19.972656 7.305504
>300 19.972656 2.5455475
>400 33.347656 9.997406
>500 33.347656 5.209381
>600 54.476562 21.393051
>700 54.476562 15.805206
>800 54.476562 10.835602
>900 54.476562 5.9031982
>1000 54.726562 1.1683655
>1100 63.75 7.1124954
>1200 63.75 0.77523804
>
>I have followed up the thread, but I have not completely understood your
>point on the scheduler issue. Do you mean that the memory leak may go
>off if I reduce the mail flow? Is it possible to estimate a maximum
>throughput for the scheduler issue to disappear?
>
>Thanks,
>
>
>Diego
>
>-----Message d'origine-----
>De : Noel J. Bergman [mailto:noel@devtech.com]
>Envoyé : vendredi 11 octobre 2002 17:46
>À : James Users List; diego.castillo@inexbee.com
>Objet : RE: James memory leak
>
>Known issue. Please try the 2.1 code, and let us know what you see.
>
> --- Noel
>
>-----Original Message-----
>From: Diego Castillo [mailto:diego.castillo@inexbee.com]
>Sent: Friday, October 11, 2002 9:29
>To: james-user@jakarta.apache.org
>Cc: Diego Castillo, INEXBEE
>Subject: James memory leak
>
>
>Hi,
>
>I am using James 2.0a3 with JDK 1.3.1 on W2K machine.
>
>I am experiencing a memory leak in my application. During my
>troubleshooting I have finally removed all my mailets in order to test
>James alone. The memory leak persists!
>
>This is my scenario:
> - I have a load generator that sends 10 mail/second. Mails have
>different sizes, with an average of 11Kb.
> - On the James size, I have a mailet that prints the JVM total and free
>memory (in Mb) every 100 messages. Here you have my spoolmanager
>configuration and the results that I get:
>
><spoolmanager>
> <threads>1</threads>
> <mailetpackages>
> <mailetpackage>org.apache.james.transport.mailets</mailetpackage>
> <mailetpackage>test</mailetpackage>
> </mailetpackages>
> <matcherpackages>
> <matcherpackage>org.apache.james.transport.matchers</matcherpackage>
> </matcherpackages>
> <processor name="root">
> <mailet match="All" class="MemoryPrinter"/>
> <mailet match="All" class="Null"/>
> </processor>
></spoolmanager>
>
>Mail # Total memory Free memory
>0 2.5820312 0.5600738
>100 5.6132812 1.6391296
>200 9.0234375 2.3683395
>300 15.128906 5.4931946
>400 15.128906 3.0394592
>500 25.488281 10.329994
>600 25.488281 7.8969116
>700 25.488281 3.9112778
>800 25.488281 1.5132217
>900 42.125 16.21582
>1000 42.125 12.281708
>1100 42.125 10.111511
>1200 42.125 7.9458237
>1300 42.125 3.9470978
>1400 42.125 1.725914
>1500 63.75 23.18071
>1600 63.75 19.276215
>1700 63.75 17.146133
>1800 63.75 14.976295
>1900 63.75 11.164001
>2000 63.75 9.04203
>2100 63.75 7.0878906
>2200 63.75 4.38665
>
>James crashes in less than 4min with an OutOfMemoryError. JVM options
>that increase heap size would only make this happen a little bit later
>:-(
>
>Has anyone experienced this before? Any suggestions about a better JVM
>or a different James version? May this come from Avalon?
>
>Regards,
>
>
>Diego
>
>
>--
>To unsubscribe, e-mail:
><ma...@jakarta.apache.org>
>For additional commands, e-mail:
><ma...@jakarta.apache.org>
>
>
>--
>To unsubscribe, e-mail:
><ma...@jakarta.apache.org>
>For additional commands, e-mail:
><ma...@jakarta.apache.org>
>
>
>--
>To unsubscribe, e-mail: <ma...@jakarta.apache.org>
>For additional commands, e-mail: <ma...@jakarta.apache.org>
>
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: RE : James memory leak
Posted by "Noel J. Bergman" <no...@devtech.com>.
Diego,
I'm sure it does. The problem you are seeing is most likely caused by a
known issue, but even with two different fixes for that issue, I am still
seeing a problem, even though not the one you're seeing. If you check the
James Developer's mailing list, you'll see the reports, and details. The
difference is that it takes 15 minutes at over 2000 messages per minute to
consume about 128MB of free memory.
--- Noel
-----Original Message-----
From: Diego Castillo [mailto:diego.castillo@inexbee.com]
Sent: Wednesday, October 16, 2002 7:12
To: 'James Users List'
Subject: RE : James memory leak
Hi Noel,
I upgraded to 2.1a1-2002-09-24 and repeated my test scenario:
- W2K SP2 + JDK 1.3.1 + James 2.1a1
- Continuous load of 10 mails/second
JVM heap size increases and causes James to hang up with an
OutOfMemoryError in less than 2 minutes: same behaviour as in 2.0a3.
Here go my figures:
Mail # Total memory Free memory
0 4.6484375 1.7053909
100 11.8828125 5.2133255
200 19.972656 7.305504
300 19.972656 2.5455475
400 33.347656 9.997406
500 33.347656 5.209381
600 54.476562 21.393051
700 54.476562 15.805206
800 54.476562 10.835602
900 54.476562 5.9031982
1000 54.726562 1.1683655
1100 63.75 7.1124954
1200 63.75 0.77523804
I have followed up the thread, but I have not completely understood your
point on the scheduler issue. Do you mean that the memory leak may go
off if I reduce the mail flow? Is it possible to estimate a maximum
throughput for the scheduler issue to disappear?
Thanks,
Diego
-----Message d'origine-----
De : Noel J. Bergman [mailto:noel@devtech.com]
Envoyé : vendredi 11 octobre 2002 17:46
À : James Users List; diego.castillo@inexbee.com
Objet : RE: James memory leak
Known issue. Please try the 2.1 code, and let us know what you see.
--- Noel
-----Original Message-----
From: Diego Castillo [mailto:diego.castillo@inexbee.com]
Sent: Friday, October 11, 2002 9:29
To: james-user@jakarta.apache.org
Cc: Diego Castillo, INEXBEE
Subject: James memory leak
Hi,
I am using James 2.0a3 with JDK 1.3.1 on W2K machine.
I am experiencing a memory leak in my application. During my
troubleshooting I have finally removed all my mailets in order to test
James alone. The memory leak persists!
This is my scenario:
- I have a load generator that sends 10 mail/second. Mails have
different sizes, with an average of 11Kb.
- On the James size, I have a mailet that prints the JVM total and free
memory (in Mb) every 100 messages. Here you have my spoolmanager
configuration and the results that I get:
<spoolmanager>
<threads>1</threads>
<mailetpackages>
<mailetpackage>org.apache.james.transport.mailets</mailetpackage>
<mailetpackage>test</mailetpackage>
</mailetpackages>
<matcherpackages>
<matcherpackage>org.apache.james.transport.matchers</matcherpackage>
</matcherpackages>
<processor name="root">
<mailet match="All" class="MemoryPrinter"/>
<mailet match="All" class="Null"/>
</processor>
</spoolmanager>
Mail # Total memory Free memory
0 2.5820312 0.5600738
100 5.6132812 1.6391296
200 9.0234375 2.3683395
300 15.128906 5.4931946
400 15.128906 3.0394592
500 25.488281 10.329994
600 25.488281 7.8969116
700 25.488281 3.9112778
800 25.488281 1.5132217
900 42.125 16.21582
1000 42.125 12.281708
1100 42.125 10.111511
1200 42.125 7.9458237
1300 42.125 3.9470978
1400 42.125 1.725914
1500 63.75 23.18071
1600 63.75 19.276215
1700 63.75 17.146133
1800 63.75 14.976295
1900 63.75 11.164001
2000 63.75 9.04203
2100 63.75 7.0878906
2200 63.75 4.38665
James crashes in less than 4min with an OutOfMemoryError. JVM options
that increase heap size would only make this happen a little bit later
:-(
Has anyone experienced this before? Any suggestions about a better JVM
or a different James version? May this come from Avalon?
Regards,
Diego
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE : James memory leak
Posted by Diego Castillo <di...@inexbee.com>.
Hi Noel,
I upgraded to 2.1a1-2002-09-24 and repeated my test scenario:
- W2K SP2 + JDK 1.3.1 + James 2.1a1
- Continuous load of 10 mails/second
JVM heap size increases and causes James to hang up with an
OutOfMemoryError in less than 2 minutes: same behaviour as in 2.0a3.
Here go my figures:
Mail # Total memory Free memory
0 4.6484375 1.7053909
100 11.8828125 5.2133255
200 19.972656 7.305504
300 19.972656 2.5455475
400 33.347656 9.997406
500 33.347656 5.209381
600 54.476562 21.393051
700 54.476562 15.805206
800 54.476562 10.835602
900 54.476562 5.9031982
1000 54.726562 1.1683655
1100 63.75 7.1124954
1200 63.75 0.77523804
I have followed up the thread, but I have not completely understood your
point on the scheduler issue. Do you mean that the memory leak may go
off if I reduce the mail flow? Is it possible to estimate a maximum
throughput for the scheduler issue to disappear?
Thanks,
Diego
-----Message d'origine-----
De : Noel J. Bergman [mailto:noel@devtech.com]
Envoyé : vendredi 11 octobre 2002 17:46
À : James Users List; diego.castillo@inexbee.com
Objet : RE: James memory leak
Known issue. Please try the 2.1 code, and let us know what you see.
--- Noel
-----Original Message-----
From: Diego Castillo [mailto:diego.castillo@inexbee.com]
Sent: Friday, October 11, 2002 9:29
To: james-user@jakarta.apache.org
Cc: Diego Castillo, INEXBEE
Subject: James memory leak
Hi,
I am using James 2.0a3 with JDK 1.3.1 on W2K machine.
I am experiencing a memory leak in my application. During my
troubleshooting I have finally removed all my mailets in order to test
James alone. The memory leak persists!
This is my scenario:
- I have a load generator that sends 10 mail/second. Mails have
different sizes, with an average of 11Kb.
- On the James size, I have a mailet that prints the JVM total and free
memory (in Mb) every 100 messages. Here you have my spoolmanager
configuration and the results that I get:
<spoolmanager>
<threads>1</threads>
<mailetpackages>
<mailetpackage>org.apache.james.transport.mailets</mailetpackage>
<mailetpackage>test</mailetpackage>
</mailetpackages>
<matcherpackages>
<matcherpackage>org.apache.james.transport.matchers</matcherpackage>
</matcherpackages>
<processor name="root">
<mailet match="All" class="MemoryPrinter"/>
<mailet match="All" class="Null"/>
</processor>
</spoolmanager>
Mail # Total memory Free memory
0 2.5820312 0.5600738
100 5.6132812 1.6391296
200 9.0234375 2.3683395
300 15.128906 5.4931946
400 15.128906 3.0394592
500 25.488281 10.329994
600 25.488281 7.8969116
700 25.488281 3.9112778
800 25.488281 1.5132217
900 42.125 16.21582
1000 42.125 12.281708
1100 42.125 10.111511
1200 42.125 7.9458237
1300 42.125 3.9470978
1400 42.125 1.725914
1500 63.75 23.18071
1600 63.75 19.276215
1700 63.75 17.146133
1800 63.75 14.976295
1900 63.75 11.164001
2000 63.75 9.04203
2100 63.75 7.0878906
2200 63.75 4.38665
James crashes in less than 4min with an OutOfMemoryError. JVM options
that increase heap size would only make this happen a little bit later
:-(
Has anyone experienced this before? Any suggestions about a better JVM
or a different James version? May this come from Avalon?
Regards,
Diego
--
To unsubscribe, e-mail:
<ma...@jakarta.apache.org>
For additional commands, e-mail:
<ma...@jakarta.apache.org>
--
To unsubscribe, e-mail:
<ma...@jakarta.apache.org>
For additional commands, e-mail:
<ma...@jakarta.apache.org>
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: James memory leak
Posted by "Noel J. Bergman" <no...@devtech.com>.
Known issue. Please try the 2.1 code, and let us know what you see.
--- Noel
-----Original Message-----
From: Diego Castillo [mailto:diego.castillo@inexbee.com]
Sent: Friday, October 11, 2002 9:29
To: james-user@jakarta.apache.org
Cc: Diego Castillo, INEXBEE
Subject: James memory leak
Hi,
I am using James 2.0a3 with JDK 1.3.1 on W2K machine.
I am experiencing a memory leak in my application. During my
troubleshooting I have finally removed all my mailets in order to test
James alone. The memory leak persists!
This is my scenario:
- I have a load generator that sends 10 mail/second. Mails have
different sizes, with an average of 11Kb.
- On the James size, I have a mailet that prints the JVM total and free
memory (in Mb) every 100 messages. Here you have my spoolmanager
configuration and the results that I get:
<spoolmanager>
<threads>1</threads>
<mailetpackages>
<mailetpackage>org.apache.james.transport.mailets</mailetpackage>
<mailetpackage>test</mailetpackage>
</mailetpackages>
<matcherpackages>
<matcherpackage>org.apache.james.transport.matchers</matcherpackage>
</matcherpackages>
<processor name="root">
<mailet match="All" class="MemoryPrinter"/>
<mailet match="All" class="Null"/>
</processor>
</spoolmanager>
Mail # Total memory Free memory
0 2.5820312 0.5600738
100 5.6132812 1.6391296
200 9.0234375 2.3683395
300 15.128906 5.4931946
400 15.128906 3.0394592
500 25.488281 10.329994
600 25.488281 7.8969116
700 25.488281 3.9112778
800 25.488281 1.5132217
900 42.125 16.21582
1000 42.125 12.281708
1100 42.125 10.111511
1200 42.125 7.9458237
1300 42.125 3.9470978
1400 42.125 1.725914
1500 63.75 23.18071
1600 63.75 19.276215
1700 63.75 17.146133
1800 63.75 14.976295
1900 63.75 11.164001
2000 63.75 9.04203
2100 63.75 7.0878906
2200 63.75 4.38665
James crashes in less than 4min with an OutOfMemoryError. JVM options
that increase heap size would only make this happen a little bit later
:-(
Has anyone experienced this before? Any suggestions about a better JVM
or a different James version? May this come from Avalon?
Regards,
Diego
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>