You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by "Reich, Matthias" <ma...@siemens.com> on 2007/04/24 16:00:55 UTC
Memory Leak with Comet
Hello,
I am sorry that I did not address the memory leak issue to the users
list first, before sending this bug report:
http://issues.apache.org/bugzilla/show_bug.cgi?id=42217
I am aware of the fact that memory allocation occurs in Java progams,
and that Tomcat will exploit some resource pools before resources are
recycled.
I am afraid that I did not consider that I should have been more
explicit in my description.
If memory allocation of Tomcat continues until I see messages like
org.apache.coyote.http11.Http11NioProcessor: Error processing request
java.lang.OutOfMemoryError: Java heap space
when I run my simple test webapp with JAVA_OPTS=-Xmx512m, this looks for
me as if there is a memory leak.
Surely, the memory leak could be in my Servlet, and the queue which
holds events until they are answered asynchronously is the first
candidate to look at in more detail.
For this reason, the Servlet logs the size of the queue whenever an
element is removed from the queue.
I saw that the queue size did not increase over the time, but memory
consumption of Tomcat did.
Therefore, I am pretty sure that there is either a memory leak in Tomcat
or I still don't understand how the Comet interfaces shall be used.
If I modify the index page of my webapp in a way that it sends only
requests to the 'traditional' servlet, memory does not increase.
Also, if it sends requests to the Comet servlet which are answered
synchronously, memory consumption is stable.
Only if comet requests are answered asynchronously, memory does
increase.
I did not always wait until I saw the OutOfMemoryError, but the effect
is reproducible on my machine.
Any opinions about this?
Regards,
Matthias Reich
Re: Memory Leak with Comet
Posted by Rémy Maucherat <re...@gmail.com>.
On 4/25/07, Reich, Matthias <ma...@siemens.com> wrote:
> So far, I have the feeling that I am the only one who has seen these memory leaks with asynchronous Comet responses.
>
> I tried with a number of variations, e.g.
> - let the browser wait for 500 millis before sending the next poll after receiving a result
> - keep the events queued in the serverfor at least 500 millis before sending the response
> - modify synchronization and the places where I invoke event.close()
> - tried with connector attribute maxKeepAliveRequests="-1" and maxKeepAliveRequests="2"
> but the effect is always the same.
>
> Thus, I doubt that it is really 'wrong' usage of the interfaces, and others should have seen similar effects.
>
> Has anyone out there run an application with browsers as clients and with asynchronous responses for a longer time without loosing memory?
>
> Or am I the only one so far working on such a scenario?
Processors are only recycled when the IO layer gets an event (read,
disconnect, timeout), and if:
- the event is a non timeout error
- the event is closed by the servlet
- the event has been closed asynchronously
If the client is sitting there without sending any data (if it
disconnects, it becomes either a read event or an error/disconnect
event, so it's the same), then you have to wait for the timeout (which
can be set per event with the NIO connector - this is not doable at
this time for the APR connector).
Rémy
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Memory Leak with Comet
Posted by Rémy Maucherat <re...@gmail.com>.
On 4/28/07, Reich, Matthias <ma...@siemens.com> wrote:
> The problem is that the socket is added to the poller already within the
> Http11AprProcessor.event method.
> Due to this the process method can be invoked before the event method
> has done it's cleanup.
I don't really understand how it can happen in the real world, but
since the processor event method does return the socket to a poller,
it is indeed possible that one processor gets lost during the
execution of the process method if the "new" processor is mistakenly
removed concurrently by the event method.
Rémy
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Memory Leak with Comet
Posted by "Reich, Matthias" <ma...@siemens.com>.
Hello again,
I would like to give you another chance to see the situation where the
processing objects are not recycled.
After all my observations, I had the feeling that the reason for the
non-recycled processing objects must be problems with concurrent access
to queues or maps that hold these objects, and that it should not be TOO
hard to find it - and I found it.
I modified the lines that are marked in the following code fragments
from class Http11AprProtocol to verify it:
public SocketState event(long socket, SocketStatus status) {
Http11AprProcessor result = connections.get(socket);
SocketState state = SocketState.CLOSED;
if (result != null) {
// Call the appropriate event
try {
state = result.event(status);
....
} finally {
if (state != SocketState.LONG) {
>>>> Http11AprProcessor removed =
connections.remove(socket);
>>>> if (removed != result) {
>>>> Http11AprProtocol.log.error ("removed wrong
processor from connections") ;
>>>> }
recycledProcessors.offer(result);
}
}
}
return state;
}
public SocketState process(long socket) {
Http11AprProcessor processor = recycledProcessors.poll();
try {
if (processor == null) {
processor = createProcessor();
}
if (processor instanceof ActionHook) {
((ActionHook)
processor).action(ActionCode.ACTION_START, null);
}
SocketState state = processor.process(socket);
if (state == SocketState.LONG) {
// Associate the connection with the processor. The
next request
// processed by this thread will use either a new or
a recycled
// processor.
>>>> Http11AprProcessor replaced =
connections.put(socket, processor);
>>>> if (replaced != null) {
>>>> Http11AprProtocol.log.error ("there was still
another processor assigned to the socket - it is not recycled!");
>>>> }
proto.endpoint.getCometPoller().add(socket);
} else {
recycledProcessors.offer(processor);
}
...
Now I ran my test application again, keeping the connection alive
between subsequent requests of the browser.
This is what I found in the log file:
WARNUNG: BEGIN(4) POST /comettest/comet/request?action=poll&count=4
28.04.2007 00:20:35 comettest.CometServlet$EventProvider run
WARNUNG: queue size: 1
28.04.2007 00:20:35 comettest.CometServlet$EventProvider sendResponse
WARNUNG: RESPONSE(4)
28.04.2007 00:20:35 comettest.CometServlet$EventProvider run
WARNUNG: queue size: 0
28.04.2007 00:20:35 comettest.CometServlet$EventProvider run
WARNUNG: sleeping 900 millis
28.04.2007 00:20:35 comettest.CometServlet event
WARNUNG: READ(4) POST /comettest/comet/request?action=poll&count=4,
stream closed
28.04.2007 00:20:35 comettest.CometServlet closeEvent
WARNUNG: CLOSE(4) by event processor
28.04.2007 00:20:35 comettest.CometServlet event
WARNUNG: BEGIN(5) POST /comettest/comet/request?action=poll&count=5
28.04.2007 00:20:35
org.apache.coyote.http11.Http11AprProtocol$Http11ConnectionHandler
process
SCHWERWIEGEND: there was still another processor assigned to the socket
- it is not recycled!
28.04.2007 00:20:35
org.apache.coyote.http11.Http11AprProtocol$Http11ConnectionHandler event
SCHWERWIEGEND: removed wrong processor from connections
This situation does not occur with each request - it seems to depends on
the actual thread scheduling.
The problem is that the socket is added to the poller already within the
Http11AprProcessor.event method.
Due to this the process method can be invoked before the event method
has done it's cleanup.
This time I am a little caucious with advice how to fix the problem, but
I hope I can convince you now that the problem exists ;-)
Matthias
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Memory Leak with Comet
Posted by Rémy Maucherat <re...@gmail.com>.
On 4/26/07, Reich, Matthias <ma...@siemens.com> wrote:
> Thanks for the clear statement!
> As I do see processing objects that are not recycled, I now know that I have to postpone my efforts of using the CometProcessor interface. Maybe I'll retry with Tomcat 6.1 when it is released ;-)
>
> If some day you like to take a look at a situation where processing objects are not recycled - feel free to ask me for an updated version of my test webapp which reflects my findings of this week.
>
No problem.
Rémy
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Memory Leak with Comet
Posted by "Reich, Matthias" <ma...@siemens.com>.
Thanks for the clear statement!
As I do see processing objects that are not recycled, I now know that I have to postpone my efforts of using the CometProcessor interface. Maybe I'll retry with Tomcat 6.1 when it is released ;-)
If some day you like to take a look at a situation where processing objects are not recycled - feel free to ask me for an updated version of my test webapp which reflects my findings of this week.
Regards,
Matthias
> -----Original Message-----
> From: Rémy Maucherat [mailto:remy.maucherat@gmail.com]
> Sent: Thursday, April 26, 2007 1:04 PM
> To: Tomcat Users List
> Subject: Re: Memory Leak with Comet
>
> On 4/26/07, Reich, Matthias <ma...@siemens.com> wrote:
> > The clear semantics of this approach would be: Processing
> is the same as for synchronous requests , except that the
> RequestProcessor thread is released from processing early,
> and another thread (or several threads that synchronize when
> accessing the request) continues processing at any time it
> wants, and the application fully controls the lifetime of
> CometRequest instances.
> >
>
> Thanks for the theories and the suggestions. However, it is not going
> to be possible to make the API changes you suggested (since I don't
> like them), and I also don't see a situation where the processing
> objects are not recycled.
>
> Rémy
>
> ---------------------------------------------------------------------
> To start a new topic, e-mail: users@tomcat.apache.org
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Memory Leak with Comet
Posted by Rémy Maucherat <re...@gmail.com>.
On 4/26/07, Reich, Matthias <ma...@siemens.com> wrote:
> The clear semantics of this approach would be: Processing is the same as for synchronous requests , except that the RequestProcessor thread is released from processing early, and another thread (or several threads that synchronize when accessing the request) continues processing at any time it wants, and the application fully controls the lifetime of CometRequest instances.
>
Thanks for the theories and the suggestions. However, it is not going
to be possible to make the API changes you suggested (since I don't
like them), and I also don't see a situation where the processing
objects are not recycled.
Rémy
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Memory Leak with Comet
Posted by "Reich, Matthias" <ma...@siemens.com>.
Excuse me that I left you alone during the discussion yesterday!
Let me clarify: The scenario is that the client polls for events.
Thus, it sends a subsequent poll request once it got an answer from the server.
(And how should I bring Tomcat to create hundreds of request objects with four browsers, if each of them issues only a single request?)
In the log file I can see a very regular pattern (where request parameter count=80 is the client side request count
and (80) s the server side request count)
25.04.2007 19:41:53 comettest.CometServlet event
WARNUNG: BEGIN(80) POST /comettest/comet/request?action=poll&count=80
25.04.2007 19:41:53 comettest.CometServlet$EventProvider run
WARNUNG: queue size: 1
25.04.2007 19:41:53 comettest.CometServlet$EventProvider sendResponse
WARNUNG: RESPONSE(80)
25.04.2007 19:41:53 comettest.CometServlet closeEvent
WARNUNG: CLOSE(80) by response provider
25.04.2007 19:41:53 comettest.CometServlet$EventProvider run
WARNUNG: queue size: 0
25.04.2007 19:41:53 comettest.CometServlet$EventProvider run
WARNUNG: sleeping 750 millis
25.04.2007 19:41:54 comettest.CometServlet event
WARNUNG: END(80) POST /comettest/comet/request?action=poll&count=80
25.04.2007 19:41:54 comettest.CometServlet closeEvent
WARNUNG: CLOSE(80) by event processor
25.04.2007 19:41:54 comettest.CometServlet event
WARNUNG: BEGIN(81) POST /comettest/comet/request?action=poll&count=81
Now I look at this statement from a former post again:
> Processors are only recycled when the IO layer gets an event (read,
> disconnect, timeout), and if:
> - the event is a non timeout error
> - the event is closed by the servlet
> - the event has been closed asynchronously
I must admit that there is no read, disconnect, or timeout event - but there is an END event,
and as the CoyoteAdapter sets the type to END instead READ because the event was closed before,
I think the mentioned precondition for recycling is fulfilled.
The servlet was able to process 467 requests before the OutOfMemoryError occurred,
but in the memory dump I can see only 103 processor instances, thus the majority of the Processors was in fact recycled.
The mismatch might result from the fact the browser sometimes closed the connection in between, and sometimes did not - see below.
If I replace the event.close() in the response provider thread with an outputStream.close(), I also get an END event, regardless if I set Connection: close or not, and also with the NIOConnector.
But there is a difference: If the response provider thread sets the Connection: close header before closing the stream,
memory consumption is fairly stable for a long time (I let it run for over an hour and saw no significant increase in the Windows task manager), so let us assume that recycling works fine in that situation.
MY CONCLUSION: I have to closely couple the connection lifecycle with the request lifecycle to make things work.
However, this is bad news for me - I would like to keep connections open as long as possible,
because especially with an SSL connector opening a new connection after each poll request is expensive.
(And what about malicious clients that ignore the Connection: close ?)
If I try to keep the connection open (i.e. the content length in the response header and flush the OutputStream, but don't close it),
the client sends the subsequent request on the same connection.
The RequestProcessor for the old Request (yes, Request and not Connection - that's why it is called RequestProcessor and not ConnectionProcessor) issues a READ event in this case.
The Servlet tries to read and gets -1. This means that it does not actually read from the connection - otherwise it would read the unparsed subsequent request. (We could say that, upon a READ event, the servlet reads further portions of the request, it does not simple read from the connection).
As it got -1, the Servlet calls event.close() which means: I am done with this request (it does not mean: I am done with the connection).
Then a new (or a recycled) RequestProcessor is associated with the same connection and processes the BEGIN event for the new request, but it seems as if the old RequestProcessor is not always recycled.
I think that recycling could be organized much clearer with a small change in the execution model which would also have the benefit
of descibing the lifecycle of asynchronous requests more clearly:
- First of all, I would like to rename CometEvent into CometRequest, and let it be part of the contract between container and Servlet that one instance of this class represents a request during the whole request lifecycle.
(I.e. the Servlet is allowed to hold references to CometRequest objects for further processing)
- Recycling of a CometRequest (and all associated objects including the RequestProcessor) should be shifted completely to the CometRequest.close() method.
- The only restriction in that case would be that CometRequest.close then cannot be invoked from within the CometProcessor.event() method (Obviously the RequestProcessor cannot recycle itself while it is processing a request).
For regular request processing CometRequest.close will be invoked by a different thread anyway.
There could be some overhead for handling ERROR events as these must be passed to another thread for closing the CometRequest,
but hopefully we could get rid of most of such situations with that change.
E.g. I would expect that a READ event after sending the response (and keeping the connection alive)
would no longer happen if the sender of the response uses CometRequest.close, because the RequestProcessor would then be detached from the connection early enough.
There would surely be a need for READ events to enable the Servlet to trigger further processing by other threads
(i.e. another thread would actually read from the input stream and react to an IOException or a -1 response).
ERROR events will be required to enable the Servlet to trigger some cleanup in another thread (e.g. remove the CometRequest instance from internal queues ...), including a call to CometRequest.close.
The clear semantics of this approach would be: Processing is the same as for synchronous requests , except that the RequestProcessor thread is released from processing early, and another thread (or several threads that synchronize when accessing the request) continues processing at any time it wants, and the application fully controls the lifetime of CometRequest instances.
Matthias
-----Original Message-----
From: Rémy Maucherat [mailto:remy.maucherat@gmail.com]
Sent: Wednesday, April 25, 2007 6:42 PM
To: Tomcat Users List
Subject: Re: Memory Leak with Comet
On 4/25/07, Filip Hanik - Dev Lists <de...@hanik.com> wrote:
> then it's normal behavior, he'll just have to wait for a timeout or the
> client will disconnect
I don't know for sure, of course, but it's my theory at the moment.
Rémy
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Memory Leak with Comet
Posted by Rémy Maucherat <re...@gmail.com>.
On 4/25/07, Filip Hanik - Dev Lists <de...@hanik.com> wrote:
> then it's normal behavior, he'll just have to wait for a timeout or the
> client will disconnect
I don't know for sure, of course, but it's my theory at the moment.
Rémy
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Memory Leak with Comet
Posted by Filip Hanik - Dev Lists <de...@hanik.com>.
Rémy Maucherat wrote:
> On 4/25/07, Filip Hanik - Dev Lists <de...@hanik.com> wrote:
>> Do you have a small test case,
>> if nothing else, your connections should eventually timeout and that
>> should recycle the processors.
>> As long as the comet connection is active, the processor is spoken for.
>
> He submitted a test webapp that generated a full response
> asynchronously, but I don't see anything which wouldn't cause the
> client to just "sit there" after the response is sent. As a result, I
> don't see any way to get an IO event (unless the client disconnects -
> which happens if there's a Connection: close header -, the client
> sends a subsequent request, or a timeout occurs).
then it's normal behavior, he'll just have to wait for a timeout or the
client will disconnect
Filip
>
> Rémy
>
> ---------------------------------------------------------------------
> To start a new topic, e-mail: users@tomcat.apache.org
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
>
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Memory Leak with Comet
Posted by Rémy Maucherat <re...@gmail.com>.
On 4/25/07, Filip Hanik - Dev Lists <de...@hanik.com> wrote:
> Do you have a small test case,
> if nothing else, your connections should eventually timeout and that
> should recycle the processors.
> As long as the comet connection is active, the processor is spoken for.
He submitted a test webapp that generated a full response
asynchronously, but I don't see anything which wouldn't cause the
client to just "sit there" after the response is sent. As a result, I
don't see any way to get an IO event (unless the client disconnects -
which happens if there's a Connection: close header -, the client
sends a subsequent request, or a timeout occurs).
Rémy
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Memory Leak with Comet
Posted by Filip Hanik - Dev Lists <de...@hanik.com>.
Do you have a small test case,
if nothing else, your connections should eventually timeout and that
should recycle the processors.
As long as the comet connection is active, the processor is spoken for.
Filip
Reich, Matthias wrote:
> So far, I have the feeling that I am the only one who has seen these memory leaks with asynchronous Comet responses.
>
> I tried with a number of variations, e.g.
> - let the browser wait for 500 millis before sending the next poll after receiving a result
> - keep the events queued in the serverfor at least 500 millis before sending the response
> - modify synchronization and the places where I invoke event.close()
> - tried with connector attribute maxKeepAliveRequests="-1" and maxKeepAliveRequests="2"
> but the effect is always the same.
>
> Thus, I doubt that it is really 'wrong' usage of the interfaces, and others should have seen similar effects.
>
> Has anyone out there run an application with browsers as clients and with asynchronous responses for a longer time without loosing memory?
>
> Or am I the only one so far working on such a scenario?
>
> Matthias
>
>
> -----Original Message-----
> From: Rémy Maucherat [mailto:remy.maucherat@gmail.com]
> Sent: Wednesday, April 25, 2007 3:11 PM
> To: Tomcat Users List
> Subject: Re: Memory Leak with Comet
>
> On 4/25/07, Reich, Matthias <ma...@siemens.com> wrote:
>
>> First I ran Tomcat with a single connector with attribute maxThreads=20,
>> and started it with -Xmx10m.
>> In the dump I saw 103 instances of class
>> org.apache.coyote.http11.Http11AprProcessor and the same number of all
>> the attached objects like Request, Response ...
>>
>
> This basically means what I said: the connections and their associated
> processors are never properly closed and recycled for some reason.
>
> Rémy
>
> ---------------------------------------------------------------------
> To start a new topic, e-mail: users@tomcat.apache.org
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
> ---------------------------------------------------------------------
> To start a new topic, e-mail: users@tomcat.apache.org
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
>
>
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Memory Leak with Comet
Posted by "Reich, Matthias" <ma...@siemens.com>.
So far, I have the feeling that I am the only one who has seen these memory leaks with asynchronous Comet responses.
I tried with a number of variations, e.g.
- let the browser wait for 500 millis before sending the next poll after receiving a result
- keep the events queued in the serverfor at least 500 millis before sending the response
- modify synchronization and the places where I invoke event.close()
- tried with connector attribute maxKeepAliveRequests="-1" and maxKeepAliveRequests="2"
but the effect is always the same.
Thus, I doubt that it is really 'wrong' usage of the interfaces, and others should have seen similar effects.
Has anyone out there run an application with browsers as clients and with asynchronous responses for a longer time without loosing memory?
Or am I the only one so far working on such a scenario?
Matthias
-----Original Message-----
From: Rémy Maucherat [mailto:remy.maucherat@gmail.com]
Sent: Wednesday, April 25, 2007 3:11 PM
To: Tomcat Users List
Subject: Re: Memory Leak with Comet
On 4/25/07, Reich, Matthias <ma...@siemens.com> wrote:
> First I ran Tomcat with a single connector with attribute maxThreads=20,
> and started it with -Xmx10m.
> In the dump I saw 103 instances of class
> org.apache.coyote.http11.Http11AprProcessor and the same number of all
> the attached objects like Request, Response ...
This basically means what I said: the connections and their associated
processors are never properly closed and recycled for some reason.
Rémy
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Memory Leak with Comet
Posted by Rémy Maucherat <re...@gmail.com>.
On 4/25/07, Reich, Matthias <ma...@siemens.com> wrote:
> First I ran Tomcat with a single connector with attribute maxThreads=20,
> and started it with -Xmx10m.
> In the dump I saw 103 instances of class
> org.apache.coyote.http11.Http11AprProcessor and the same number of all
> the attached objects like Request, Response ...
This basically means what I said: the connections and their associated
processors are never properly closed and recycled for some reason.
Rémy
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Memory Leak with Comet
Posted by "Reich, Matthias" <ma...@siemens.com>.
I tried with Tomcat6 compiled from trunk revision 532271 and two
browsers sending the requests.
Unfortunately it is not that easy for me to put the dump on a public web
server.
Instead, I had a look at the dump myself.
First I ran Tomcat with a single connector with attribute maxThreads=20,
and started it with -Xmx10m.
In the dump I saw 103 instances of class
org.apache.coyote.http11.Http11AprProcessor and the same number of all
the attached objects like Request, Response ...
I tried again with -Xmx20m (same configuration otherwise) and saw 249
instances of class org.apache.coyote.http11.Http11AprProcessor.
Thus, let us guess that the number of instances increases if I spend
more memory.
As far as I could see the ConcurrentLinkedQueue<Http11AprProcessor>
recycledProcessors contains a single entry.
But the RequestGroupInfo holds 249 instances of
org.apache.coyote.RequestInfo:
Instance of java.util.ArrayList (12 bytes)
Class:
class java.util.ArrayList
Instance data members:
elementData (L) : Instance of [Ljava.lang.Object;@0x27e379b0 (1204
bytes)
modCount (I) : 249
size (I) : 249
Object allocated from:
References to this object:
org.apache.coyote.RequestGroupInfo@0x26e596f8 (44 bytes) : field
processors
-----Original Message-----
From: Filip Hanik - Dev Lists [mailto:devlists@hanik.com]
Sent: Tuesday, April 24, 2007 11:47 PM
To: Tomcat Users List
Subject: Re: Memory Leak with Comet
Why don't you enable -XX:+HeapDumpOnOutOfMemoryError and send us a link
where we can download the HPROF files that get generated.
Also send us your config as well when you get the error
Filip
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Memory Leak with Comet
Posted by Filip Hanik - Dev Lists <de...@hanik.com>.
Reich, Matthias wrote:
> I did a fresh build this morning from the SVN trunk, and I see the memory leak with both NIO and APR connector.
>
Why don't you enable -XX:+HeapDumpOnOutOfMemoryError and send us a link
where we can download the HPROF files that get generated.
Also send us your config as well when you get the error
Filip
> For each browser I run, I have at most 3 active connections at a time.
> I think, acually there are at most 2 active connections per browser, because by default both IE and FireFox don't open more than 2 connections to a single server.
>
> Thus, if I use 4 browser instances for my test, I have at most 12 (or 8) active connections.
>
> Matthias
>
> -----Original Message-----
> From: Rémy Maucherat [mailto:remy.maucherat@gmail.com]
> Sent: Tuesday, April 24, 2007 4:58 PM
> To: Tomcat Users List
> Subject: Re: Memory Leak with Comet
>
> On 4/24/07, Reich, Matthias <ma...@siemens.com> wrote:
>
>> If memory allocation of Tomcat continues until I see messages like
>>
>> org.apache.coyote.http11.Http11NioProcessor: Error processing request
>> java.lang.OutOfMemoryError: Java heap space
>>
>
> There are leaks that have been fixed in the NIO connector, which exist
> in 6.0.10. Regardless of the amount of waiting events in your queue,
> each active Comet connection will use resources.
>
> There is also at least one relatively recent thread talking about Comet issues.
>
> Rémy
>
> ---------------------------------------------------------------------
> To start a new topic, e-mail: users@tomcat.apache.org
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
> ---------------------------------------------------------------------
> To start a new topic, e-mail: users@tomcat.apache.org
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
>
>
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Memory Leak with Comet
Posted by Rémy Maucherat <re...@gmail.com>.
On 4/24/07, Reich, Matthias <ma...@siemens.com> wrote:
> I did a fresh build this morning from the SVN trunk, and I see the memory leak with both NIO and APR connector.
>
> For each browser I run, I have at most 3 active connections at a time.
> I think, acually there are at most 2 active connections per browser, because by default both IE and FireFox don't open more than 2 connections to a single server.
>
> Thus, if I use 4 browser instances for my test, I have at most 12 (or 8) active connections.
I have trouble believing that. I think the resources associated with
the connections are never released by Tomcat, causing the problem.
However, since it works for me, you'll have to look a bit further to
find out why,
Rémy
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Memory Leak with Comet
Posted by "Reich, Matthias" <ma...@siemens.com>.
I did a fresh build this morning from the SVN trunk, and I see the memory leak with both NIO and APR connector.
For each browser I run, I have at most 3 active connections at a time.
I think, acually there are at most 2 active connections per browser, because by default both IE and FireFox don't open more than 2 connections to a single server.
Thus, if I use 4 browser instances for my test, I have at most 12 (or 8) active connections.
Matthias
-----Original Message-----
From: Rémy Maucherat [mailto:remy.maucherat@gmail.com]
Sent: Tuesday, April 24, 2007 4:58 PM
To: Tomcat Users List
Subject: Re: Memory Leak with Comet
On 4/24/07, Reich, Matthias <ma...@siemens.com> wrote:
> If memory allocation of Tomcat continues until I see messages like
>
> org.apache.coyote.http11.Http11NioProcessor: Error processing request
> java.lang.OutOfMemoryError: Java heap space
There are leaks that have been fixed in the NIO connector, which exist
in 6.0.10. Regardless of the amount of waiting events in your queue,
each active Comet connection will use resources.
There is also at least one relatively recent thread talking about Comet issues.
Rémy
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Memory Leak with Comet
Posted by Rémy Maucherat <re...@gmail.com>.
On 4/24/07, Reich, Matthias <ma...@siemens.com> wrote:
> If memory allocation of Tomcat continues until I see messages like
>
> org.apache.coyote.http11.Http11NioProcessor: Error processing request
> java.lang.OutOfMemoryError: Java heap space
There are leaks that have been fixed in the NIO connector, which exist
in 6.0.10. Regardless of the amount of waiting events in your queue,
each active Comet connection will use resources.
There is also at least one relatively recent thread talking about Comet issues.
Rémy
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org