You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Martin Perez <mp...@gmail.com> on 2007/04/30 16:25:03 UTC

Exception with Comet in Tomcat with high load

Hi all,

I have been testing Tomcat comet support during the last days. I've created
a test application that simulates data streaming to multiple clients.
Clients open a connection to the server and the connection is hold open. In
the server there are several "sender" threads that leverage comet support to
periodically send stream updates to the subscribed clients. Once a client
receives a specified amount of data the communication is closed.

So, I'm running the test with tomcat svn version from 1 week ago. I have
configured 150 tomcat threads, plus 25 acceptor threads. Tomcat runs in a
Sun-Fire-V240 box with 2 CPU cores. I have 75 sender threads that
periodically sleep (100ms) to try to reduce the CPU usage. Once a sender
thread wakes up, it will send all the data to the assigned clients.

So basically the story is that the box is able to manage about 3500
persistent comet connections smoothly. But when I raise the level to 4000
(80% CPU usage, 1.2G mem.) I start to get these exceptions for some (not
all) connections:

Exception in thread "MessageSender[]" java.lang.NullPointerException
        at org.apache.coyote.http11.InternalNioOutputBuffer.addToBB(
InternalNioOutputBuffer.java:607)
        at org.apache.coyote.http11.InternalNioOutputBuffer.commit(
InternalNioOutputBuffer.java:600)
        at org.apache.coyote.http11.Http11NioProcessor.action(
Http11NioProcessor.java:1032)
        at org.apache.coyote.Response.action(Response.java:181)
        at org.apache.coyote.http11.InternalNioOutputBuffer.doWrite(
InternalNioOutputBuffer.java:571)
        at org.apache.coyote.Response.doWrite(Response.java:560)
        at org.apache.catalina.connector.OutputBuffer.realWriteBytes(
OutputBuffer.java:353)
        at org.apache.tomcat.util.buf.ByteChunk.flushBuffer(ByteChunk.java
:434)
        at org.apache.tomcat.util.buf.ByteChunk.append(ByteChunk.java:349)
        at org.apache.catalina.connector.OutputBuffer.writeBytes(
OutputBuffer.java:381)
        at org.apache.catalina.connector.OutputBuffer.write(
OutputBuffer.java:370)
        at org.apache.catalina.connector.CoyoteOutputStream.write(
CoyoteOutputStream.java:89)
        at org.apache.catalina.connector.CoyoteOutputStream.write(
CoyoteOutputStream.java:83)
        at com.cognotec.streaming.CometRateSender.run(CometRateSender.java
:86)
        at java.lang.Thread.run(Thread.java:595)

So well, does anybody knows what can be the problem? Any suggestions? It can
be also my problem, so I will be happy to provide more information.

Cheers,
Martin

Re: Exception with Comet in Tomcat with high load

Posted by Martin Perez <mp...@gmail.com>.
Sebastiaan,

Yes you're right. I forgot to add something like isClosed() to your
SynchronizedOutputStream so I can check if it's closed before writing.

Anyways, I found that after a certain limit (for ex. 6000 -> 8000 users) the
system starts to behave incorrectly. Some clients never get an answer from
the server, some clients only receive partial data, and some of them really
end. I guess I found the limit in the box, but attached are the source code
files if someone wants to see the sample or simply spot something in it.

Cheers,
Martin

On 5/1/07, Sebastiaan van Erk <se...@sebster.com> wrote:
>
> Hi,
>
> Did you synchronize the event.close() with same lock you use to write to
> the output stream (the SynchronizedOuputStream instance)?
>
> I saw exactly the same stack trace you are getting due to (incorrect)
> concurrent access to the CoyoteOutputStream. The NullPointerException
> suggest that the request has already been cleaned up/recycled and yet
> you still write to it. Make sure that when you close the event, either
> the write becomes a null operation, or better, does not get called. It
> is a bug to call write on the response output stream after the event has
> been closed.
>
> With the SynchronizedOutputStream construction I am relying on Tomcat
> not to close or call any methods on the CoyoteOutputStream outside of
> the CometProcessor's event() method. I don't know if this is correct (it
> has worked for me in practice so far, and I've been doing some heavy
> duty testing); however I do not know what else I could synchronize on.
> If anybody knows a better approach, I would like to know. :-)
>
> Regards,
> Sebastiaan
>
> Martin Perez wrote:
> > Hi,
> >
> > Thanks for the tip Sebastian, but it doesn't work for me. I added your
> > class, wrapped all the output streams and synchronized close events
> > but the
> > result is the same.
> >
> > Exception in thread "MessageSender[]" java.lang.NullPointerException
> >        at org.apache.coyote.http11.InternalNioOutputBuffer.addToBB(
> > InternalNioOutputBuffer.java:607)
> >        at org.apache.coyote.http11.InternalNioOutputBuffer.commit(
> > InternalNioOutputBuffer.java:600)
> >        at org.apache.coyote.http11.Http11NioProcessor.action(
> > Http11NioProcessor.java:1032)
> >        at org.apache.coyote.Response.action(Response.java:181)
> >        at org.apache.coyote.http11.InternalNioOutputBuffer.doWrite(
> > InternalNioOutputBuffer.java:571)
> >        at org.apache.coyote.Response.doWrite(Response.java:560)
> >        at org.apache.catalina.connector.OutputBuffer.realWriteBytes(
> > OutputBuffer.java:353)
> >        at org.apache.tomcat.util.buf.ByteChunk.flushBuffer(
> ByteChunk.java
> > :434)
> >        at org.apache.tomcat.util.buf.ByteChunk.append(ByteChunk.java
> :349)
> >        at org.apache.catalina.connector.OutputBuffer.writeBytes(
> > OutputBuffer.java:381)
> >        at org.apache.catalina.connector.OutputBuffer.write(
> > OutputBuffer.java:370)
> >        at org.apache.catalina.connector.CoyoteOutputStream.write(
> > CoyoteOutputStream.java:89)
> >        at org.apache.catalina.connector.CoyoteOutputStream.write(
> > CoyoteOutputStream.java:83)
> >        at com.cognotec.streaming.SynchronizedOutputStream.write(
> > SynchronizedOutputStream.java:25)
> >        at com.cognotec.streaming.CometRateSender.run(
> CometRateSender.java
> > :86)
> >        at java.lang.Thread.run(Thread.java:595)
> >
> > (As I said, I also synchronized all the close method calls either in
> > stream
> > either in comet event)
> >
> > So, the question is: Is this a bug or simply I have reached the
> > scalability
> > limit for my box?
> >
> > Cheers,
> > Martin
> >
> > On 4/30/07, Sebastiaan van Erk <se...@sebster.com> wrote:
> >>
> >> Hi,
> >>
> >> I had the same problems. It was a synchronization issue with me. What I
> >> did was make a synchronized output stream wrapper (see attached file)
> >> which wraps all access to the output stream, and I synchronized on this
> >> output stream whenever I called event.close() on the Comet event
> >> (especially in the READ event). When doing serveral IO operations on
> the
> >> output stream which should not be interrupted, you can simply
> >> synchronize on it (since SynchronizedOutputStream locks on itself).
> >>
> >> Regards,
> >> Sebastiaan
> >>
> >>
> >> Martin Perez wrote:
> >> > Hi all,
> >> >
> >> > I have been testing Tomcat comet support during the last days. I've
> >> > created
> >> > a test application that simulates data streaming to multiple clients.
> >> > Clients open a connection to the server and the connection is hold
> >> > open. In
> >> > the server there are several "sender" threads that leverage comet
> >> > support to
> >> > periodically send stream updates to the subscribed clients. Once a
> >> client
> >> > receives a specified amount of data the communication is closed.
> >> >
> >> > So, I'm running the test with tomcat svn version from 1 week ago. I
> >> have
> >> > configured 150 tomcat threads, plus 25 acceptor threads. Tomcat
> >> runs in
> >> a
> >> > Sun-Fire-V240 box with 2 CPU cores. I have 75 sender threads that
> >> > periodically sleep (100ms) to try to reduce the CPU usage. Once a
> >> sender
> >> > thread wakes up, it will send all the data to the assigned clients.
> >> >
> >> > So basically the story is that the box is able to manage about 3500
> >> > persistent comet connections smoothly. But when I raise the level to
> >> 4000
> >> > (80% CPU usage, 1.2G mem.) I start to get these exceptions for some
> >> (not
> >> > all) connections:
> >> >
> >> > Exception in thread "MessageSender[]" java.lang.NullPointerException
> >> >        at org.apache.coyote.http11.InternalNioOutputBuffer.addToBB(
> >> > InternalNioOutputBuffer.java:607)
> >> >        at org.apache.coyote.http11.InternalNioOutputBuffer.commit(
> >> > InternalNioOutputBuffer.java:600)
> >> >        at org.apache.coyote.http11.Http11NioProcessor.action(
> >> > Http11NioProcessor.java:1032)
> >> >        at org.apache.coyote.Response.action(Response.java:181)
> >> >        at org.apache.coyote.http11.InternalNioOutputBuffer.doWrite(
> >> > InternalNioOutputBuffer.java:571)
> >> >        at org.apache.coyote.Response.doWrite(Response.java:560)
> >> >        at org.apache.catalina.connector.OutputBuffer.realWriteBytes(
> >> > OutputBuffer.java:353)
> >> >        at org.apache.tomcat.util.buf.ByteChunk.flushBuffer(
> >> ByteChunk.java
> >> > :434)
> >> >        at org.apache.tomcat.util.buf.ByteChunk.append(ByteChunk.java
> >> :349)
> >> >        at org.apache.catalina.connector.OutputBuffer.writeBytes(
> >> > OutputBuffer.java:381)
> >> >        at org.apache.catalina.connector.OutputBuffer.write(
> >> > OutputBuffer.java:370)
> >> >        at org.apache.catalina.connector.CoyoteOutputStream.write(
> >> > CoyoteOutputStream.java:89)
> >> >        at org.apache.catalina.connector.CoyoteOutputStream.write(
> >> > CoyoteOutputStream.java:83)
> >> >        at com.cognotec.streaming.CometRateSender.run(
> >> CometRateSender.java
> >> > :86)
> >> >        at java.lang.Thread.run(Thread.java:595)
> >> >
> >> > So well, does anybody knows what can be the problem? Any suggestions?
> >> > It can
> >> > be also my problem, so I will be happy to provide more information.
> >> >
> >> > Cheers,
> >> > Martin
> >> >
> >>
> >> ---------------------------------------------------------------------
> >> To start a new topic, e-mail: users@tomcat.apache.org
> >> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> >> For additional commands, e-mail: users-help@tomcat.apache.org
> >>
> >>
> >
>
> ---------------------------------------------------------------------
> To start a new topic, e-mail: users@tomcat.apache.org
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>

Re: Exception with Comet in Tomcat with high load

Posted by Sebastiaan van Erk <se...@sebster.com>.
Hi,

Did you synchronize the event.close() with same lock you use to write to 
the output stream (the SynchronizedOuputStream instance)?

I saw exactly the same stack trace you are getting due to (incorrect) 
concurrent access to the CoyoteOutputStream. The NullPointerException 
suggest that the request has already been cleaned up/recycled and yet 
you still write to it. Make sure that when you close the event, either 
the write becomes a null operation, or better, does not get called. It 
is a bug to call write on the response output stream after the event has 
been closed.

With the SynchronizedOutputStream construction I am relying on Tomcat 
not to close or call any methods on the CoyoteOutputStream outside of 
the CometProcessor's event() method. I don't know if this is correct (it 
has worked for me in practice so far, and I've been doing some heavy 
duty testing); however I do not know what else I could synchronize on. 
If anybody knows a better approach, I would like to know. :-)

Regards,
Sebastiaan

Martin Perez wrote:
> Hi,
>
> Thanks for the tip Sebastian, but it doesn't work for me. I added your
> class, wrapped all the output streams and synchronized close events 
> but the
> result is the same.
>
> Exception in thread "MessageSender[]" java.lang.NullPointerException
>        at org.apache.coyote.http11.InternalNioOutputBuffer.addToBB(
> InternalNioOutputBuffer.java:607)
>        at org.apache.coyote.http11.InternalNioOutputBuffer.commit(
> InternalNioOutputBuffer.java:600)
>        at org.apache.coyote.http11.Http11NioProcessor.action(
> Http11NioProcessor.java:1032)
>        at org.apache.coyote.Response.action(Response.java:181)
>        at org.apache.coyote.http11.InternalNioOutputBuffer.doWrite(
> InternalNioOutputBuffer.java:571)
>        at org.apache.coyote.Response.doWrite(Response.java:560)
>        at org.apache.catalina.connector.OutputBuffer.realWriteBytes(
> OutputBuffer.java:353)
>        at org.apache.tomcat.util.buf.ByteChunk.flushBuffer(ByteChunk.java
> :434)
>        at org.apache.tomcat.util.buf.ByteChunk.append(ByteChunk.java:349)
>        at org.apache.catalina.connector.OutputBuffer.writeBytes(
> OutputBuffer.java:381)
>        at org.apache.catalina.connector.OutputBuffer.write(
> OutputBuffer.java:370)
>        at org.apache.catalina.connector.CoyoteOutputStream.write(
> CoyoteOutputStream.java:89)
>        at org.apache.catalina.connector.CoyoteOutputStream.write(
> CoyoteOutputStream.java:83)
>        at com.cognotec.streaming.SynchronizedOutputStream.write(
> SynchronizedOutputStream.java:25)
>        at com.cognotec.streaming.CometRateSender.run(CometRateSender.java
> :86)
>        at java.lang.Thread.run(Thread.java:595)
>
> (As I said, I also synchronized all the close method calls either in 
> stream
> either in comet event)
>
> So, the question is: Is this a bug or simply I have reached the 
> scalability
> limit for my box?
>
> Cheers,
> Martin
>
> On 4/30/07, Sebastiaan van Erk <se...@sebster.com> wrote:
>>
>> Hi,
>>
>> I had the same problems. It was a synchronization issue with me. What I
>> did was make a synchronized output stream wrapper (see attached file)
>> which wraps all access to the output stream, and I synchronized on this
>> output stream whenever I called event.close() on the Comet event
>> (especially in the READ event). When doing serveral IO operations on the
>> output stream which should not be interrupted, you can simply
>> synchronize on it (since SynchronizedOutputStream locks on itself).
>>
>> Regards,
>> Sebastiaan
>>
>>
>> Martin Perez wrote:
>> > Hi all,
>> >
>> > I have been testing Tomcat comet support during the last days. I've
>> > created
>> > a test application that simulates data streaming to multiple clients.
>> > Clients open a connection to the server and the connection is hold
>> > open. In
>> > the server there are several "sender" threads that leverage comet
>> > support to
>> > periodically send stream updates to the subscribed clients. Once a
>> client
>> > receives a specified amount of data the communication is closed.
>> >
>> > So, I'm running the test with tomcat svn version from 1 week ago. I 
>> have
>> > configured 150 tomcat threads, plus 25 acceptor threads. Tomcat 
>> runs in
>> a
>> > Sun-Fire-V240 box with 2 CPU cores. I have 75 sender threads that
>> > periodically sleep (100ms) to try to reduce the CPU usage. Once a 
>> sender
>> > thread wakes up, it will send all the data to the assigned clients.
>> >
>> > So basically the story is that the box is able to manage about 3500
>> > persistent comet connections smoothly. But when I raise the level to
>> 4000
>> > (80% CPU usage, 1.2G mem.) I start to get these exceptions for some 
>> (not
>> > all) connections:
>> >
>> > Exception in thread "MessageSender[]" java.lang.NullPointerException
>> >        at org.apache.coyote.http11.InternalNioOutputBuffer.addToBB(
>> > InternalNioOutputBuffer.java:607)
>> >        at org.apache.coyote.http11.InternalNioOutputBuffer.commit(
>> > InternalNioOutputBuffer.java:600)
>> >        at org.apache.coyote.http11.Http11NioProcessor.action(
>> > Http11NioProcessor.java:1032)
>> >        at org.apache.coyote.Response.action(Response.java:181)
>> >        at org.apache.coyote.http11.InternalNioOutputBuffer.doWrite(
>> > InternalNioOutputBuffer.java:571)
>> >        at org.apache.coyote.Response.doWrite(Response.java:560)
>> >        at org.apache.catalina.connector.OutputBuffer.realWriteBytes(
>> > OutputBuffer.java:353)
>> >        at org.apache.tomcat.util.buf.ByteChunk.flushBuffer(
>> ByteChunk.java
>> > :434)
>> >        at org.apache.tomcat.util.buf.ByteChunk.append(ByteChunk.java
>> :349)
>> >        at org.apache.catalina.connector.OutputBuffer.writeBytes(
>> > OutputBuffer.java:381)
>> >        at org.apache.catalina.connector.OutputBuffer.write(
>> > OutputBuffer.java:370)
>> >        at org.apache.catalina.connector.CoyoteOutputStream.write(
>> > CoyoteOutputStream.java:89)
>> >        at org.apache.catalina.connector.CoyoteOutputStream.write(
>> > CoyoteOutputStream.java:83)
>> >        at com.cognotec.streaming.CometRateSender.run(
>> CometRateSender.java
>> > :86)
>> >        at java.lang.Thread.run(Thread.java:595)
>> >
>> > So well, does anybody knows what can be the problem? Any suggestions?
>> > It can
>> > be also my problem, so I will be happy to provide more information.
>> >
>> > Cheers,
>> > Martin
>> >
>>
>> ---------------------------------------------------------------------
>> To start a new topic, e-mail: users@tomcat.apache.org
>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>> For additional commands, e-mail: users-help@tomcat.apache.org
>>
>>
>

---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Re: Exception with Comet in Tomcat with high load

Posted by Martin Perez <mp...@gmail.com>.
Hi,

Thanks for the tip Sebastian, but it doesn't work for me. I added your
class, wrapped all the output streams and synchronized close events but the
result is the same.

Exception in thread "MessageSender[]" java.lang.NullPointerException
        at org.apache.coyote.http11.InternalNioOutputBuffer.addToBB(
InternalNioOutputBuffer.java:607)
        at org.apache.coyote.http11.InternalNioOutputBuffer.commit(
InternalNioOutputBuffer.java:600)
        at org.apache.coyote.http11.Http11NioProcessor.action(
Http11NioProcessor.java:1032)
        at org.apache.coyote.Response.action(Response.java:181)
        at org.apache.coyote.http11.InternalNioOutputBuffer.doWrite(
InternalNioOutputBuffer.java:571)
        at org.apache.coyote.Response.doWrite(Response.java:560)
        at org.apache.catalina.connector.OutputBuffer.realWriteBytes(
OutputBuffer.java:353)
        at org.apache.tomcat.util.buf.ByteChunk.flushBuffer(ByteChunk.java
:434)
        at org.apache.tomcat.util.buf.ByteChunk.append(ByteChunk.java:349)
        at org.apache.catalina.connector.OutputBuffer.writeBytes(
OutputBuffer.java:381)
        at org.apache.catalina.connector.OutputBuffer.write(
OutputBuffer.java:370)
        at org.apache.catalina.connector.CoyoteOutputStream.write(
CoyoteOutputStream.java:89)
        at org.apache.catalina.connector.CoyoteOutputStream.write(
CoyoteOutputStream.java:83)
        at com.cognotec.streaming.SynchronizedOutputStream.write(
SynchronizedOutputStream.java:25)
        at com.cognotec.streaming.CometRateSender.run(CometRateSender.java
:86)
        at java.lang.Thread.run(Thread.java:595)

(As I said, I also synchronized all the close method calls either in stream
either in comet event)

So, the question is: Is this a bug or simply I have reached the scalability
limit for my box?

Cheers,
Martin

On 4/30/07, Sebastiaan van Erk <se...@sebster.com> wrote:
>
> Hi,
>
> I had the same problems. It was a synchronization issue with me. What I
> did was make a synchronized output stream wrapper (see attached file)
> which wraps all access to the output stream, and I synchronized on this
> output stream whenever I called event.close() on the Comet event
> (especially in the READ event). When doing serveral IO operations on the
> output stream which should not be interrupted, you can simply
> synchronize on it (since SynchronizedOutputStream locks on itself).
>
> Regards,
> Sebastiaan
>
>
> Martin Perez wrote:
> > Hi all,
> >
> > I have been testing Tomcat comet support during the last days. I've
> > created
> > a test application that simulates data streaming to multiple clients.
> > Clients open a connection to the server and the connection is hold
> > open. In
> > the server there are several "sender" threads that leverage comet
> > support to
> > periodically send stream updates to the subscribed clients. Once a
> client
> > receives a specified amount of data the communication is closed.
> >
> > So, I'm running the test with tomcat svn version from 1 week ago. I have
> > configured 150 tomcat threads, plus 25 acceptor threads. Tomcat runs in
> a
> > Sun-Fire-V240 box with 2 CPU cores. I have 75 sender threads that
> > periodically sleep (100ms) to try to reduce the CPU usage. Once a sender
> > thread wakes up, it will send all the data to the assigned clients.
> >
> > So basically the story is that the box is able to manage about 3500
> > persistent comet connections smoothly. But when I raise the level to
> 4000
> > (80% CPU usage, 1.2G mem.) I start to get these exceptions for some (not
> > all) connections:
> >
> > Exception in thread "MessageSender[]" java.lang.NullPointerException
> >        at org.apache.coyote.http11.InternalNioOutputBuffer.addToBB(
> > InternalNioOutputBuffer.java:607)
> >        at org.apache.coyote.http11.InternalNioOutputBuffer.commit(
> > InternalNioOutputBuffer.java:600)
> >        at org.apache.coyote.http11.Http11NioProcessor.action(
> > Http11NioProcessor.java:1032)
> >        at org.apache.coyote.Response.action(Response.java:181)
> >        at org.apache.coyote.http11.InternalNioOutputBuffer.doWrite(
> > InternalNioOutputBuffer.java:571)
> >        at org.apache.coyote.Response.doWrite(Response.java:560)
> >        at org.apache.catalina.connector.OutputBuffer.realWriteBytes(
> > OutputBuffer.java:353)
> >        at org.apache.tomcat.util.buf.ByteChunk.flushBuffer(
> ByteChunk.java
> > :434)
> >        at org.apache.tomcat.util.buf.ByteChunk.append(ByteChunk.java
> :349)
> >        at org.apache.catalina.connector.OutputBuffer.writeBytes(
> > OutputBuffer.java:381)
> >        at org.apache.catalina.connector.OutputBuffer.write(
> > OutputBuffer.java:370)
> >        at org.apache.catalina.connector.CoyoteOutputStream.write(
> > CoyoteOutputStream.java:89)
> >        at org.apache.catalina.connector.CoyoteOutputStream.write(
> > CoyoteOutputStream.java:83)
> >        at com.cognotec.streaming.CometRateSender.run(
> CometRateSender.java
> > :86)
> >        at java.lang.Thread.run(Thread.java:595)
> >
> > So well, does anybody knows what can be the problem? Any suggestions?
> > It can
> > be also my problem, so I will be happy to provide more information.
> >
> > Cheers,
> > Martin
> >
>
> ---------------------------------------------------------------------
> To start a new topic, e-mail: users@tomcat.apache.org
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>

Re: Exception with Comet in Tomcat with high load

Posted by Sebastiaan van Erk <se...@sebster.com>.
Hi,

I had the same problems. It was a synchronization issue with me. What I 
did was make a synchronized output stream wrapper (see attached file) 
which wraps all access to the output stream, and I synchronized on this 
output stream whenever I called event.close() on the Comet event 
(especially in the READ event). When doing serveral IO operations on the 
output stream which should not be interrupted, you can simply 
synchronize on it (since SynchronizedOutputStream locks on itself).

Regards,
Sebastiaan


Martin Perez wrote:
> Hi all,
>
> I have been testing Tomcat comet support during the last days. I've 
> created
> a test application that simulates data streaming to multiple clients.
> Clients open a connection to the server and the connection is hold 
> open. In
> the server there are several "sender" threads that leverage comet 
> support to
> periodically send stream updates to the subscribed clients. Once a client
> receives a specified amount of data the communication is closed.
>
> So, I'm running the test with tomcat svn version from 1 week ago. I have
> configured 150 tomcat threads, plus 25 acceptor threads. Tomcat runs in a
> Sun-Fire-V240 box with 2 CPU cores. I have 75 sender threads that
> periodically sleep (100ms) to try to reduce the CPU usage. Once a sender
> thread wakes up, it will send all the data to the assigned clients.
>
> So basically the story is that the box is able to manage about 3500
> persistent comet connections smoothly. But when I raise the level to 4000
> (80% CPU usage, 1.2G mem.) I start to get these exceptions for some (not
> all) connections:
>
> Exception in thread "MessageSender[]" java.lang.NullPointerException
>        at org.apache.coyote.http11.InternalNioOutputBuffer.addToBB(
> InternalNioOutputBuffer.java:607)
>        at org.apache.coyote.http11.InternalNioOutputBuffer.commit(
> InternalNioOutputBuffer.java:600)
>        at org.apache.coyote.http11.Http11NioProcessor.action(
> Http11NioProcessor.java:1032)
>        at org.apache.coyote.Response.action(Response.java:181)
>        at org.apache.coyote.http11.InternalNioOutputBuffer.doWrite(
> InternalNioOutputBuffer.java:571)
>        at org.apache.coyote.Response.doWrite(Response.java:560)
>        at org.apache.catalina.connector.OutputBuffer.realWriteBytes(
> OutputBuffer.java:353)
>        at org.apache.tomcat.util.buf.ByteChunk.flushBuffer(ByteChunk.java
> :434)
>        at org.apache.tomcat.util.buf.ByteChunk.append(ByteChunk.java:349)
>        at org.apache.catalina.connector.OutputBuffer.writeBytes(
> OutputBuffer.java:381)
>        at org.apache.catalina.connector.OutputBuffer.write(
> OutputBuffer.java:370)
>        at org.apache.catalina.connector.CoyoteOutputStream.write(
> CoyoteOutputStream.java:89)
>        at org.apache.catalina.connector.CoyoteOutputStream.write(
> CoyoteOutputStream.java:83)
>        at com.cognotec.streaming.CometRateSender.run(CometRateSender.java
> :86)
>        at java.lang.Thread.run(Thread.java:595)
>
> So well, does anybody knows what can be the problem? Any suggestions? 
> It can
> be also my problem, so I will be happy to provide more information.
>
> Cheers,
> Martin
>