OutOfMemory when the output queue piles up

[Paul Chen <ch...@gmail.com>]

Hi, folks,

I got this error often when my Mina-based proxy server is under get lots of
requests
from clients. And the service my Mina proxy forwards the client to becomes
very slow.
So I added probes in Mina code and saw this queue.size() grew significantly
to > 200K.
BTW I use SocketSessionImpl.writeRequestQueue.size() to get the queue size.

Add more computing power solves this problem but something it is not within
my
control.

In general, this kind of problem should happen when the other side Mina is
talking
to is either too slow or too fast, either the input queue or output queue
piles up,
right?

On the reading side, I put some read throttle filter and it works just fine.

On the writing side, I'm trying to do something similar. Not sure if you
experienced
such issues ever and how you dealt w/ them.

Thanks for comments

Paul

Re: OutOfMemory when the output queue piles up

[Paul Chen <ch...@gmail.com>]

Trustin,

Yes that does work for me to some extent. But on my multi-CPU box there are
several output queues and it is the sum of them causes OutOfMemory. It looks

like I need to put a "quota" on each queue in order to cap the total.

Or, an alternative is to take a snapshot across all in/out queues to get the

sum-up.

Thanks for comments

On 4/17/07, Trustin Lee <tr...@gmail.com> wrote:
>
> Hi Paul,
>
> On 4/18/07, Paul Chen <ch...@gmail.com> wrote:
> > Hi, folks,
> >
> > I got this error often when my Mina-based proxy server is under get lots
> of
> > requests
> > from clients. And the service my Mina proxy forwards the client to
> becomes
> > very slow.
> > So I added probes in Mina code and saw this queue.size() grew
> significantly
> > to > 200K.
> > BTW I use SocketSessionImpl.writeRequestQueue.size() to get the queue
> size.
> >
> > Add more computing power solves this problem but something it is not
> within
> > my
> > control.
> >
> > In general, this kind of problem should happen when the other side Mina
> is
> > talking
> > to is either too slow or too fast, either the input queue or output
> queue
> > piles up,
> > right?
> >
> > On the reading side, I put some read throttle filter and it works just
> fine.
> >
> > On the writing side, I'm trying to do something similar. Not sure if you
> > experienced
> > such issues ever and how you dealt w/ them.
>
> You can check the number of bytes in the write queue using
> IoSession.getScheduledWriteBytes() before you write a message.  If the
> number of bytes exceeds certain threshold, you can write the message
> later when messageSent() event is invoked.
>
> Another alternative is to use the following user contribution.
>
> http://issues.apache.org/jira/browse/DIRMINA-302
>
> I didn't review the code yet though.
>
> HTH,
> Trustin
> --
> what we call human nature is actually human habit
> --
> http://gleamynode.net/
> --
> PGP Key ID: 0x0255ECA6
>

Re: OutOfMemory when the output queue piles up

[Trustin Lee <tr...@gmail.com>]

Hi Paul,

On 4/18/07, Paul Chen <ch...@gmail.com> wrote:
> Hi, folks,
>
> I got this error often when my Mina-based proxy server is under get lots of
> requests
> from clients. And the service my Mina proxy forwards the client to becomes
> very slow.
> So I added probes in Mina code and saw this queue.size() grew significantly
> to > 200K.
> BTW I use SocketSessionImpl.writeRequestQueue.size() to get the queue size.
>
> Add more computing power solves this problem but something it is not within
> my
> control.
>
> In general, this kind of problem should happen when the other side Mina is
> talking
> to is either too slow or too fast, either the input queue or output queue
> piles up,
> right?
>
> On the reading side, I put some read throttle filter and it works just fine.
>
> On the writing side, I'm trying to do something similar. Not sure if you
> experienced
> such issues ever and how you dealt w/ them.

You can check the number of bytes in the write queue using
IoSession.getScheduledWriteBytes() before you write a message.  If the
number of bytes exceeds certain threshold, you can write the message
later when messageSent() event is invoked.

Another alternative is to use the following user contribution.

http://issues.apache.org/jira/browse/DIRMINA-302

I didn't review the code yet though.

HTH,
Trustin
-- 
what we call human nature is actually human habit
--
http://gleamynode.net/
--
PGP Key ID: 0x0255ECA6