You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mina.apache.org by Richard Lowe <mi...@webcogs.com> on 2007/05/24 21:43:40 UTC
Packets drying up...
Hi,
I'm using Mina 1.1 under Java 5 for both my server and client. I'm
sending roughly 4 packets a second from the client to the server and
from the server back to the client, each one tends to be around 48 bytes
or so.
My TCP send and receive buffers are set to 128 bytes currently.
It works great for a while (maybe five minutes), but then the packets
seem to dry up suddenly.
The stream of packets from the server to the client start dribbling
through and I only get one packet every 5 seconds or so.
At first I thought that they might be getting delayed on the Internet
somewhere, but it appears that isn't the case.
1. I ran TCPDump on the client and it really is only receiving a packet
every 5 seconds (almost exactly 5 seconds - regular as clockwork).
2. I then ran TCPDump on the server and it's only *sending* one packet
every 5 seconds (again, like clockwork).
My application log reveals that everything has been passed to the
IoHandlerAdapter write method - there's nothing left for my application
to send, so the data is getting queued somewhere deeper.
I'm scratching my head here. The server is Windows 2000 and the client
is Mac OS X.
If anybody has any suggestions, they would be most greatly appreciated!
Many thanks,
Richard.
--
Re: Packets drying up...
Posted by Trustin Lee <tr...@gmail.com>.
Hi Richard,
On 5/25/07, Richard Lowe <mi...@webcogs.com> wrote:
> Hi,
>
> I'm using Mina 1.1 under Java 5 for both my server and client. I'm
> sending roughly 4 packets a second from the client to the server and
> from the server back to the client, each one tends to be around 48 bytes
> or so.
Is this problem also reproduceable with MINA 1.0.x? Otherwise, it
would be really nice to have some test code.
Trustin
--
what we call human nature is actually human habit
--
http://gleamynode.net/
--
PGP Key ID: 0x0255ECA6
Re: Packets drying up...
Posted by mat <fo...@gmail.com>.
Did you solve this problem? I faced the same problem.
On 5/25/07, Gaston Dombiak <ga...@jivesoftware.com> wrote:
>
> Hey Richard,
>
> We observed the exact problem. It seems like SocketIoProcessor is
> failing to flush sessions and traffic is kept in
> SocketSessionImpl#writeRequestQueue. Moreover, if you leave the server
> running for a long time this will end up in an OOM. I'm still trying to
> get to the bottom of this problem but I'm trapped in some other tasks as
> well so I'm not able to debug this problem further.
>
> Any help is greatly appreciated. :)
>
> Thanks,
>
> -- Gato
>
>
> -----Original Message-----
> From: Richard Lowe [mailto:mina@webcogs.com]
> Sent: Thursday, May 24, 2007 12:44 PM
> To: dev@mina.apache.org
> Subject: Packets drying up...
>
> Hi,
>
> I'm using Mina 1.1 under Java 5 for both my server and client. I'm
> sending roughly 4 packets a second from the client to the server and
> from the server back to the client, each one tends to be around 48 bytes
>
> or so.
>
> My TCP send and receive buffers are set to 128 bytes currently.
>
> It works great for a while (maybe five minutes), but then the packets
> seem to dry up suddenly.
>
> The stream of packets from the server to the client start dribbling
> through and I only get one packet every 5 seconds or so.
>
> At first I thought that they might be getting delayed on the Internet
> somewhere, but it appears that isn't the case.
>
> 1. I ran TCPDump on the client and it really is only receiving a packet
> every 5 seconds (almost exactly 5 seconds - regular as clockwork).
> 2. I then ran TCPDump on the server and it's only *sending* one packet
> every 5 seconds (again, like clockwork).
>
> My application log reveals that everything has been passed to the
> IoHandlerAdapter write method - there's nothing left for my application
> to send, so the data is getting queued somewhere deeper.
>
> I'm scratching my head here. The server is Windows 2000 and the client
> is Mac OS X.
>
> If anybody has any suggestions, they would be most greatly appreciated!
>
> Many thanks,
> Richard.
> --
>
>
>
>
>
>
>
>
>
Re: Packets drying up...
Posted by Richard Lowe <mi...@webcogs.com>.
PS. I'm not at all familiar with the Mina code-base and so if anybody
has any suggestions on where I should be looking, that'd be really
helpful. Thank you all again!
Richard Lowe wrote:
> Hey Gato,
>
> Thank you for the pointers - it's nice to know that I'm not alone with
> this problem! :-)
>
> I put a line of code in my IoHandlerAdapter.messageSent method and
> Mina is definately holding onto the data and slowing releasing it
> roughly every 5 seconds.
>
> It's a mystery to me why Mina would suddenly choose to queue up the
> packets like that, but I'll have a sniff around SocketIoProcessor to
> see if I can understand what's going on.
>
> Puzzling!!
>
> Many thanks,
> Richard.
> --
>
> Gaston Dombiak wrote:
>> Hey Richard,
>>
>> We observed the exact problem. It seems like SocketIoProcessor is
>> failing to flush sessions and traffic is kept in
>> SocketSessionImpl#writeRequestQueue. Moreover, if you leave the server
>> running for a long time this will end up in an OOM. I'm still trying to
>> get to the bottom of this problem but I'm trapped in some other tasks as
>> well so I'm not able to debug this problem further.
>>
>> Any help is greatly appreciated. :)
>>
>> Thanks,
>>
>> -- Gato
>>
>>
>> -----Original Message-----
>> From: Richard Lowe [mailto:mina@webcogs.com] Sent: Thursday, May 24,
>> 2007 12:44 PM
>> To: dev@mina.apache.org
>> Subject: Packets drying up...
>>
>> Hi,
>>
>> I'm using Mina 1.1 under Java 5 for both my server and client. I'm
>> sending roughly 4 packets a second from the client to the server and
>> from the server back to the client, each one tends to be around 48 bytes
>>
>> or so.
>>
>> My TCP send and receive buffers are set to 128 bytes currently.
>>
>> It works great for a while (maybe five minutes), but then the packets
>> seem to dry up suddenly.
>>
>> The stream of packets from the server to the client start dribbling
>> through and I only get one packet every 5 seconds or so.
>>
>> At first I thought that they might be getting delayed on the Internet
>> somewhere, but it appears that isn't the case.
>>
>> 1. I ran TCPDump on the client and it really is only receiving a
>> packet every 5 seconds (almost exactly 5 seconds - regular as
>> clockwork).
>> 2. I then ran TCPDump on the server and it's only *sending* one
>> packet every 5 seconds (again, like clockwork).
>>
>> My application log reveals that everything has been passed to the
>> IoHandlerAdapter write method - there's nothing left for my
>> application to send, so the data is getting queued somewhere deeper.
>>
>> I'm scratching my head here. The server is Windows 2000 and the
>> client is Mac OS X.
>>
>> If anybody has any suggestions, they would be most greatly appreciated!
>>
>> Many thanks,
>> Richard.
>> --
>>
>>
>>
>>
>>
>>
>>
>>
>>
>
Re: Packets drying up...
Posted by Richard Lowe <mi...@webcogs.com>.
Hey Gato,
Thank you for the pointers - it's nice to know that I'm not alone with
this problem! :-)
I put a line of code in my IoHandlerAdapter.messageSent method and Mina
is definately holding onto the data and slowing releasing it roughly
every 5 seconds.
It's a mystery to me why Mina would suddenly choose to queue up the
packets like that, but I'll have a sniff around SocketIoProcessor to see
if I can understand what's going on.
Puzzling!!
Many thanks,
Richard.
--
Gaston Dombiak wrote:
> Hey Richard,
>
> We observed the exact problem. It seems like SocketIoProcessor is
> failing to flush sessions and traffic is kept in
> SocketSessionImpl#writeRequestQueue. Moreover, if you leave the server
> running for a long time this will end up in an OOM. I'm still trying to
> get to the bottom of this problem but I'm trapped in some other tasks as
> well so I'm not able to debug this problem further.
>
> Any help is greatly appreciated. :)
>
> Thanks,
>
> -- Gato
>
>
> -----Original Message-----
> From: Richard Lowe [mailto:mina@webcogs.com]
> Sent: Thursday, May 24, 2007 12:44 PM
> To: dev@mina.apache.org
> Subject: Packets drying up...
>
> Hi,
>
> I'm using Mina 1.1 under Java 5 for both my server and client. I'm
> sending roughly 4 packets a second from the client to the server and
> from the server back to the client, each one tends to be around 48 bytes
>
> or so.
>
> My TCP send and receive buffers are set to 128 bytes currently.
>
> It works great for a while (maybe five minutes), but then the packets
> seem to dry up suddenly.
>
> The stream of packets from the server to the client start dribbling
> through and I only get one packet every 5 seconds or so.
>
> At first I thought that they might be getting delayed on the Internet
> somewhere, but it appears that isn't the case.
>
> 1. I ran TCPDump on the client and it really is only receiving a packet
> every 5 seconds (almost exactly 5 seconds - regular as clockwork).
> 2. I then ran TCPDump on the server and it's only *sending* one packet
> every 5 seconds (again, like clockwork).
>
> My application log reveals that everything has been passed to the
> IoHandlerAdapter write method - there's nothing left for my application
> to send, so the data is getting queued somewhere deeper.
>
> I'm scratching my head here. The server is Windows 2000 and the client
> is Mac OS X.
>
> If anybody has any suggestions, they would be most greatly appreciated!
>
> Many thanks,
> Richard.
> --
>
>
>
>
>
>
>
>
>
RE: Packets drying up...
Posted by Gaston Dombiak <ga...@jivesoftware.com>.
Hey Richard,
We observed the exact problem. It seems like SocketIoProcessor is
failing to flush sessions and traffic is kept in
SocketSessionImpl#writeRequestQueue. Moreover, if you leave the server
running for a long time this will end up in an OOM. I'm still trying to
get to the bottom of this problem but I'm trapped in some other tasks as
well so I'm not able to debug this problem further.
Any help is greatly appreciated. :)
Thanks,
-- Gato
-----Original Message-----
From: Richard Lowe [mailto:mina@webcogs.com]
Sent: Thursday, May 24, 2007 12:44 PM
To: dev@mina.apache.org
Subject: Packets drying up...
Hi,
I'm using Mina 1.1 under Java 5 for both my server and client. I'm
sending roughly 4 packets a second from the client to the server and
from the server back to the client, each one tends to be around 48 bytes
or so.
My TCP send and receive buffers are set to 128 bytes currently.
It works great for a while (maybe five minutes), but then the packets
seem to dry up suddenly.
The stream of packets from the server to the client start dribbling
through and I only get one packet every 5 seconds or so.
At first I thought that they might be getting delayed on the Internet
somewhere, but it appears that isn't the case.
1. I ran TCPDump on the client and it really is only receiving a packet
every 5 seconds (almost exactly 5 seconds - regular as clockwork).
2. I then ran TCPDump on the server and it's only *sending* one packet
every 5 seconds (again, like clockwork).
My application log reveals that everything has been passed to the
IoHandlerAdapter write method - there's nothing left for my application
to send, so the data is getting queued somewhere deeper.
I'm scratching my head here. The server is Windows 2000 and the client
is Mac OS X.
If anybody has any suggestions, they would be most greatly appreciated!
Many thanks,
Richard.
--