You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mina.apache.org by Michael Bauroth <Mi...@falcom.de> on 2006/12/21 23:20:25 UTC
Closing sessions problem
Hi,
remembering my previous mail I have a (maybe) follow up:
When I run the first test wiht fewer connections and afterwards perform
a session close for all sessions (see code snippet), then I get problems
while performing test2(). I think, that all connections are closed
properly (no errors or messages are fired). But when I now run a new
connection test (e.g. test2()), after a sumup of created connections
from first and second test hits 28230 connections, the test fails again.
Here is a little example:
create 20000 connections.
close all sessions.
create 10000 connections -> error after 8230 !?
Here is the sample code:
private void connect( int tConnectionCount )
{
for ( int i = 0; i < tConnectionCount; i++ )
{
ConnectFuture tFuture = mConnector.connect( mBindAdress );
tFuture.join();
if ( !tFuture.isConnected() )
System.out.println("not connected");
if ( !tFuture.isReady() )
System.out.println("open not ready");
}
}
private void disconnect()
{
for ( Object tSession : mConnector.getManagedSessions() )
{
CloseFuture tFuture = ( ( IoSession )tSession ).close();
tFuture.join();
if ( !tFuture.isClosed() )
System.out.println("not closed");
if ( !tFuture.isReady() )
System.out.println("close not ready");
}
}
public void test1()
{
int tCount = 28000;
mLogger.info( "##### Test 1 #####" );
mLogger.info( "Number of connections: {}", tCount );
mLogger.info( "Test phase 1 (creating connections) ..." );
connect( tCount );
mLogger.info( "Test phase 2 (closing connections) ..." );
disconnect();
mLogger.info( "Test 1 finished." );
}
public void test2()
{
int tConnectionCount = 10000;
int tCycleCount = 100;
mLogger.info( "##### Test 2 #####" );
mLogger.info( "Number of connections: {}", tConnectionCount );
mLogger.info( "Number of cycles: {}", tCycleCount );
for ( int i = 1; i <= tCycleCount; i++ )
{
mLogger.info( "Test cycle {}", i );
mLogger.info( "Test phase 1 (creating connections) ..." );
connect( tConnectionCount );
mLogger.info( "Test phase 2 (closing connections) ..." );
disconnect();
}
mLogger.info( "Test 2 finished." );
}
I forgot to mention, that I work on the Mina trunk
@version $Rev: 436993 $, $Date: 2006/11/21 23:30:42 $
Best Regards
Michael
Re: Closing sessions problem
Posted by Michael Bauroth <Mi...@falcom.de>.
But in my tests I use the Futures, which should return only, if the
session is really closed (what seems to be happen, because in the other
case I would get a message in the disconnect method) ... ?
Michael
Hanson Char schrieb:
> Not sure if this is related, but from my experience (painful) with Mina
> 0.8.2, closing a session does NOT mean session.isConnected() would return
> false. At least not immediately after.
>
> Hanson
>
> On 12/21/06, Michael Bauroth <Mi...@falcom.de> wrote:
>>
>> Hi,
>>
>> remembering my previous mail I have a (maybe) follow up:
>> When I run the first test wiht fewer connections and afterwards perform
>> a session close for all sessions (see code snippet), then I get problems
>> while performing test2(). I think, that all connections are closed
>> properly (no errors or messages are fired). But when I now run a new
>> connection test (e.g. test2()), after a sumup of created connections
>> from first and second test hits 28230 connections, the test fails again.
>> Here is a little example:
>>
>> create 20000 connections.
>> close all sessions.
>> create 10000 connections -> error after 8230 !?
>>
>>
>> Here is the sample code:
>>
>> private void connect( int tConnectionCount )
>> {
>> for ( int i = 0; i < tConnectionCount; i++ )
>> {
>> ConnectFuture tFuture = mConnector.connect( mBindAdress );
>> tFuture.join();
>>
>> if ( !tFuture.isConnected() )
>> System.out.println("not connected");
>> if ( !tFuture.isReady() )
>> System.out.println("open not ready");
>> }
>> }
>>
>> private void disconnect()
>> {
>> for ( Object tSession : mConnector.getManagedSessions() )
>> {
>> CloseFuture tFuture = ( ( IoSession )tSession ).close();
>> tFuture.join();
>>
>> if ( !tFuture.isClosed() )
>> System.out.println("not closed");
>> if ( !tFuture.isReady() )
>> System.out.println("close not ready");
>> }
>> }
>>
>> public void test1()
>> {
>> int tCount = 28000;
>>
>> mLogger.info( "##### Test 1 #####" );
>> mLogger.info( "Number of connections: {}", tCount );
>> mLogger.info( "Test phase 1 (creating connections) ..." );
>> connect( tCount );
>> mLogger.info( "Test phase 2 (closing connections) ..." );
>> disconnect();
>> mLogger.info( "Test 1 finished." );
>> }
>>
>> public void test2()
>> {
>> int tConnectionCount = 10000;
>> int tCycleCount = 100;
>>
>> mLogger.info( "##### Test 2 #####" );
>> mLogger.info( "Number of connections: {}", tConnectionCount );
>> mLogger.info( "Number of cycles: {}", tCycleCount );
>>
>> for ( int i = 1; i <= tCycleCount; i++ )
>> {
>> mLogger.info( "Test cycle {}", i );
>> mLogger.info( "Test phase 1 (creating connections) ..." );
>> connect( tConnectionCount );
>> mLogger.info( "Test phase 2 (closing connections) ..." );
>> disconnect();
>> }
>>
>> mLogger.info( "Test 2 finished." );
>> }
>>
>>
>> I forgot to mention, that I work on the Mina trunk
>> @version $Rev: 436993 $, $Date: 2006/11/21 23:30:42 $
>>
>> Best Regards
>> Michael
>>
>
Re: Closing sessions problem
Posted by Hanson Char <ha...@gmail.com>.
Not sure if this is related, but from my experience (painful) with Mina
0.8.2, closing a session does NOT mean session.isConnected() would return
false. At least not immediately after.
Hanson
On 12/21/06, Michael Bauroth <Mi...@falcom.de> wrote:
>
> Hi,
>
> remembering my previous mail I have a (maybe) follow up:
> When I run the first test wiht fewer connections and afterwards perform
> a session close for all sessions (see code snippet), then I get problems
> while performing test2(). I think, that all connections are closed
> properly (no errors or messages are fired). But when I now run a new
> connection test (e.g. test2()), after a sumup of created connections
> from first and second test hits 28230 connections, the test fails again.
> Here is a little example:
>
> create 20000 connections.
> close all sessions.
> create 10000 connections -> error after 8230 !?
>
>
> Here is the sample code:
>
> private void connect( int tConnectionCount )
> {
> for ( int i = 0; i < tConnectionCount; i++ )
> {
> ConnectFuture tFuture = mConnector.connect( mBindAdress );
> tFuture.join();
>
> if ( !tFuture.isConnected() )
> System.out.println("not connected");
> if ( !tFuture.isReady() )
> System.out.println("open not ready");
> }
> }
>
> private void disconnect()
> {
> for ( Object tSession : mConnector.getManagedSessions() )
> {
> CloseFuture tFuture = ( ( IoSession )tSession ).close();
> tFuture.join();
>
> if ( !tFuture.isClosed() )
> System.out.println("not closed");
> if ( !tFuture.isReady() )
> System.out.println("close not ready");
> }
> }
>
> public void test1()
> {
> int tCount = 28000;
>
> mLogger.info( "##### Test 1 #####" );
> mLogger.info( "Number of connections: {}", tCount );
> mLogger.info( "Test phase 1 (creating connections) ..." );
> connect( tCount );
> mLogger.info( "Test phase 2 (closing connections) ..." );
> disconnect();
> mLogger.info( "Test 1 finished." );
> }
>
> public void test2()
> {
> int tConnectionCount = 10000;
> int tCycleCount = 100;
>
> mLogger.info( "##### Test 2 #####" );
> mLogger.info( "Number of connections: {}", tConnectionCount );
> mLogger.info( "Number of cycles: {}", tCycleCount );
>
> for ( int i = 1; i <= tCycleCount; i++ )
> {
> mLogger.info( "Test cycle {}", i );
> mLogger.info( "Test phase 1 (creating connections) ..." );
> connect( tConnectionCount );
> mLogger.info( "Test phase 2 (closing connections) ..." );
> disconnect();
> }
>
> mLogger.info( "Test 2 finished." );
> }
>
>
> I forgot to mention, that I work on the Mina trunk
> @version $Rev: 436993 $, $Date: 2006/11/21 23:30:42 $
>
> Best Regards
> Michael
>
Re: Closing sessions problem
Posted by Trustin Lee <tr...@gmail.com>.
On 12/23/06, Vinod Panicker <vi...@gmail.com> wrote:
>
> On 12/22/06, Trustin Lee <tr...@gmail.com> wrote:
> > On 12/22/06, Vinod Panicker <vi...@gmail.com> wrote:
> > >
> > > On 12/22/06, Michael Bauroth <mi...@falcom.de> wrote:
> > > > Ok,
> > > >
> > > > have new insights now:
> > > >
> > > > The problem occurs in first line because of a low ulimit value.
> > > >
> > > > On the other side Mina or the underlying socket has a non zero
> SoLinger
> > > > value set. It seems, that CloseFuture already returns isClosed and
> > > > isReady, but the underlying socket is still waiting a given amount
> of
> > > > time for a graceful shutdown of the tcp connection itself. So the
> still
> > > > closed and the new created sessions accumulate to greater number
> than
> > > > expected.
> > > >
> > > > When I set SoLinger to 0, the problem doesn't occur anymore. On the
> > > > other side I receive now IoExceptions over
> > > > SessionHandler.exceptionCaught() method, that the connection was
> > > > resetted. So far it's true, but not very helpful, because I must now
> > > > catch the exceptions silently instead of showing them.
> > > >
> > > > Any additional ideas here?
> > >
> > > I'm not sure I understand what exactly are you looking for.
> > >
> > > On an internet server, you will always have to account for TCP
> > > connections going into TIME_WAIT, FIN_WAIT, FIN_WAIT_2 and other
> > > states due to problems in the network or buggy implementations.
> > >
> > > So keep in mind that the limit on the number of open files on the
> > > system should allow for some headroom for a given concurrency level.
> >
> >
> > If using Linux, this blog entry I wrote might help people here:
> >
> > http://trustinlee.blogspot.com/2006/11/linux-default-ulimit.html
>
> It would be great if we can put this up on the wiki since it's a
> standard issue when running servers on Linux.
Sounds like a good idea. Let me take care of it.
Trustin
--
what we call human nature is actually human habit
--
http://gleamynode.net/
--
PGP key fingerprints:
* E167 E6AF E73A CBCE EE41 4A29 544D DE48 FE95 4E7E
* B693 628E 6047 4F8F CFA4 455E 1C62 A7DC 0255 ECA6
Re: Closing sessions problem
Posted by Vinod Panicker <vi...@gmail.com>.
On 12/22/06, Trustin Lee <tr...@gmail.com> wrote:
> On 12/22/06, Vinod Panicker <vi...@gmail.com> wrote:
> >
> > On 12/22/06, Michael Bauroth <mi...@falcom.de> wrote:
> > > Ok,
> > >
> > > have new insights now:
> > >
> > > The problem occurs in first line because of a low ulimit value.
> > >
> > > On the other side Mina or the underlying socket has a non zero SoLinger
> > > value set. It seems, that CloseFuture already returns isClosed and
> > > isReady, but the underlying socket is still waiting a given amount of
> > > time for a graceful shutdown of the tcp connection itself. So the still
> > > closed and the new created sessions accumulate to greater number than
> > > expected.
> > >
> > > When I set SoLinger to 0, the problem doesn't occur anymore. On the
> > > other side I receive now IoExceptions over
> > > SessionHandler.exceptionCaught() method, that the connection was
> > > resetted. So far it's true, but not very helpful, because I must now
> > > catch the exceptions silently instead of showing them.
> > >
> > > Any additional ideas here?
> >
> > I'm not sure I understand what exactly are you looking for.
> >
> > On an internet server, you will always have to account for TCP
> > connections going into TIME_WAIT, FIN_WAIT, FIN_WAIT_2 and other
> > states due to problems in the network or buggy implementations.
> >
> > So keep in mind that the limit on the number of open files on the
> > system should allow for some headroom for a given concurrency level.
>
>
> If using Linux, this blog entry I wrote might help people here:
>
> http://trustinlee.blogspot.com/2006/11/linux-default-ulimit.html
It would be great if we can put this up on the wiki since it's a
standard issue when running servers on Linux.
Regards,
Vinod.
Re: Closing sessions problem
Posted by Trustin Lee <tr...@gmail.com>.
On 12/22/06, Vinod Panicker <vi...@gmail.com> wrote:
>
> On 12/22/06, Michael Bauroth <mi...@falcom.de> wrote:
> > Ok,
> >
> > have new insights now:
> >
> > The problem occurs in first line because of a low ulimit value.
> >
> > On the other side Mina or the underlying socket has a non zero SoLinger
> > value set. It seems, that CloseFuture already returns isClosed and
> > isReady, but the underlying socket is still waiting a given amount of
> > time for a graceful shutdown of the tcp connection itself. So the still
> > closed and the new created sessions accumulate to greater number than
> > expected.
> >
> > When I set SoLinger to 0, the problem doesn't occur anymore. On the
> > other side I receive now IoExceptions over
> > SessionHandler.exceptionCaught() method, that the connection was
> > resetted. So far it's true, but not very helpful, because I must now
> > catch the exceptions silently instead of showing them.
> >
> > Any additional ideas here?
>
> I'm not sure I understand what exactly are you looking for.
>
> On an internet server, you will always have to account for TCP
> connections going into TIME_WAIT, FIN_WAIT, FIN_WAIT_2 and other
> states due to problems in the network or buggy implementations.
>
> So keep in mind that the limit on the number of open files on the
> system should allow for some headroom for a given concurrency level.
If using Linux, this blog entry I wrote might help people here:
http://trustinlee.blogspot.com/2006/11/linux-default-ulimit.html
HTH,
Trustin
--
what we call human nature is actually human habit
--
http://gleamynode.net/
--
PGP key fingerprints:
* E167 E6AF E73A CBCE EE41 4A29 544D DE48 FE95 4E7E
* B693 628E 6047 4F8F CFA4 455E 1C62 A7DC 0255 ECA6
Re: Closing sessions problem
Posted by Vinod Panicker <vi...@gmail.com>.
On 12/22/06, Michael Bauroth <mi...@falcom.de> wrote:
> Ok,
>
> have new insights now:
>
> The problem occurs in first line because of a low ulimit value.
>
> On the other side Mina or the underlying socket has a non zero SoLinger
> value set. It seems, that CloseFuture already returns isClosed and
> isReady, but the underlying socket is still waiting a given amount of
> time for a graceful shutdown of the tcp connection itself. So the still
> closed and the new created sessions accumulate to greater number than
> expected.
>
> When I set SoLinger to 0, the problem doesn't occur anymore. On the
> other side I receive now IoExceptions over
> SessionHandler.exceptionCaught() method, that the connection was
> resetted. So far it's true, but not very helpful, because I must now
> catch the exceptions silently instead of showing them.
>
> Any additional ideas here?
I'm not sure I understand what exactly are you looking for.
On an internet server, you will always have to account for TCP
connections going into TIME_WAIT, FIN_WAIT, FIN_WAIT_2 and other
states due to problems in the network or buggy implementations.
So keep in mind that the limit on the number of open files on the
system should allow for some headroom for a given concurrency level.
Regards,
Vinod.
Re: Closing sessions problem
Posted by Michael Bauroth <mi...@falcom.de>.
Ok,
have new insights now:
The problem occurs in first line because of a low ulimit value.
On the other side Mina or the underlying socket has a non zero SoLinger
value set. It seems, that CloseFuture already returns isClosed and
isReady, but the underlying socket is still waiting a given amount of
time for a graceful shutdown of the tcp connection itself. So the still
closed and the new created sessions accumulate to greater number than
expected.
When I set SoLinger to 0, the problem doesn't occur anymore. On the
other side I receive now IoExceptions over
SessionHandler.exceptionCaught() method, that the connection was
resetted. So far it's true, but not very helpful, because I must now
catch the exceptions silently instead of showing them.
Any additional ideas here?
Best Regards
Michael