You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mina.apache.org by Jeff Mesnil <jm...@redhat.com> on 2007/11/26 11:38:58 UTC
MINA acceptors open a lot of pipes
Hi,
I'm using MINA trunk in an application and I've encountered an issue
with it in our test suite.
Basically, for each test, our test suite starts & stops a MINA server.
Then depending on the tests, it opens connections or not.
However, even when our tests do not open connection (i.e. only MINA
acceptor is created), an excessive(?) number of pipes is created by MINA
leading to a "too many open files" exception after a certain number of
tests.
All the first tests run by our test suite does not use MINA connectors.
I commented out the code to start & stop MINA acceptor to see if it was
the cause of the number of pipes and it seems it is.
* without MINA start & stop:
$ lsof -p $PID | grep pipe | wc -l
3
* with MINA start & stop:
$ lsof -p $PID | grep pipe | wc -l
203
...
987 -> generate an exception "too many open files"
First question: is it normal that a succession of MINA acceptors
creation/disposal generates such a number of pipe?
The code I use to start & stop MINA is:
private void startMINAServer() throws Exception
{
if (acceptor == null)
{
acceptor = new NioSocketAcceptor();
// Prepare the configuration
acceptor.getFilterChain().addLast("mdc", new
MdcInjectionFilter());
acceptor.getFilterChain().addLast("codec",
new ProtocolCodecFilter(new PacketCodecFactory()));
acceptor.getFilterChain().addLast("logger", new LoggingFilter());
// Bind
acceptor.setLocalAddress(new InetSocketAddress(PORT));
acceptor.setReuseAddress(true);
acceptor.getSessionConfig().setReuseAddress(true);
acceptor.getSessionConfig().setKeepAlive(true);
acceptor.setDisconnectOnUnbind(false);
acceptor.setHandler(new ServerHandler());
acceptor.bind();
info("Started MINA on port " + PORT);
}
}
private void stopMINAServer()
{
if (acceptor != null)
{
acceptor.unbind();
acceptor.dispose();
info("Stopped MINA ");
}
}
I thought at first that I was responsible of the increase of pipes by
having unclosed streams. I don't think it is the case: I only used
ByteArray streams in the codecs.
It also seems that for each start of MINA, the increase is by 8 open
pipes every time and it does not decrease every time the acceptor is
disposed.
I'm still investigating and trying to isolate this problem but I was
wondering if it rings anyone's bell.
Do you have any idea of the cause of all these open pipes?
Thanks for the help,
jeff
Re: MINA acceptors open a lot of pipes
Posted by Trustin Lee <tr...@gmail.com>.
Hi Jeff,
On Nov 26, 2007 7:38 PM, Jeff Mesnil <jm...@redhat.com> wrote:
> Hi,
>
> I'm using MINA trunk in an application and I've encountered an issue
> with it in our test suite.
>
> Basically, for each test, our test suite starts & stops a MINA server.
> Then depending on the tests, it opens connections or not.
>
> However, even when our tests do not open connection (i.e. only MINA
> acceptor is created), an excessive(?) number of pipes is created by MINA
> leading to a "too many open files" exception after a certain number of
> tests.
>
> All the first tests run by our test suite does not use MINA connectors.
> I commented out the code to start & stop MINA acceptor to see if it was
> the cause of the number of pipes and it seems it is.
>
> * without MINA start & stop:
>
> $ lsof -p $PID | grep pipe | wc -l
> 3
>
> * with MINA start & stop:
>
> $ lsof -p $PID | grep pipe | wc -l
> 203
> ...
> 987 -> generate an exception "too many open files"
>
>
> First question: is it normal that a succession of MINA acceptors
> creation/disposal generates such a number of pipe?
>
> The code I use to start & stop MINA is:
>
> private void startMINAServer() throws Exception
> {
> if (acceptor == null)
> {
> acceptor = new NioSocketAcceptor();
>
> // Prepare the configuration
> acceptor.getFilterChain().addLast("mdc", new
> MdcInjectionFilter());
> acceptor.getFilterChain().addLast("codec",
> new ProtocolCodecFilter(new PacketCodecFactory()));
> acceptor.getFilterChain().addLast("logger", new LoggingFilter());
>
> // Bind
> acceptor.setLocalAddress(new InetSocketAddress(PORT));
> acceptor.setReuseAddress(true);
> acceptor.getSessionConfig().setReuseAddress(true);
> acceptor.getSessionConfig().setKeepAlive(true);
> acceptor.setDisconnectOnUnbind(false);
>
> acceptor.setHandler(new ServerHandler());
> acceptor.bind();
>
> info("Started MINA on port " + PORT);
> }
> }
>
> private void stopMINAServer()
> {
> if (acceptor != null)
> {
> acceptor.unbind();
> acceptor.dispose();
>
> info("Stopped MINA ");
> }
> }
>
> I thought at first that I was responsible of the increase of pipes by
> having unclosed streams. I don't think it is the case: I only used
> ByteArray streams in the codecs.
>
> It also seems that for each start of MINA, the increase is by 8 open
> pipes every time and it does not decrease every time the acceptor is
> disposed.
>
> I'm still investigating and trying to isolate this problem but I was
> wondering if it rings anyone's bell.
> Do you have any idea of the cause of all these open pipes?
It seems like you turned off disconnectOnUnbind flag. Are you sure
you closed all connections after unbind?
Trustin
--
what we call human nature is actually human habit
--
http://gleamynode.net/
--
PGP Key ID: 0x0255ECA6
Re: MINA acceptors open a lot of pipes
Posted by Jeff Mesnil <jm...@redhat.com>.
Hi Trustin,
Trustin Lee wrote:
> Hi Jeff,
>
> I've just succeeded to reproduce leakage and checked in the fix. The
> selector was not closed when dispose() is invoked against an acceptor
> which is not bound.
I've checked out your fix and I confirm than the number of open pipes
now remains steady.
thanks,
jeff
Re: MINA acceptors open a lot of pipes
Posted by Trustin Lee <tr...@gmail.com>.
Hi Jeff,
I've just succeeded to reproduce leakage and checked in the fix. The
selector was not closed when dispose() is invoked against an acceptor
which is not bound.
HTH,
Trustin
On Nov 26, 2007 7:38 PM, Jeff Mesnil <jm...@redhat.com> wrote:
> Hi,
>
> I'm using MINA trunk in an application and I've encountered an issue
> with it in our test suite.
>
> Basically, for each test, our test suite starts & stops a MINA server.
> Then depending on the tests, it opens connections or not.
>
> However, even when our tests do not open connection (i.e. only MINA
> acceptor is created), an excessive(?) number of pipes is created by MINA
> leading to a "too many open files" exception after a certain number of
> tests.
>
> All the first tests run by our test suite does not use MINA connectors.
> I commented out the code to start & stop MINA acceptor to see if it was
> the cause of the number of pipes and it seems it is.
>
> * without MINA start & stop:
>
> $ lsof -p $PID | grep pipe | wc -l
> 3
>
> * with MINA start & stop:
>
> $ lsof -p $PID | grep pipe | wc -l
> 203
> ...
> 987 -> generate an exception "too many open files"
>
>
> First question: is it normal that a succession of MINA acceptors
> creation/disposal generates such a number of pipe?
>
> The code I use to start & stop MINA is:
>
> private void startMINAServer() throws Exception
> {
> if (acceptor == null)
> {
> acceptor = new NioSocketAcceptor();
>
> // Prepare the configuration
> acceptor.getFilterChain().addLast("mdc", new
> MdcInjectionFilter());
> acceptor.getFilterChain().addLast("codec",
> new ProtocolCodecFilter(new PacketCodecFactory()));
> acceptor.getFilterChain().addLast("logger", new LoggingFilter());
>
> // Bind
> acceptor.setLocalAddress(new InetSocketAddress(PORT));
> acceptor.setReuseAddress(true);
> acceptor.getSessionConfig().setReuseAddress(true);
> acceptor.getSessionConfig().setKeepAlive(true);
> acceptor.setDisconnectOnUnbind(false);
>
> acceptor.setHandler(new ServerHandler());
> acceptor.bind();
>
> info("Started MINA on port " + PORT);
> }
> }
>
> private void stopMINAServer()
> {
> if (acceptor != null)
> {
> acceptor.unbind();
> acceptor.dispose();
>
> info("Stopped MINA ");
> }
> }
>
> I thought at first that I was responsible of the increase of pipes by
> having unclosed streams. I don't think it is the case: I only used
> ByteArray streams in the codecs.
>
> It also seems that for each start of MINA, the increase is by 8 open
> pipes every time and it does not decrease every time the acceptor is
> disposed.
>
> I'm still investigating and trying to isolate this problem but I was
> wondering if it rings anyone's bell.
> Do you have any idea of the cause of all these open pipes?
>
> Thanks for the help,
> jeff
>
>
>
>
--
what we call human nature is actually human habit
--
http://gleamynode.net/
--
PGP Key ID: 0x0255ECA6