You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mina.apache.org by Coding Horse <zh...@hotmail.com> on 2007/04/05 05:40:40 UTC
Re: Question About Port Stucking(It has troubled me 4 a very long
term)
But why didn't I have this problem before using mina? I used nio directly
and never called setReuseAddres(). The server was in the same linux box
before using mina.
Vinod Panicker wrote:
>
> On 3/26/07, James Ling <cl...@tengshang.com> wrote:
>> It's An Problem that happened from time to time.
>> It still doesn't work well even if we enable reuse the address of the
>> acceptor nor every single connection's reuse the address option
>
> I believe this is a JVM bug on the mentioned platforms. AFAIK
> setReuseAddress() works as expected only on Windows.
>
> HTH,
> Vinod.
>
>
--
View this message in context: http://www.nabble.com/Question-About-Port-Stucking%28It-has-troubled-me-4-a-very-long-term%29-tf2697476.html#a9849620
Sent from the mina dev mailing list archive at Nabble.com.
Re: Question About Port Stucking(It has troubled me 4 a very long
term)
Posted by Niklas Therning <ni...@trillian.se>.
Trustin Lee wrote:
> On 4/5/07, Trustin Lee <tr...@gmail.com> wrote:
> <snip/>
>
>> The tough part is setting the default reuseAddr value in MINA 2.0. In
>> MINA 2.0, a user can call setSessionConfig(IoSessionConfig) with a new
>> DefaultSocketSessionConfig instance. It is especially useful when
>> Springframework is used to configure the acceptor. But, when a user
>> creates a new DefaultSocketSessionConfig instance, the instance itself
>> can't tell if it will be used for acceptor or connector. The default
>> reuseAddr property can't be set correctly at least for one side.
>>
>> There are three possible solutions:
>>
>> * Remove setSessionConfig()
>> * Add a constructor parameter so user can choose
>> * Provide two classes; ConnectorSocketSessionConfig and
>> AcceptorSocketSessionConfig
>>
>> I think the first one is the best for user friendliness if we can
>> provide proper spring integration. Any idea on spring integration?
>> Or, you could tell us what the best solution is.
>
> I found the first one doesn't hurt spring support that much because...
>
> <property name="sessionConfig.reuseAddress" value="true" />
>
> works! :D
>
> Niklas, what do you think about removing setSessionConfig() and hide
> default IoSessionConfig implementations from users like we did for
> 1.x?
I must admit that I didn't know you could use nested properties like
that! That's awesome! Since Spring does support that I don't think
setSessionConfig() is necessary. IMO we can remove it again.
--
Niklas Therning
www.spamdrain.net
Re: Question About Port Stucking(It has troubled me 4 a very long term)
Posted by Trustin Lee <tr...@gmail.com>.
On 4/5/07, Trustin Lee <tr...@gmail.com> wrote:
<snip/>
> The tough part is setting the default reuseAddr value in MINA 2.0. In
> MINA 2.0, a user can call setSessionConfig(IoSessionConfig) with a new
> DefaultSocketSessionConfig instance. It is especially useful when
> Springframework is used to configure the acceptor. But, when a user
> creates a new DefaultSocketSessionConfig instance, the instance itself
> can't tell if it will be used for acceptor or connector. The default
> reuseAddr property can't be set correctly at least for one side.
>
> There are three possible solutions:
>
> * Remove setSessionConfig()
> * Add a constructor parameter so user can choose
> * Provide two classes; ConnectorSocketSessionConfig and
> AcceptorSocketSessionConfig
>
> I think the first one is the best for user friendliness if we can
> provide proper spring integration. Any idea on spring integration?
> Or, you could tell us what the best solution is.
I found the first one doesn't hurt spring support that much because...
<property name="sessionConfig.reuseAddress" value="true" />
works! :D
Niklas, what do you think about removing setSessionConfig() and hide
default IoSessionConfig implementations from users like we did for
1.x?
Trustin
--
what we call human nature is actually human habit
--
http://gleamynode.net/
--
PGP Key ID: 0x0255ECA6
Re: Question About Port Stucking(It has troubled me 4 a very long term)
Posted by Trustin Lee <tr...@gmail.com>.
On 4/5/07, Trustin Lee <tr...@gmail.com> wrote:
> Hi Vinod and Coding horse,
>
> On 4/5/07, Vinod Panicker <vi...@gmail.com> wrote:
> <snip/>
> > > I tried the following solutions:
> > >
> > > 1) acceptor.setReuseAddress(true); -> doesn't work.
> > > 2) acceptor.getSessionConfig().setReuseAddress(true); -> works!
> > >
> > > It seems like Linux doesn't clean up connection state and prevents
> > > binding the port which the dangling connection is referencing. It
> > > seems like setting a SO_REUSE_ADDR on an accepted socket disables the
> > > protection. I'm not sure how this differs in Windows.
> >
> > Cool. It's interesting how its working when you set the option on the
> > SessionConfig. Technically speaking, an application is not supposed
> > to bind again on a socket/port that is in a *_WAIT state, since the
> > state exists so data "in transmission" may be safely expired.
> > Rebinding a server on a socket in a wait state using the SO_REUSE_ADDR
> > may be necessary for server systems (to reduce downtime), but could
> > also potentially generate some interesting issues.
> >
> > The way it works is exactly the same on both Linux and Windows, and
> > it's not a Linux issue, but rather the way the JVM is setting the
> > SO_REUSE_ADDR option on the socket. I've coded servers in C++ on Linux
> > and successfully managed to use this option for rebinding.
> >
> > Let me try your workaround - will get back on if it works for me too.
>
> I found why. :D
>
> Take a look at here:
>
> SocketSessionConfigImpl: http://tinyurl.com/3468gf
>
> DEFAULT_REUSE_ADDRESS is set to 'false' in Linux. But, according to
> the bug reporters, this problem didn't show up in non-MINA NIO
> servers. So I printed the REUSE_ADDR option as soon as a new
> connection is accepted. Guess what? The default value of REUSE_ADDR
> option for an accepted socket was 'true'! This means that the default
> values of socket options are different between a client socket and an
> accepted socket. (and also probably true for a server socket?)
>
> Therefore, we can conclude that all option parameters have to be
> calculated for three kinds of sockets to fix this problem.
I quickly checked the default option values, and the following is the
list of differences I've found.
* reuseAddr is true for an accepted socket, while others return false.
* new Socket().getSendBufferSize() returns 8192, but new
Socket("localhost", avaialblePort).getSendBufferSize() and
acceptedSocket.getSendBufferSize() returns 25290. It seems like we
need to create a connected socket by opening a port to get the correct
value.
Considering the differences above, I think we don't need to get all
option values for all kinds of sockets, and we can fix the buffer size
difference very easily. Setting the default reuseAddr value in MINA
1.x will also be a piece of cake because we can simply change the
reuseAddr property after a SocketSessionImpl is created.
The tough part is setting the default reuseAddr value in MINA 2.0. In
MINA 2.0, a user can call setSessionConfig(IoSessionConfig) with a new
DefaultSocketSessionConfig instance. It is especially useful when
Springframework is used to configure the acceptor. But, when a user
creates a new DefaultSocketSessionConfig instance, the instance itself
can't tell if it will be used for acceptor or connector. The default
reuseAddr property can't be set correctly at least for one side.
There are three possible solutions:
* Remove setSessionConfig()
* Add a constructor parameter so user can choose
* Provide two classes; ConnectorSocketSessionConfig and
AcceptorSocketSessionConfig
I think the first one is the best for user friendliness if we can
provide proper spring integration. Any idea on spring integration?
Or, you could tell us what the best solution is.
Trustin
--
what we call human nature is actually human habit
--
http://gleamynode.net/
--
PGP Key ID: 0x0255ECA6
Re: Question About Port Stucking(It has troubled me 4 a very long term)
Posted by Trustin Lee <tr...@gmail.com>.
Hi Vinod and Coding horse,
On 4/5/07, Vinod Panicker <vi...@gmail.com> wrote:
<snip/>
> > I tried the following solutions:
> >
> > 1) acceptor.setReuseAddress(true); -> doesn't work.
> > 2) acceptor.getSessionConfig().setReuseAddress(true); -> works!
> >
> > It seems like Linux doesn't clean up connection state and prevents
> > binding the port which the dangling connection is referencing. It
> > seems like setting a SO_REUSE_ADDR on an accepted socket disables the
> > protection. I'm not sure how this differs in Windows.
>
> Cool. It's interesting how its working when you set the option on the
> SessionConfig. Technically speaking, an application is not supposed
> to bind again on a socket/port that is in a *_WAIT state, since the
> state exists so data "in transmission" may be safely expired.
> Rebinding a server on a socket in a wait state using the SO_REUSE_ADDR
> may be necessary for server systems (to reduce downtime), but could
> also potentially generate some interesting issues.
>
> The way it works is exactly the same on both Linux and Windows, and
> it's not a Linux issue, but rather the way the JVM is setting the
> SO_REUSE_ADDR option on the socket. I've coded servers in C++ on Linux
> and successfully managed to use this option for rebinding.
>
> Let me try your workaround - will get back on if it works for me too.
I found why. :D
Take a look at here:
SocketSessionConfigImpl: http://tinyurl.com/3468gf
DEFAULT_REUSE_ADDRESS is set to 'false' in Linux. But, according to
the bug reporters, this problem didn't show up in non-MINA NIO
servers. So I printed the REUSE_ADDR option as soon as a new
connection is accepted. Guess what? The default value of REUSE_ADDR
option for an accepted socket was 'true'! This means that the default
values of socket options are different between a client socket and an
accepted socket. (and also probably true for a server socket?)
Therefore, we can conclude that all option parameters have to be
calculated for three kinds of sockets to fix this problem.
Cheers,
Trustin
--
what we call human nature is actually human habit
--
http://gleamynode.net/
--
PGP Key ID: 0x0255ECA6
Re: Question About Port Stucking(It has troubled me 4 a very long term)
Posted by Vinod Panicker <vi...@gmail.com>.
On 4/5/07, Trustin Lee <tr...@gmail.com> wrote:
> Hi Vinod,
>
> On 4/5/07, Vinod Panicker <vi...@gmail.com> wrote:
> > On 4/5/07, Trustin Lee <tr...@gmail.com> wrote:
> > > On 4/5/07, Coding Horse <zh...@hotmail.com> wrote:
> > > >
> > > > But why didn't I have this problem before using mina? I used nio directly
> > > > and never called setReuseAddres(). The server was in the same linux box
> > > > before using mina.
> > >
> > > Why don't you trace the problem and let us know why if you are having
> > > the problem? I can't fix it because it works OK for me.
> >
> > I believe this problem will show itself only when the disconnected
> > client sockets are in a *_WAIT state. To reproduce this, you will
> > need to use clients that are not local (since the sockets get recycled
> > pretty fast), but rather on the internet or on some simulated low
> > bandwidth / high latency connection.
>
> I was able to reproduce this problem in my Linux box today. I tried
> three scenarios:
>
> 1) Server starts up, and killed -9: OK with rebinding
> 2) Server starts up, a client connects, the connection closes, and
> killed -9: OK with rebinding
> 3) Server starts up, a client connects, and server gets killed -9.:
> Gets BindException on rebinding.
>
> Looking closely the scenario 3, I found a connection with TIME_WAIT
> state through netstat. I was able to bind again after the TIME_WAIT
> state goes away.
>
> I tried the following solutions:
>
> 1) acceptor.setReuseAddress(true); -> doesn't work.
> 2) acceptor.getSessionConfig().setReuseAddress(true); -> works!
>
> It seems like Linux doesn't clean up connection state and prevents
> binding the port which the dangling connection is referencing. It
> seems like setting a SO_REUSE_ADDR on an accepted socket disables the
> protection. I'm not sure how this differs in Windows.
Cool. It's interesting how its working when you set the option on the
SessionConfig. Technically speaking, an application is not supposed
to bind again on a socket/port that is in a *_WAIT state, since the
state exists so data "in transmission" may be safely expired.
Rebinding a server on a socket in a wait state using the SO_REUSE_ADDR
may be necessary for server systems (to reduce downtime), but could
also potentially generate some interesting issues.
The way it works is exactly the same on both Linux and Windows, and
it's not a Linux issue, but rather the way the JVM is setting the
SO_REUSE_ADDR option on the socket. I've coded servers in C++ on Linux
and successfully managed to use this option for rebinding.
Let me try your workaround - will get back on if it works for me too.
Regards,
Vinod.
Re: Question About Port Stucking(It has troubled me 4 a very long
term)
Posted by Coding Horse <zh...@hotmail.com>.
Great job! Trustin.
I just updated mina trunk and tried
acceptor.getSessionConfig().setReuseAddress(true);
and it works like magic in the production server!
Sorry about not providing enough problem trace when I reported the issue on
my server - the major reason among others is that the water is still too
deep to me.
Trustin Lee wrote:
>
> Hi Vinod,
>
> On 4/5/07, Vinod Panicker <vi...@gmail.com> wrote:
>> On 4/5/07, Trustin Lee <tr...@gmail.com> wrote:
>> > On 4/5/07, Coding Horse <zh...@hotmail.com> wrote:
>> > >
>> > > But why didn't I have this problem before using mina? I used nio
>> directly
>> > > and never called setReuseAddres(). The server was in the same linux
>> box
>> > > before using mina.
>> >
>> > Why don't you trace the problem and let us know why if you are having
>> > the problem? I can't fix it because it works OK for me.
>>
>> I believe this problem will show itself only when the disconnected
>> client sockets are in a *_WAIT state. To reproduce this, you will
>> need to use clients that are not local (since the sockets get recycled
>> pretty fast), but rather on the internet or on some simulated low
>> bandwidth / high latency connection.
>
> I was able to reproduce this problem in my Linux box today. I tried
> three scenarios:
>
> 1) Server starts up, and killed -9: OK with rebinding
> 2) Server starts up, a client connects, the connection closes, and
> killed -9: OK with rebinding
> 3) Server starts up, a client connects, and server gets killed -9.:
> Gets BindException on rebinding.
>
> Looking closely the scenario 3, I found a connection with TIME_WAIT
> state through netstat. I was able to bind again after the TIME_WAIT
> state goes away.
>
> I tried the following solutions:
>
> 1) acceptor.setReuseAddress(true); -> doesn't work.
> 2) acceptor.getSessionConfig().setReuseAddress(true); -> works!
>
> It seems like Linux doesn't clean up connection state and prevents
> binding the port which the dangling connection is referencing. It
> seems like setting a SO_REUSE_ADDR on an accepted socket disables the
> protection. I'm not sure how this differs in Windows.
>
> Trustin
> --
> what we call human nature is actually human habit
> --
> http://gleamynode.net/
> --
> PGP Key ID: 0x0255ECA6
>
>
--
View this message in context: http://www.nabble.com/Question-About-Port-Stucking%28It-has-troubled-me-4-a-very-long-term%29-tf2697476.html#a9856385
Sent from the mina dev mailing list archive at Nabble.com.
Re: Question About Port Stucking(It has troubled me 4 a very long term)
Posted by Trustin Lee <tr...@gmail.com>.
Hi Vinod,
On 4/5/07, Vinod Panicker <vi...@gmail.com> wrote:
> On 4/5/07, Trustin Lee <tr...@gmail.com> wrote:
> > On 4/5/07, Coding Horse <zh...@hotmail.com> wrote:
> > >
> > > But why didn't I have this problem before using mina? I used nio directly
> > > and never called setReuseAddres(). The server was in the same linux box
> > > before using mina.
> >
> > Why don't you trace the problem and let us know why if you are having
> > the problem? I can't fix it because it works OK for me.
>
> I believe this problem will show itself only when the disconnected
> client sockets are in a *_WAIT state. To reproduce this, you will
> need to use clients that are not local (since the sockets get recycled
> pretty fast), but rather on the internet or on some simulated low
> bandwidth / high latency connection.
I was able to reproduce this problem in my Linux box today. I tried
three scenarios:
1) Server starts up, and killed -9: OK with rebinding
2) Server starts up, a client connects, the connection closes, and
killed -9: OK with rebinding
3) Server starts up, a client connects, and server gets killed -9.:
Gets BindException on rebinding.
Looking closely the scenario 3, I found a connection with TIME_WAIT
state through netstat. I was able to bind again after the TIME_WAIT
state goes away.
I tried the following solutions:
1) acceptor.setReuseAddress(true); -> doesn't work.
2) acceptor.getSessionConfig().setReuseAddress(true); -> works!
It seems like Linux doesn't clean up connection state and prevents
binding the port which the dangling connection is referencing. It
seems like setting a SO_REUSE_ADDR on an accepted socket disables the
protection. I'm not sure how this differs in Windows.
Trustin
--
what we call human nature is actually human habit
--
http://gleamynode.net/
--
PGP Key ID: 0x0255ECA6
Re: Question About Port Stucking(It has troubled me 4 a very long term)
Posted by Vinod Panicker <vi...@gmail.com>.
On 4/5/07, Trustin Lee <tr...@gmail.com> wrote:
> On 4/5/07, Coding Horse <zh...@hotmail.com> wrote:
> >
> > But why didn't I have this problem before using mina? I used nio directly
> > and never called setReuseAddres(). The server was in the same linux box
> > before using mina.
>
> Why don't you trace the problem and let us know why if you are having
> the problem? I can't fix it because it works OK for me.
I believe this problem will show itself only when the disconnected
client sockets are in a *_WAIT state. To reproduce this, you will
need to use clients that are not local (since the sockets get recycled
pretty fast), but rather on the internet or on some simulated low
bandwidth / high latency connection.
HTH,
Vinod.
Re: Question About Port Stucking(It has troubled me 4 a very long term)
Posted by Trustin Lee <tr...@gmail.com>.
On 4/5/07, Coding Horse <zh...@hotmail.com> wrote:
>
> But why didn't I have this problem before using mina? I used nio directly
> and never called setReuseAddres(). The server was in the same linux box
> before using mina.
Why don't you trace the problem and let us know why if you are having
the problem? I can't fix it because it works OK for me.
Trustin
--
what we call human nature is actually human habit
--
http://gleamynode.net/
--
PGP Key ID: 0x0255ECA6