You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@rocketmq.apache.org by GitBox <gi...@apache.org> on 2021/11/23 10:48:37 UTC
[GitHub] [rocketmq] XiaoyiPeng opened a new issue #3528: Connections to RocketMQ brokers are often refused in high-load production environments
XiaoyiPeng opened a new issue #3528:
URL: https://github.com/apache/rocketmq/issues/3528
**BUG REPORT**
1. Please describe the issue you observed:
Connections to RocketMQ brokers are often refused in high-load production environments !
On the server, the number of sockets in the ESTABLISHED state is less than one thousand.
2. Please tell us about your environment:
CentOS Linux release 7.6.1810
3. Other information (e.g. detailed explanation, logs, related issues, suggestions how to fix, etc):
Connections to RocketMQ brokers are often refused in high-load production environments, and on the server, the number of sockets in the ESTABLISHED state is less than one thousand.
We debug the code, find `ChannelOption.SO_BACKLOG` is hard coded with value 1024 in class org.apache.rocketmq.remoting.netty.NettyRemotingServer, as shown below :
```
ServerBootstrap childHandler =
this.serverBootstrap.group(this.eventLoopGroupBoss, this.eventLoopGroupSelector)
.channel(useEpoll() ? EpollServerSocketChannel.class : NioServerSocketChannel.class)
.option(ChannelOption.SO_BACKLOG, 1024)
.option(ChannelOption.SO_REUSEADDR, true)
.option(ChannelOption.SO_KEEPALIVE, false)
.childOption(ChannelOption.TCP_NODELAY, true)
.childOption(ChannelOption.SO_SNDBUF, nettyServerConfig.getServerSocketSndBufSize())
.childOption(ChannelOption.SO_RCVBUF, nettyServerConfig.getServerSocketRcvBufSize())
.localAddress(new InetSocketAddress(this.nettyServerConfig.getListenPort())
```
But according to the JDK documentation in **`java.nio.channels.ServerSocketChannel`**, this value **`backlog`** means **maximum number of pending connections** instead of completely established connection, as shown below :
```
* @param local
* The address to bind the socket, or {@code null} to bind to an
* automatically assigned socket address
* @param backlog
* The maximum number of **pending connections**
*
* @return This channel
*/
public abstract ServerSocketChannel bind(SocketAddress local, int backlog)
throws IOException;
```
I looked at the implementation of this method `sun.nio.ch.ServerSocketChannelImpl#bind` in the OpenJDK ,as show below :
```
private SocketAddress netBind(SocketAddress local, int backlog) throws IOException {
InetSocketAddress isa;
if (local == null) {
isa = new InetSocketAddress(Net.anyLocalAddress(family), 0);
} else {
isa = Net.checkAddress(local, family);
}
@SuppressWarnings("removal")
SecurityManager sm = System.getSecurityManager();
if (sm != null)
sm.checkListen(isa.getPort());
NetHooks.beforeTcpBind(fd, isa.getAddress(), isa.getPort());
Net.bind(family, fd, isa.getAddress(), isa.getPort());
Net.listen(fd, backlog < 1 ? 50 : backlog);
return Net.localAddress(fd);
}
```
in class `sun.nio.ch.Net`, it calls the native method as shown below:
```static native void listen(FileDescriptor fd, int backlog) throws IOException;```
And I guess it invoke linux system call `int listen(int sockfd, int backlog);` , In the official documentation, the following explanation is found:
**Attention :**
**The behavior of the `**backlog** `argument on TCP sockets changed with Linux 2.2. Now it specifies the queue length for completely established sockets waiting to be accepted, instead of the number of incomplete connection requests.**
To summarize, everything can be explained.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] RongtongJin closed issue #3528: The value of ChannelOption.SO_BACKLOG should not be hard coded
Posted by GitBox <gi...@apache.org>.
RongtongJin closed issue #3528:
URL: https://github.com/apache/rocketmq/issues/3528
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] RongtongJin commented on issue #3528: The value of ChannelOption.SO_BACKLOG should not be hard coded
Posted by GitBox <gi...@apache.org>.
RongtongJin commented on issue #3528:
URL: https://github.com/apache/rocketmq/issues/3528#issuecomment-977446594
Merged
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org