You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@brpc.apache.org by "binarycopycode (via GitHub)" <gi...@apache.org> on 2023/12/20 07:53:15 UTC
[I] 使用rdma传输1G的数据，接收端接收不到 (brpc)
binarycopycode opened a new issue, #2475:
URL: https://github.com/apache/brpc/issues/2475
**Describe the bug (描述bug)**
自己写了一个[测试](https://github.com/binarycopycode/brpc_rdma_tcp_perf)rdma和tcp速度差别的小测试，自己测试了128,256,512Mb都没问题。当传输大小为1G的数据时，rdma无任何报错，但是client接收到的attachment().size()是0，而使用tcp传输大小为1G的数据时，则出现报错
```
W1220 15:39:25.600674 5980 /home/binarycopycode/work/brpc_rdma_tcp_perf/src/brpc/input_messenger.cpp:375] Fail to read from Socket{id=102 fd=9 addr=192.168.182.129:40614:8011} (0x7f0d58026820): Connection reset by peer
W1220 15:39:25.600733 5980 /home/binarycopycode/work/brpc_rdma_tcp_perf/src/brpc/socket.cpp:1774] Fail to keep-write into Socket{id=102 fd=9 addr=192.168.182.129:40614:8011} (0x7f0d58026820): Broken pipe
```
**To Reproduce (复现方法)**
按照https://github.com/binarycopycode/brpc_rdma_tcp_perf 编译
测试rdma，注意我的池子大小和brpc的max_body_size都设为了2G，注意修改运行client命令的ip地址
`./server -port=8010 -data_size_mb=1023 -rdma_memory_pool_initial_size_mb=2048 -max_body_size=2147483648`
`./client -server=192.168.182.129:8010 -rdma_memory_pool_initial_size_mb=2048 -max_body_size=2147483648`
测试tcp,加上-use_rdma=false即可
`./server -use_rdma=false -port=8011 -data_size_mb=1023 -rdma_memory_pool_initial_size_mb=2048 -max_body_size=2147483648`
**Expected behavior (期望行为)**
client能正常接收attachment
**Versions (各种版本)**
OS: CentOS8.5
Compiler: g++ (GCC) 8.5.0 20210514 (Red Hat 8.5.0-4)
brpc:
protobuf: libprotoc 3.5.0
**Additional context/screenshots (更多上下文/截图)**
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@brpc.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@brpc.apache.org
For additional commands, e-mail: dev-help@brpc.apache.org
Re: [I] 使用rdma传输1G的数据，接收端接收不到 (brpc)
Posted by "binarycopycode (via GitHub)" <gi...@apache.org>.
binarycopycode commented on issue #2475:
URL: https://github.com/apache/brpc/issues/2475#issuecomment-1865406094
以上为本地虚拟机测试。后来在两台有mlx的RDMA网卡的服务器上测试又不一样了。其中tcp能传1G的数据，但是到2G就出现一样的Broken pipe错误，但是RDMA可以传2G数据，但是4G数据被iobuf的限制给写死了，这个有办法解决吗？ @Tuvie 是否能去掉这个限制，还是只能自己手动分段传数据了？
```
int IOBuf::append_user_data_with_meta(void* data,
size_t size,
void (*deleter)(void*),
uint64_t meta) {
if (size > 0xFFFFFFFFULL - 100) {
LOG(FATAL) << "data_size=" << size << " is too large";
return -1;
}
......
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@brpc.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@brpc.apache.org
For additional commands, e-mail: dev-help@brpc.apache.org
Re: [I] rdma的IOBuf::append_user_data_with_meta无法传输4G以上的数据 (brpc)
Posted by "Superskyyy (via GitHub)" <gi...@apache.org>.
Superskyyy commented on issue #2475:
URL: https://github.com/apache/brpc/issues/2475#issuecomment-1934799033
请问问题有解决吗 @binarycopycode
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@brpc.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@brpc.apache.org
For additional commands, e-mail: dev-help@brpc.apache.org