You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@brpc.apache.org by GitBox <gi...@apache.org> on 2020/07/24 03:01:23 UTC
[GitHub] [incubator-brpc] lzueclipse opened a new issue #1183: RDMA 对于bonding的支持有开发计划么？
lzueclipse opened a new issue #1183:
URL: https://github.com/apache/incubator-brpc/issues/1183
**我测试的环境如下：**
client端mlx5_1，mlx5_3配置为bond0，配置IP为Client_IP
server端mlx5_0，mlx5_2配置为bond0，配置IP为Server_IP
**运行example/rdma_performance：**
1）./perf_server --port=4321 --rdma_device=mlx5_2
2)perf_client --servers=Server_IP:4321 \
--thread_num=1 \
--attachment_size=4 \
--echo_attachment=fasle \
--test_seconds=30 \
--rdma_device=mlx5_1 \
--use_rdma=true
对于perf_client, 当rdma_device=mlx5_1测试**成功**；当rdma_device=mlx5_3**失败**。
**初步分析：**
当创建QP时候出错，出错信息“Failed to rdma_create_qp: Invalid argument [22]”。
对应代码：
_if (RdmaCreateQp(cm_id, (ibv_pd*)GetRdmaProtectionDomain(), &qp_attr) < 0) {
PLOG(WARNING) << "Fail to rdma_create_qp";
return NULL;
}_
我在RdmaCreateQp前，加了如下log:
LOG(INFO) << "Device: " << (char *) IbvGetDeviceName(cm_id->verbs->device);
发现无论--rdma_device指定什么，此处的cm_id->verbs->device**一直是mlx5_1**。
**可能原因：**
有一个TODO，要支持bonding。
// Map ibdev (e.g. mlx5_0) to netdev (e.g. eth0), must compare sysfs file
// TODO: this method cannot handle bonding mode, see more in ibdev2netd
因为很多线上环境的都是两块网卡作bonding，不知道RDMA这里对bonding的支持有开发计划么？
或者大拿们提供个实现思路？
多谢。
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@brpc.apache.org
For additional commands, e-mail: dev-help@brpc.apache.org
[GitHub] [incubator-brpc] lzueclipse commented on issue #1183: RDMA 对于bonding的支持有开发计划么？
Posted by GitBox <gi...@apache.org>.
lzueclipse commented on issue #1183:
URL: https://github.com/apache/incubator-brpc/issues/1183#issuecomment-663328354
更新了描述：
发现--rdma_device=mlx5_3时，此处的cm_id->verbs->device一直是mlx5_1；而GetRdmaProtectionDomain()是基于--rdma-device=mlx5_3创建的。
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@brpc.apache.org
For additional commands, e-mail: dev-help@brpc.apache.org
[GitHub] [incubator-brpc] lzueclipse commented on issue #1183: RDMA 对于bonding的支持有开发计划么？
Posted by GitBox <gi...@apache.org>.
lzueclipse commented on issue #1183:
URL: https://github.com/apache/incubator-brpc/issues/1183#issuecomment-664813306
追了下代码，为了zero copy，RDMA的实现会托管iobuf的内存分配，目前这种方式，实现多网卡支持确实有难度。
为了支持多网卡，我准备放弃zero copy，试图实现下。
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@brpc.apache.org
For additional commands, e-mail: dev-help@brpc.apache.org