You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pegasus.apache.org by GitBox <gi...@apache.org> on 2021/10/21 03:21:21 UTC
[GitHub] [incubator-pegasus] ZhongChaoqiang opened a new issue #832: partitions are unbalanced after the table is created
ZhongChaoqiang opened a new issue #832:
URL: https://github.com/apache/incubator-pegasus/issues/832
1. What did you do?
There are 5 nodes in our onebox cluster.
We create a app of 100 partitions.But partitions are unbalanced after the table is created.
```
>>> create test_100 -p 100
create app test_100 succeed, waiting for app ready
test_100 not ready yet, still waiting... (0/100)
test_100 not ready yet, still waiting... (0/100)
test_100 not ready yet, still waiting... (0/100)
test_100 not ready yet, still waiting... (0/100)
test_100 not ready yet, still waiting... (81/100)
test_100 not ready yet, still waiting... (81/100)
test_100 not ready yet, still waiting... (81/100)
test_100 not ready yet, still waiting... (81/100)
test_100 not ready yet, still waiting... (81/100)
test_100 not ready yet, still waiting... (99/100)
test_100 not ready yet, still waiting... (99/100)
test_100 not ready yet, still waiting... (99/100)
test_100 not ready yet, still waiting... (99/100)
test_100 not ready yet, still waiting... (99/100)
test_100 is ready now: (100/100)
test_100 is ready now!
create app "test_100" succeed
>>> ls -d
[general_info]
app_id status app_name app_type partition_count replica_count is_stateful create_time drop_time drop_expire envs_count is_bulkload
3 AVAILABLE test_100 pegasus 100 3 true 2021-10-21_11:16:44 - - 0 false
[healthy_info]
app_id app_name partition_count fully_healthy unhealthy write_unhealthy read_unhealthy
3 test_100 100 100 0 0 0
[summary]
total_app_count : 1
fully_healthy_app_count : 1
unhealthy_app_count : 0
write_unhealthy_app_count : 0
read_unhealthy_app_count : 0
>>> nodes -d
[details]
address status replica_count primary_count secondary_count
172.16.16.51:24361 ALIVE 58 20 38
172.16.16.51:24362 ALIVE 67 20 47
172.16.16.51:24363 ALIVE 60 20 40
172.16.16.51:24364 ALIVE 57 20 37
172.16.16.51:24365 ALIVE 58 20 38
[summary]
total_node_count : 5
alive_node_count : 5
unalive_node_count : 0
```
4. What version of Pegasus are you using?
2.0.0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org
[GitHub] [incubator-pegasus] ZhongChaoqiang commented on issue #832: partitions are unbalanced after the table is created
Posted by GitBox <gi...@apache.org>.
ZhongChaoqiang commented on issue #832:
URL: https://github.com/apache/incubator-pegasus/issues/832#issuecomment-948403379
建表时，meta是通过cure流程来分replica的。在分配secondary的replica时，是通过寻找最少replica的节点的方式，来觉得当前replica应该分配到哪个节点上。
判断一个节点的replica数据是如下函数：
```
int32_t newly_partitions::partition_count(int32_t app_id)
{
return owner->partition_count(app_id) + partitions[app_id];
}
```
这其中包含了两个变量，第一个是节点已经上线的replica的数量，第二个是正在分配中的replica的数量。理论上当一个分配中的replica上线后，第一个变量要加1，第二个变量要减1。但是实际上两个操作并不是同时完成的，所以当某个分片上线后，就可能导致这个函数计算出来数量比实际数量要大。从而导致分配不均匀了。
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org
[GitHub] [incubator-pegasus] ZhongChaoqiang closed issue #832: partitions are unbalanced after the table is created
Posted by GitBox <gi...@apache.org>.
ZhongChaoqiang closed issue #832:
URL: https://github.com/apache/incubator-pegasus/issues/832
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org