You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pegasus.apache.org by "empiredan (via GitHub)" <gi...@apache.org> on 2024/05/07 08:20:12 UTC
[PR] feat(duplication): add options to support cluster name only used for duplication and allow any other cluster id except myself to be ignored [incubator-pegasus]
empiredan opened a new pull request, #2000:
URL: https://github.com/apache/incubator-pegasus/pull/2000
The purpose of this PR is to optimize configurations for duplications.
Firstly, many Pegasus clusters are configured with the same `cluster_name`
(namely `[replication]cluster_name`). However, once we decide to duplicate
tables between them, their `cluster_name` have to be changed to be distinguished
from each other -- this might lead to side effects.
Secondly, consider a scenario where many small clusters are duplicated to
a big cluster. This means we have to add many cluster ids to the `.ini` file
of the big cluster, and the big cluster might be restarted very frequently.
This PR is to solve both problems by adding options:
```diff
[replication]
+ dup_cluster_name =
+ dup_ignore_other_cluster_ids = false
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org
Re: [PR] feat(duplication): add options to support cluster name only used for duplication and allow any other cluster id except myself to be ignored [incubator-pegasus]
Posted by "empiredan (via GitHub)" <gi...@apache.org>.
empiredan merged PR #2000:
URL: https://github.com/apache/incubator-pegasus/pull/2000
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org
Re: [PR] feat(duplication): add options to support cluster name only used for duplication and allow any other cluster id except myself to be ignored [incubator-pegasus]
Posted by "acelyc111 (via GitHub)" <gi...@apache.org>.
acelyc111 commented on code in PR #2000:
URL: https://github.com/apache/incubator-pegasus/pull/2000#discussion_r1597412463
##########
src/common/duplication_common.cpp:
##########
@@ -37,6 +39,17 @@ DSN_DEFINE_uint32(replication,
"send mutation log batch bytes size per rpc");
DSN_TAG_VARIABLE(duplicate_log_batch_bytes, FT_MUTABLE);
+// While many small clusters are duplicated to a big cluster, we have to add many cluster
Review Comment:
The "small" and "big" are not very accurate, this situaction would happen on any scale of clusters.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org
Re: [PR] feat(duplication): add options to support cluster name only used for duplication and allow any other cluster id except myself to be ignored [incubator-pegasus]
Posted by "acelyc111 (via GitHub)" <gi...@apache.org>.
acelyc111 commented on PR #2000:
URL: https://github.com/apache/incubator-pegasus/pull/2000#issuecomment-2101902883
Thumbs for the 2000th!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org
Re: [PR] feat(duplication): add options to support cluster name only used for duplication and allow any other cluster id except myself to be ignored [incubator-pegasus]
Posted by "empiredan (via GitHub)" <gi...@apache.org>.
empiredan commented on code in PR #2000:
URL: https://github.com/apache/incubator-pegasus/pull/2000#discussion_r1597427996
##########
src/common/duplication_common.cpp:
##########
@@ -37,6 +39,17 @@ DSN_DEFINE_uint32(replication,
"send mutation log batch bytes size per rpc");
DSN_TAG_VARIABLE(duplicate_log_batch_bytes, FT_MUTABLE);
+// While many small clusters are duplicated to a big cluster, we have to add many cluster
Review Comment:
OK, I'll change the comments.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org