You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@rocketmq.apache.org by GitBox <gi...@apache.org> on 2021/11/04 11:30:29 UTC
[GitHub] [rocketmq] iamqq23ue opened a new issue #3461: Can rocketmq have one master corresponding to two slaves?
iamqq23ue opened a new issue #3461:
URL: https://github.com/apache/rocketmq/issues/3461
version：4.9.1
If a broker sets SYNC_MASTER, corresponding to two slaves, can three nodes write synchronously?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] vongosling edited a comment on issue #3461: Can rocketmq have one master corresponding to two slaves?
Posted by GitBox <gi...@apache.org>.
vongosling edited a comment on issue #3461:
URL: https://github.com/apache/rocketmq/issues/3461#issuecomment-962398039
> Dledger has some bugs and cannot realize disaster recovery. So I don't plan to choose Dledger yet.
> […](#)
> ------------------ 原始邮件 ------------------ 发件人: "apache/rocketmq" ***@***.***>; 发送时间: 2021年11月4日(星期四) 晚上9:49 ***@***.***>; ***@***.******@***.***>; 主题: Re: [apache/rocketmq] Can rocketmq have one master corresponding to two slaves? (Issue #3461) version：4.9.1 If a broker sets SYNC_MASTER, corresponding to two slaves, can three nodes write synchronously? Using the DLedger mode will ensure that more than half of the nodes will return immediately if synchronization is successful, so when there are 3 nodes, as long as there is a slave that is successfully synchronized, it will return. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.
DLedger is our state-of-the-art HA implementation since 4.5. Many guys have deployed it in their production env. If you have other opt ideas, pls let's know @dongeforever @RongtongJin are our mainly dledger authors.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] Git-Yang commented on issue #3461: Can rocketmq have one master corresponding to two slaves?
Posted by GitBox <gi...@apache.org>.
Git-Yang commented on issue #3461:
URL: https://github.com/apache/rocketmq/issues/3461#issuecomment-960971542
> version：4.9.1 If a broker sets SYNC_MASTER, corresponding to two slaves, can three nodes write synchronously?
Using the DLedger mode will ensure that more than half of the nodes will return immediately if synchronization is successful, so when there are 3 nodes, as long as there is a slave that is successfully synchronized, it will return.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] dongeforever commented on issue #3461: Can rocketmq have one master corresponding to two slaves?
Posted by GitBox <gi...@apache.org>.
dongeforever commented on issue #3461:
URL: https://github.com/apache/rocketmq/issues/3461#issuecomment-961662094
This is the cold data problem。
For in-sync slave, the master loads data from page cache.
But for out-of-sync slave, the master needs to load data from disk, this will be much slower than page cache.
If the incoming tps is large，the out of sync slave may never have the chance to catch up with the master.
IMO，We may need to polish the cold data sync performance.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] iamqq23ue commented on issue #3461: Can rocketmq have one master corresponding to two slaves?
Posted by GitBox <gi...@apache.org>.
iamqq23ue commented on issue #3461:
URL: https://github.com/apache/rocketmq/issues/3461#issuecomment-961563275
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] vongosling commented on issue #3461: Can rocketmq have one master corresponding to two slaves?
Posted by GitBox <gi...@apache.org>.
vongosling commented on issue #3461:
URL: https://github.com/apache/rocketmq/issues/3461#issuecomment-962398039
> Dledger has some bugs and cannot realize disaster recovery. So I don't plan to choose Dledger yet.
> […](#)
> ------------------ 原始邮件 ------------------ 发件人: "apache/rocketmq" ***@***.***>; 发送时间: 2021年11月4日(星期四) 晚上9:49 ***@***.***>; ***@***.******@***.***>; 主题: Re: [apache/rocketmq] Can rocketmq have one master corresponding to two slaves? (Issue #3461) version：4.9.1 If a broker sets SYNC_MASTER, corresponding to two slaves, can three nodes write synchronously? Using the DLedger mode will ensure that more than half of the nodes will return immediately if synchronization is successful, so when there are 3 nodes, as long as there is a slave that is successfully synchronized, it will return. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.
I don't understand what your mentioned bug in dledger, this is our state-of-the-art HA implementation. If you point this problem, I will drive the community to opt here. @dongeforever @RongtongJin are our mainly dledger authors.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] iamqq23ue commented on issue #3461: Can rocketmq have one master corresponding to two slaves?
Posted by GitBox <gi...@apache.org>.
iamqq23ue commented on issue #3461:
URL: https://github.com/apache/rocketmq/issues/3461#issuecomment-961140450
Dledger has some bugs and cannot realize disaster recovery. So I don't plan to choose Dledger yet.
------------------ 原始邮件 ------------------
发件人: "apache/rocketmq" ***@***.***>;
发送时间: 2021年11月4日(星期四) 晚上9:49
***@***.***>;
***@***.******@***.***>;
主题: Re: [apache/rocketmq] Can rocketmq have one master corresponding to two slaves? (Issue #3461)
version：4.9.1 If a broker sets SYNC_MASTER, corresponding to two slaves, can three nodes write synchronously?
Using the DLedger mode will ensure that more than half of the nodes will return immediately if synchronization is successful, so when there are 3 nodes, as long as there is a slave that is successfully synchronized, it will return.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] dongeforever commented on issue #3461: Can rocketmq have one master corresponding to two slaves?
Posted by GitBox <gi...@apache.org>.
dongeforever commented on issue #3461:
URL: https://github.com/apache/rocketmq/issues/3461#issuecomment-961662094
This is the cold data problem。
For in-sync slave, the master loads data from page cache.
But for out-of-sync slave, the master needs to load data from disk, this will be much slower than page cache.
If the incoming tps is large，the out of sync slave may never have the chance to catch up with the master.
IMO，We may need to polish the cold data sync performance.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] iamqq23ue commented on issue #3461: Can rocketmq have one master corresponding to two slaves?
Posted by GitBox <gi...@apache.org>.
iamqq23ue commented on issue #3461:
URL: https://github.com/apache/rocketmq/issues/3461#issuecomment-961563275
ISSUE#2582 still looks unresolved. You also can read my blog。
https://blog.csdn.net/qq23ue/article/details/112764507
------------------ 原始邮件 ------------------
发件人: "apache/rocketmq" ***@***.***>;
发送时间: 2021年11月4日(星期四) 晚上11:40
***@***.***>;
***@***.******@***.***>;
主题: Re: [apache/rocketmq] Can rocketmq have one master corresponding to two slaves? (Issue #3461)
Dledger has some bugs and cannot realize disaster recovery. So I don't plan to choose Dledger yet.
…
------------------ 原始邮件 ------------------ 发件人: "apache/rocketmq" @.>; 发送时间: 2021年11月4日(星期四) 晚上9:49 @.>; @.@.>; 主题: Re: [apache/rocketmq] Can rocketmq have one master corresponding to two slaves? (Issue #3461) version：4.9.1 If a broker sets SYNC_MASTER, corresponding to two slaves, can three nodes write synchronously? Using the DLedger mode will ensure that more than half of the nodes will return immediately if synchronization is successful, so when there are 3 nodes, as long as there is a slave that is successfully synchronized, it will return. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.
If you use HA mode, SYNC_MASTER will only ensure that a slava synchronization is completed, and it will return immediately.
In addition, can you describe in detail the problems encountered when using the DLedger?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] Git-Yang commented on issue #3461: Can rocketmq have one master corresponding to two slaves?
Posted by GitBox <gi...@apache.org>.
Git-Yang commented on issue #3461:
URL: https://github.com/apache/rocketmq/issues/3461#issuecomment-961644280
> ISSUE#2582 still looks unresolved. You also can read my blog。 https://blog.csdn.net/qq23ue/article/details/112764507
> […](#)
> ------------------ 原始邮件 ------------------ 发件人: "apache/rocketmq" ***@***.***>; 发送时间: 2021年11月4日(星期四) 晚上11:40 ***@***.***>; ***@***.******@***.***>; 主题: Re: [apache/rocketmq] Can rocketmq have one master corresponding to two slaves? (Issue #3461) Dledger has some bugs and cannot realize disaster recovery. So I don't plan to choose Dledger yet. … ------------------ 原始邮件 ------------------ 发件人: "apache/rocketmq" @.>; 发送时间: 2021年11月4日(星期四) 晚上9:49 @.>; @.@.>; 主题: Re: [apache/rocketmq] Can rocketmq have one master corresponding to two slaves? (Issue #3461) version：4.9.1 If a broker sets SYNC_MASTER, corresponding to two slaves, can three nodes write synchronously? Using the DLedger mode will ensure that more than half of the nodes will return immediately if synchronization is successful, so when there are 3 nodes, as long as there is a slav
e that is successfully synchronized, it will return. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. If you use HA mode, SYNC_MASTER will only ensure that a slava synchronization is completed, and it will return immediately. In addition, can you describe in detail the problems encountered when using the DLedger? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.
Okay, I will try to reproduce this problem.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] Git-Yang commented on issue #3461: Can rocketmq have one master corresponding to two slaves?
Posted by GitBox <gi...@apache.org>.
Git-Yang commented on issue #3461:
URL: https://github.com/apache/rocketmq/issues/3461#issuecomment-961644280
> ISSUE#2582 still looks unresolved. You also can read my blog。 https://blog.csdn.net/qq23ue/article/details/112764507
> […](#)
> ------------------ 原始邮件 ------------------ 发件人: "apache/rocketmq" ***@***.***>; 发送时间: 2021年11月4日(星期四) 晚上11:40 ***@***.***>; ***@***.******@***.***>; 主题: Re: [apache/rocketmq] Can rocketmq have one master corresponding to two slaves? (Issue #3461) Dledger has some bugs and cannot realize disaster recovery. So I don't plan to choose Dledger yet. … ------------------ 原始邮件 ------------------ 发件人: "apache/rocketmq" @.>; 发送时间: 2021年11月4日(星期四) 晚上9:49 @.>; @.@.>; 主题: Re: [apache/rocketmq] Can rocketmq have one master corresponding to two slaves? (Issue #3461) version：4.9.1 If a broker sets SYNC_MASTER, corresponding to two slaves, can three nodes write synchronously? Using the DLedger mode will ensure that more than half of the nodes will return immediately if synchronization is successful, so when there are 3 nodes, as long as there is a slav
e that is successfully synchronized, it will return. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. If you use HA mode, SYNC_MASTER will only ensure that a slava synchronization is completed, and it will return immediately. In addition, can you describe in detail the problems encountered when using the DLedger? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.
Okay, I will try to reproduce this problem.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] iamqq23ue commented on issue #3461: Can rocketmq have one master corresponding to two slaves?
Posted by GitBox <gi...@apache.org>.
iamqq23ue commented on issue #3461:
URL: https://github.com/apache/rocketmq/issues/3461#issuecomment-961801219
thanks for the reply。In addition, I suggest that if only one node exists in the deleger mode, this node can provide read services.
In this way, the requirements of disaster recovery can be met.
------------------ 原始邮件 ------------------
发件人: "apache/rocketmq" ***@***.***>;
发送时间: 2021年11月5日(星期五) 下午2:45
***@***.***>;
***@***.******@***.***>;
主题: Re: [apache/rocketmq] Can rocketmq have one master corresponding to two slaves? (Issue #3461)
This is the cold data problem。
For in-sync slave, the master loads data from page cache.
But for out-of-sync slave, the master needs to load data from disk, this will be much slower than page cache.
If the incoming tps is large，the out of sync slave may never have the chance to catch up with the master.
IMO，We may need to polish the cold data sync performance.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] Git-Yang commented on issue #3461: Can rocketmq have one master corresponding to two slaves?
Posted by GitBox <gi...@apache.org>.
Git-Yang commented on issue #3461:
URL: https://github.com/apache/rocketmq/issues/3461#issuecomment-961644280
> ISSUE#2582 still looks unresolved. You also can read my blog。 https://blog.csdn.net/qq23ue/article/details/112764507
> […](#)
> ------------------ 原始邮件 ------------------ 发件人: "apache/rocketmq" ***@***.***>; 发送时间: 2021年11月4日(星期四) 晚上11:40 ***@***.***>; ***@***.******@***.***>; 主题: Re: [apache/rocketmq] Can rocketmq have one master corresponding to two slaves? (Issue #3461) Dledger has some bugs and cannot realize disaster recovery. So I don't plan to choose Dledger yet. … ------------------ 原始邮件 ------------------ 发件人: "apache/rocketmq" @.>; 发送时间: 2021年11月4日(星期四) 晚上9:49 @.>; @.@.>; 主题: Re: [apache/rocketmq] Can rocketmq have one master corresponding to two slaves? (Issue #3461) version：4.9.1 If a broker sets SYNC_MASTER, corresponding to two slaves, can three nodes write synchronously? Using the DLedger mode will ensure that more than half of the nodes will return immediately if synchronization is successful, so when there are 3 nodes, as long as there is a slav
e that is successfully synchronized, it will return. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. If you use HA mode, SYNC_MASTER will only ensure that a slava synchronization is completed, and it will return immediately. In addition, can you describe in detail the problems encountered when using the DLedger? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.
Okay, I will try to reproduce this problem.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] Git-Yang commented on issue #3461: Can rocketmq have one master corresponding to two slaves?
Posted by GitBox <gi...@apache.org>.
Git-Yang commented on issue #3461:
URL: https://github.com/apache/rocketmq/issues/3461#issuecomment-961166167
> Dledger has some bugs and cannot realize disaster recovery. So I don't plan to choose Dledger yet.
> […](#)
> ------------------ 原始邮件 ------------------ 发件人: "apache/rocketmq" ***@***.***>; 发送时间: 2021年11月4日(星期四) 晚上9:49 ***@***.***>; ***@***.******@***.***>; 主题: Re: [apache/rocketmq] Can rocketmq have one master corresponding to two slaves? (Issue #3461) version：4.9.1 If a broker sets SYNC_MASTER, corresponding to two slaves, can three nodes write synchronously? Using the DLedger mode will ensure that more than half of the nodes will return immediately if synchronization is successful, so when there are 3 nodes, as long as there is a slave that is successfully synchronized, it will return. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.
If you use HA mode, SYNC_MASTER will only ensure that a slava synchronization is completed, and it will return immediately.
In addition, can you describe in detail the problems encountered when using the DLedger?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] dongeforever commented on issue #3461: Can rocketmq have one master corresponding to two slaves?
Posted by GitBox <gi...@apache.org>.
dongeforever commented on issue #3461:
URL: https://github.com/apache/rocketmq/issues/3461#issuecomment-961662094
This is the cold data problem。
For in-sync slave, the master loads data from page cache.
But for out-of-sync slave, the master needs to load data from disk, this will be much slower than page cache.
If the incoming tps is large，the out of sync slave may never have the chance to catch up with the master.
IMO，We may need to polish the cold data sync performance.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] vongosling closed issue #3461: Can rocketmq have one master corresponding to two slaves?
Posted by GitBox <gi...@apache.org>.
vongosling closed issue #3461:
URL: https://github.com/apache/rocketmq/issues/3461
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] dongeforever commented on issue #3461: Can rocketmq have one master corresponding to two slaves?
Posted by GitBox <gi...@apache.org>.
dongeforever commented on issue #3461:
URL: https://github.com/apache/rocketmq/issues/3461#issuecomment-962378450
The raft consensus considers consistency firstly。
In a 2n+1 group，the failure tolerance is n. If more than n nodes are failed, the left nodes may provide corrupted data.
Anyway, "Read from a slave" can be an option, both in "master-slave mode" and "dledger mode".
but need take care of some cases:
* the max offset in slave may be smaller than the currently committed offset, which may result in an out-of-range exception, then cause the committed offset to be reset.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org