You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@rocketmq.apache.org by GitBox <gi...@apache.org> on 2022/12/07 11:58:53 UTC

[GitHub] [rocketmq] ShannonDing opened a new issue, #5663: Messages may be lost when SyncStateSet expand in extreme scenarios

ShannonDing opened a new issue, #5663:
URL: https://github.com/apache/rocketmq/issues/5663

   **BUG REPORT**
   
   1. Please describe the issue you observed:
   
   - What did you do (The steps to reproduce)?
   
   当前 SyncStateSet 变更的时候，由Master Broker上报并且Controller成功应用后才会应用到本地，这保证了SyncStateSet收缩时不会有消息丢失。但是这无法保证SyncStateSet扩张时数据的一致性性，举一个例子
   
   A为Master，B为Slave，一开始SyncStatetSet为{A}，当B追上A时（slaveAckOffset > confirmOffset）, SyncStateSet将从{A}变成{A,B}，按照当前实现的逻辑，首先{A,B}要上报Controller成功应用后才会应用到本地，因此在上报Controller到Controller确认应用成功这段时间内，SyncStateSet仍然为{A}，此时发送过来的消息只需要A确认即可返回成功，假设Controller成功应用上报的信息SyncStateSet为{A，B}，但返回给A确认丢失，此时A下线，由于Controller中SyncStateSet为{A,B}，B被选为主，那么就可能会出现消息丢失。
   
   ![image](https://user-images.githubusercontent.com/16487356/206173740-124073e5-4d64-4c3f-abee-77df6301b568.png)
   
   
   
   - What is expected to see?
   
   - What did you see instead?
   
   2. Please tell us about your environment:
   
   3. Other information (e.g. detailed explanation, logs, related issues, suggestions on how to fix, etc):


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [rocketmq] lizhimins commented on issue #5663: Messages may be lost when SyncStateSet expand in extreme scenarios

Posted by GitBox <gi...@apache.org>.
lizhimins commented on issue #5663:
URL: https://github.com/apache/rocketmq/issues/5663#issuecomment-1346106845

   这个场景仅会发生在允许自动降级时。A觉得B宕了，A自己又宕了，本身已经超越了 SLA 的要求（2副本下副本全挂）
   
   本质上这里只是个选可用性还是选一致性的问题。目前的设计为了保证可用性所以允许写，在 A 重启成功后也可以通过回发把分叉的数据送给 B，类似于 container 中的 escape bridge 设计。如果要保证数据一致性，把 syncStateSet 收缩移动到 alert controller 成功之后就可以了，这样的话消息就会发送失败了。


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [rocketmq] hzh0425 commented on issue #5663: Messages may be lost when SyncStateSet expand in extreme scenarios

Posted by GitBox <gi...@apache.org>.
hzh0425 commented on issue #5663:
URL: https://github.com/apache/rocketmq/issues/5663#issuecomment-1364530180

   > 这个问题发生在syncStateSet扩张时刻，本地的syncStateSet和远程syncStateSet不一致的时候，如何决定ack数保证消息不丢。
   > 按照这个要求， 不管是syncStateSet的收缩或者扩张，
   
   是的, 我的思路是同步数据时新增一个状态判断是否正在同步新的 SyncStateSet 给 Controller (IsSynchronizing syncstateset), 是的话取本地和远程的 Syncstateset 合集.
   
   另一种方案是直接 Hold 住写请求, 等 syncstateset 修改完之后再继续, 但是性能有影响 @ShannonDing 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [rocketmq] ShannonDing commented on issue #5663: Messages may be lost when SyncStateSet expand in extreme scenarios

Posted by GitBox <gi...@apache.org>.
ShannonDing commented on issue #5663:
URL: https://github.com/apache/rocketmq/issues/5663#issuecomment-1350362435

   > 
   
   这个问题发生在syncStateSet扩张时刻，本地的syncStateSet和远程syncStateSet不一致的时候，如何决定ack数保证消息不丢。
   按照这个要求， 不管是syncStateSet的收缩或者扩张，都应该取本地与远程的合集，作为消息保存成功判断的依据，应该就可以解决消息丢失的问题。
   ![image](https://user-images.githubusercontent.com/16487356/207502329-48d7978f-9708-4591-b9db-8e314d6f1c02.png)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [rocketmq] RongtongJin closed issue #5663: Messages may be lost when SyncStateSet expand in extreme scenarios

Posted by "RongtongJin (via GitHub)" <gi...@apache.org>.
RongtongJin closed issue #5663: Messages may be lost when SyncStateSet expand in extreme scenarios
URL: https://github.com/apache/rocketmq/issues/5663


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org