You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@rocketmq.apache.org by "odbozhou (via GitHub)" <gi...@apache.org> on 2023/04/04 10:27:41 UTC

[GitHub] [rocketmq-connect] odbozhou opened a new issue, #456: Replicator support checkpoint

odbozhou opened a new issue, #456:
URL: https://github.com/apache/rocketmq-connect/issues/456

   The issue tracker is used for bug reporting purposes **ONLY** whereas feature request needs to follow the [RIP process](https://github.com/apache/rocketmq/wiki/RocketMQ-Improvement-Proposal). To avoid unnecessary duplication, please check whether there is a previous issue before filing a new one.
   
   It is recommended to start a discussion thread in the [mailing lists](http://rocketmq.apache.org/about/contact/) in cases of discussing your deployment plan, API clarification, and other non-bug-reporting issues.
   We welcome any friendly suggestions, bug fixes, collaboration, and other improvements.
   
   Please ensure that your bug report is clear and self-contained. Otherwise, it would take additional rounds of communication, thus more time, to understand the problem itself.
   
   Generally, fixing an issue goes through the following steps:
   1. Understand the issue reported;
   1. Reproduce the unexpected behavior locally;
   1. Perform root cause analysis to identify the underlying problem;
   1. Create test cases to cover the identified problem;
   1. Work out a solution to rectify the behavior and make the newly created test cases pass;
   1. Make a pull request and go through peer review;
   
   As a result, it would be very helpful yet challenging if you could provide an isolated project reproducing your reported issue. Anyway, please ensure your issue report is informative enough for the community to pick up. At a minimum, include the following hints:
   
   **BUG REPORT**
   
   1. Please describe the issue you observed:
   
   - What did you do (The steps to reproduce)?
   
   - What is expected to see?
   
   - What did you see instead?
   
   2. Please tell us about your environment:
   
   3. Other information (e.g. detailed explanation, logs, related issues, suggestions on how to fix, etc):
   
   **FEATURE REQUEST**
   
   1. Please describe the feature you are requesting.
   
   2. Provide any additional detail on your proposed use case for this feature.
   
   2. Indicate the importance of this issue to you (blocker, must-have, should-have, nice-to-have). Are you currently using any workarounds to address this issue?
   
   4. If there are some sub-tasks involved, use -[] for each sub-task and create a corresponding issue to map to the sub-task:
   
   - [sub-task1-issue-number](example_sub_issue1_link_here): sub-task1 description here, 
   - [sub-task2-issue-number](example_sub_issue2_link_here): sub-task2 description here,
   - ...
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Replicator support checkpoint [rocketmq-connect]

Posted by "frankcrc (via GitHub)" <gi...@apache.org>.
frankcrc commented on issue #456:
URL: https://github.com/apache/rocketmq-connect/issues/456#issuecomment-2568962825

   > @odbozhou Hi, I have two questions I'd like to consult about:
   > 
   > 1. Does this commit implement a synchronous checkpoint by merely writing the consumption progress of the consumer group (sync.gids) specified by the source MQ in the designated Topic to the replicator_checkpoint(default) topic of the connector MQ? This image (https://imgur.com/PmRcMaP) displays the content of one message, and I only see topic, consumerGroup, upstreamLastTimestamp, and downstreamLastTimestamp.
   > 2. How should I utilize this data to synchronize consumer progress? Is it by setting the consumer's consumeFromWhere to CONSUME_FROM_TIMESTAMP and using the latest upstreamLastTimestamp from the replicator_checkpoint?
   > 
   > 你好,有两个问题想请教,
   > 
   > 1. 这个commit实现的同步checkpoint,只是把源MQ指定的Topic的消费者组(sync.gids)的消费进度,写到中间的MQ的replicator_checkpoint(默认的) Topic吗?这个图片([https://imgur.com/PmRcMaP](url))展示了消息的内容,我只看到了topic、consumerGroup、upstreamLastTimestamp、downstreamLastTimestamp。
   > 2. 我应该如何利用这些数据来同步消费进度,是通过把消费者的consumeFromWherer配置为CONSUME_FROM_TIMESTAMP,并使用replicator_checkpoint中最新的upstreamLastTimestamp吗?
   
   or start a new consumer subscribing replicator_checkpoint to call DefaultMQAdminExt#resetOffsetByTimestamp?
   
   或者启一个新的消费者订阅replicator_checkpoint来调用DefaultMQAdminExt#resetOffsetByTimestamp?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Replicator support checkpoint [rocketmq-connect]

Posted by "frankcrc (via GitHub)" <gi...@apache.org>.
frankcrc commented on issue #456:
URL: https://github.com/apache/rocketmq-connect/issues/456#issuecomment-2568947092

   @odbozhou 
   Hi, I have two questions I'd like to consult about:
   
   1. Does this commit implement a synchronous checkpoint by merely writing the consumption progress of the consumer group (sync.gids) specified by the source MQ in the designated Topic to the replicator_checkpoint(default) topic of the connector MQ? This image (https://imgur.com/PmRcMaP) displays the content of one message, and I only see topic, consumerGroup, upstreamLastTimestamp, and downstreamLastTimestamp.
   
   2. How should I utilize this data to synchronize consumer progress? Is it by setting the consumer's consumeFromWhere to CONSUME_FROM_TIMESTAMP and using the latest upstreamLastTimestamp from the replicator_checkpoint?
   
   你好,有两个问题想请教,
   1.这个commit实现的同步checkpoint,只是把源MQ指定的Topic的消费者组(sync.gids)的消费进度,写到中间的MQ的replicator_checkpoint(默认的) Topic吗?这个图片([https://imgur.com/PmRcMaP](url))展示了消息的内容,我只看到了topic、consumerGroup、upstreamLastTimestamp、downstreamLastTimestamp。
   2.我应该如何利用这些数据来同步消费进度,是通过把消费者的consumeFromWherer配置为CONSUME_FROM_TIMESTAMP,并使用replicator_checkpoint中最新的upstreamLastTimestamp吗?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Replicator support checkpoint [rocketmq-connect]

Posted by "RockteMQ-AI (via GitHub)" <gi...@apache.org>.
RockteMQ-AI commented on issue #456:
URL: https://github.com/apache/rocketmq-connect/issues/456#issuecomment-5324837114

   **Issue Evaluation**
   
   Category: `enhancement`
   
   Checkpoint support in the replicator is essential for reliable cross-cluster replication with exactly-once or at-least-once semantics.
   
   ---
   *Automated evaluation by RockteMQ-AI*


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org