[GitHub] [rocketmq-externals] JerryTaoTao opened a new issue #671: [rocketmq-connector-flink] rebalance cause offset rollback to long time ago

[GitBox <gi...@apache.org>]

JerryTaoTao opened a new issue #671:
URL: https://github.com/apache/rocketmq-externals/issues/671


   Reproducible Steps：
   1.suppose there 3 queue for topic and consumer with two subTasks
   2.queue（0，1）consumer by subTask 1 and queue consumer by subTask 2
   3. after running for a long while.offset state such as [0:100,1:300,2:120]
   4. restart flink job with checkpoint
   5. subTask 1 recover with restoredOffsets[0:100,1:300,2:120] like subTask 2.
   6. suppose queue（0，1）still consumer by subTask 1 and queue consumer by subTask 2
   7. trigger rebalance one time,make subTask 2 consumer queue (0,1,2)
   8. running for a while,the offset will be change a lot by subTask 2 ,maye like [0:1000,1:3000,2:1200]
   9. subTask 1 offsetTable will be empty after snapshot
   10. trigger a rebalance, assign (0, 1) to subTask 1 again
   11. In subTask 1,the restored is true and offset table is empty and can't find queue(0, 1),so it will get from restoredOffsets[0:100,1:300,2:120],so will get wrong offset.
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org