You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "sjmittal (via GitHub)" <gi...@apache.org> on 2025/06/06 04:31:51 UTC
[PR] [FLINK-37909-][cdc-connector-mongodb] Update MongoDBStreamFetchTask to renew heartbeatManager [flink-cdc]
sjmittal opened a new pull request, #4039:
URL: https://github.com/apache/flink-cdc/pull/4039
https://issues.apache.org/jira/browse/FLINK-37909
Update `heartbeatManager` so it has latest `changeStreamCursor`, when it changes in the running while loop.
Right now `heartbeatManager` is created with orginal `changeStreamCursor` and not renewed whenever `changeStreamCursor` is updated upon Change stream cursor expiry.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-37909-][cdc-connector-mongodb] Update MongoDBStreamFetchTask to renew heartbeatManager [flink-cdc]
Posted by "sjmittal (via GitHub)" <gi...@apache.org>.
sjmittal commented on PR #4039:
URL: https://github.com/apache/flink-cdc/pull/4039#issuecomment-2952650154
> There will be a patch release (3.4.1) later with bug-fixes soon. Could @sjmittal please backport it to `release-3.4` branch, too?
@yuxiqian Please find another PR to backport it to `release-3.4` branch.
https://github.com/apache/flink-cdc/pull/4040
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-37909-][cdc-connector-mongodb] Update MongoDBStreamFetchTask to renew heartbeatManager [flink-cdc]
Posted by "yuxiqian (via GitHub)" <gi...@apache.org>.
yuxiqian commented on PR #4039:
URL: https://github.com/apache/flink-cdc/pull/4039#issuecomment-2952628892
There will be a patch release (3.4.1) later with bug-fixes soon. Could @sjmittal please backport it to `release-3.4` branch, too?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-37909-][cdc-connector-mongodb] Update MongoDBStreamFetchTask to renew heartbeatManager [flink-cdc]
Posted by "yuxiqian (via GitHub)" <gi...@apache.org>.
yuxiqian merged PR #4039:
URL: https://github.com/apache/flink-cdc/pull/4039
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-37909-][cdc-connector-mongodb] Update MongoDBStreamFetchTask to renew heartbeatManager [flink-cdc]
Posted by "yuxiqian (via GitHub)" <gi...@apache.org>.
yuxiqian commented on PR #4039:
URL: https://github.com/apache/flink-cdc/pull/4039#issuecomment-2948953517
Thanks for @sjmittal's nice investigation, failing CI stages are irrelevant to this change.
Can we add a test case that simulates resume token expiration to verify this change?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-37909-][cdc-connector-mongodb] Update MongoDBStreamFetchTask to renew heartbeatManager [flink-cdc]
Posted by "sjmittal (via GitHub)" <gi...@apache.org>.
sjmittal commented on PR #4039:
URL: https://github.com/apache/flink-cdc/pull/4039#issuecomment-2949158111
> Thanks for @sjmittal's nice investigation, failing CI stages are irrelevant to this change.
>
> Can we add a test case that simulates resume token expiration to verify this change?
Currently there are no test cases for `MongoDBStreamFetchTask`. Also this problem usually surfaces after the streaming job is run for days. Not sure how to simulate that.
Looking at the code fix seems obvious that `changeStreamCursor` but `heartbeatManager` still uses the older one, which essentially causes the `ChangeStreamHistoryLost` exception on the mongodb side.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-37909-][cdc-connector-mongodb] Update MongoDBStreamFetchTask to renew heartbeatManager [flink-cdc]
Posted by "sjmittal (via GitHub)" <gi...@apache.org>.
sjmittal commented on PR #4039:
URL: https://github.com/apache/flink-cdc/pull/4039#issuecomment-2952568476
> Thanks for clarifying, this change looks reasonable. Will try to add test coverages later.
@yuxiqian Please let me know if anything else needs to be done to get this PR merged.
We would like this fix to be part of whatever next release we are doing for Flink CDC.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org