You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "xiaochen-zhou (via GitHub)" <gi...@apache.org> on 2024/12/16 05:38:22 UTC
[PR] [FLINK-36906] Optimize the logic for determining if a split is finished [flink-connector-kafka]
xiaochen-zhou opened a new pull request, #141:
URL: https://github.com/apache/flink-connector-kafka/pull/141
When determining if a split is finished, process only the partitions with data from the current fetch instead of all partitions. This reduces unnecessary partition checks and improves performance and resource utilization.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-36906] Optimize the logic for determining if a split is finished [flink-connector-kafka]
Posted by "xiaochen-zhou (via GitHub)" <gi...@apache.org>.
xiaochen-zhou closed pull request #141: [FLINK-36906] Optimize the logic for determining if a split is finished
URL: https://github.com/apache/flink-connector-kafka/pull/141
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-36906] Optimize the logic for determining if a split is finished [flink-connector-kafka]
Posted by "xiaochen-zhou (via GitHub)" <gi...@apache.org>.
xiaochen-zhou commented on PR #141:
URL: https://github.com/apache/flink-connector-kafka/pull/141#issuecomment-2544655796
Friendly ping, do you have time to take a look @AHeise 🙏 ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-36906] Optimize the logic for determining if a split is finished [flink-connector-kafka]
Posted by "AHeise (via GitHub)" <gi...@apache.org>.
AHeise commented on PR #141:
URL: https://github.com/apache/flink-connector-kafka/pull/141#issuecomment-2544772849
Afaik this doesn't work and was the main reason for #100. If you last message is a transaction marker, then you would never check the stop condition on that partition at the point in time.
I'll trigger the CI which should fail for the test that was specifically added for that scenario.
I'll leave this PR open untli we figured out if it can indeed be improved. Please double check the linked PR and the respective ticket.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-36906] Optimize the logic for determining if a split is finished [flink-connector-kafka]
Posted by "xiaochen-zhou (via GitHub)" <gi...@apache.org>.
xiaochen-zhou commented on PR #141:
URL: https://github.com/apache/flink-connector-kafka/pull/141#issuecomment-2548084763
> Afaik this doesn't work and was the main reason for #100. If you last message is a transaction marker, then you would never check the stop condition on that partition at the point in time.
>
> I'll trigger the CI which should fail for the test that was specifically added for that scenario.
>
> I'll leave this PR open untli we figured out if it can indeed be improved. Please double check the linked PR and the respective ticket.
OK.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org