You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "nileshkumar3 (via GitHub)" <gi...@apache.org> on 2026/03/07 16:54:43 UTC
[PR] KAFKA-13630: Reduce deque lock scope in RecordAccumulator.drainBatchesForOneNode [kafka]
nileshkumar3 opened a new pull request, #21665:
URL: https://github.com/apache/kafka/pull/21665
This PR reduces lock hold time in RecordAccumulator.drainBatchesForOneNode by moving transactional producer state updates outside the synchronized deque block after the batch is removed.
The change keeps only deque-dependent checks and pollFirst() under the lock, which helps reduce producer-side contention without changing the existing drain logic.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-13630: Reduce deque lock scope in RecordAccumulator.drainBatchesForOneNode [kafka]
Posted by "nileshkumar3 (via GitHub)" <gi...@apache.org>.
nileshkumar3 commented on PR #21665:
URL: https://github.com/apache/kafka/pull/21665#issuecomment-4035697567
@yuyang08 Can you please review this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-13630: Reduce deque lock scope in RecordAccumulator.drainBatchesForOneNode [kafka]
Posted by "nileshkumar3 (via GitHub)" <gi...@apache.org>.
nileshkumar3 commented on PR #21665:
URL: https://github.com/apache/kafka/pull/21665#issuecomment-4016922915
@jasonk000 Can you review this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-13630: Reduce deque lock scope in RecordAccumulator.drainBatchesForOneNode [kafka]
Posted by "nileshkumar3 (via GitHub)" <gi...@apache.org>.
nileshkumar3 commented on PR #21665:
URL: https://github.com/apache/kafka/pull/21665#issuecomment-4024088391
@lucasbru Can you please review this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-13630: Reduce deque lock scope in RecordAccumulator.drainBatchesForOneNode [kafka]
Posted by "nileshkumar3 (via GitHub)" <gi...@apache.org>.
nileshkumar3 commented on PR #21665:
URL: https://github.com/apache/kafka/pull/21665#issuecomment-4016928358
@showuon can you please approve this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-13630: Reduce deque lock scope in RecordAccumulator.drainBatchesForOneNode [kafka]
Posted by "nileshkumar3 (via GitHub)" <gi...@apache.org>.
nileshkumar3 commented on PR #21665:
URL: https://github.com/apache/kafka/pull/21665#issuecomment-4063096032
KAFKA-13630 is scoped to ready() and drain(), and I don’t see a safe way to reduce the lock scope further in those paths.
In drain(), trunk already performs the heavier work (e.g., batch.close()) outside the deque lock while keeping the txn/idempotent bookkeeping inside the critical section. In ready(), the locking also already appears minimal after KAFKA-16226.
Given that, this PR does not appear to introduce a safe improvement over the current implementation. I’m closing the PR for now. If there is a specific remaining hotspot in ready() or drain() worth addressing, we can revisit with a targeted change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-13630: Reduce deque lock scope in RecordAccumulator.drainBatchesForOneNode [kafka]
Posted by "jasonk000 (via GitHub)" <gi...@apache.org>.
jasonk000 commented on PR #21665:
URL: https://github.com/apache/kafka/pull/21665#issuecomment-4016938365
I'm not a kafka committer nor familiar with this code anymore (it seems to have changed a lot since my last change to this code in 2022). thx.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-13630: Reduce deque lock scope in RecordAccumulator.drainBatchesForOneNode [kafka]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #21665:
URL: https://github.com/apache/kafka/pull/21665#issuecomment-4062150042
A label of 'needs-attention' was automatically added to this PR in order to raise the
attention of the committers. Once this issue has been triaged, the `triage` label
should be removed to prevent this automation from happening again.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-13630: Reduce deque lock scope in RecordAccumulator.drainBatchesForOneNode [kafka]
Posted by "nileshkumar3 (via GitHub)" <gi...@apache.org>.
nileshkumar3 closed pull request #21665: KAFKA-13630: Reduce deque lock scope in RecordAccumulator.drainBatchesForOneNode
URL: https://github.com/apache/kafka/pull/21665
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org