You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "kamalcph (via GitHub)" <gi...@apache.org> on 2025/10/17 06:37:16 UTC
[PR] MINOR: Update the delayed remote fetch purgatory cleanup comment [kafka]
kamalcph opened a new pull request, #20721:
URL: https://github.com/apache/kafka/pull/20721
(no comment)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-19806: Hook to enable / disable multi-partition remote fetch feature [kafka]
Posted by "showuon (via GitHub)" <gi...@apache.org>.
showuon commented on code in PR #20721:
URL: https://github.com/apache/kafka/pull/20721#discussion_r2446827115
##########
core/src/main/scala/kafka/server/ReplicaManager.scala:
##########
@@ -1644,10 +1644,13 @@ class ReplicaManager(val config: KafkaConfig,
val remoteFetchTasks = new util.HashMap[TopicIdPartition, Future[Void]]
val remoteFetchResults = new util.HashMap[TopicIdPartition, CompletableFuture[RemoteLogReadResult]]
- remoteFetchInfos.forEach { (topicIdPartition, remoteFetchInfo) =>
+ // Hook to enable / disable multi-partition remote fetch feature.
+ val isMultiPartitionFetchEnabled = true
+ remoteFetchInfos.asScala.forall { case (topicIdPartition, remoteFetchInfo) =>
val (task, result) = processRemoteFetch(remoteFetchInfo)
remoteFetchTasks.put(topicIdPartition, task)
remoteFetchResults.put(topicIdPartition, result)
+ isMultiPartitionFetchEnabled
Review Comment:
Could we add this change after the config is introduced?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Update the delayed remote fetch purgatory cleanup comment [kafka]
Posted by "kamalcph (via GitHub)" <gi...@apache.org>.
kamalcph commented on code in PR #20721:
URL: https://github.com/apache/kafka/pull/20721#discussion_r2449357022
##########
core/src/main/scala/kafka/server/DelayedRemoteFetch.scala:
##########
@@ -109,7 +109,7 @@ class DelayedRemoteFetch(remoteFetchTasks: util.Map[TopicIdPartition, Future[Voi
override def onComplete(): Unit = {
val fetchPartitionData = localReadResults.map { case (tp, result) =>
val remoteFetchResult = remoteFetchResults.get(tp)
- if (remoteFetchInfos.containsKey(tp)
+ if (remoteFetchResults.containsKey(tp)
Review Comment:
This change is minor one and required when enabling single-partition fetch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Update the delayed remote fetch purgatory cleanup comment [kafka]
Posted by "kamalcph (via GitHub)" <gi...@apache.org>.
kamalcph merged PR #20721:
URL: https://github.com/apache/kafka/pull/20721
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Update the delayed remote fetch purgatory cleanup comment [kafka]
Posted by "kamalcph (via GitHub)" <gi...@apache.org>.
kamalcph commented on code in PR #20721:
URL: https://github.com/apache/kafka/pull/20721#discussion_r2449353555
##########
core/src/main/scala/kafka/server/ReplicaManager.scala:
##########
@@ -1644,10 +1644,13 @@ class ReplicaManager(val config: KafkaConfig,
val remoteFetchTasks = new util.HashMap[TopicIdPartition, Future[Void]]
val remoteFetchResults = new util.HashMap[TopicIdPartition, CompletableFuture[RemoteLogReadResult]]
- remoteFetchInfos.forEach { (topicIdPartition, remoteFetchInfo) =>
+ // Hook to enable / disable multi-partition remote fetch feature.
+ val isMultiPartitionFetchEnabled = true
+ remoteFetchInfos.asScala.forall { case (topicIdPartition, remoteFetchInfo) =>
val (task, result) = processRemoteFetch(remoteFetchInfo)
remoteFetchTasks.put(topicIdPartition, task)
remoteFetchResults.put(topicIdPartition, result)
+ isMultiPartitionFetchEnabled
Review Comment:
ack, removed the change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-19806: Hook to enable / disable multi-partition remote fetch feature [kafka]
Posted by "kamalcph (via GitHub)" <gi...@apache.org>.
kamalcph commented on PR #20721:
URL: https://github.com/apache/kafka/pull/20721#issuecomment-3414523126
@showuon @satishd
Added a hook to enable / disable the multi-partition remote fetch feature. The broker config 'remote.multi.partition.fetch.enable' can be introduced later. Could you please review the PR? Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-19806: Hook to enable / disable multi-partition remote fetch feature [kafka]
Posted by "showuon (via GitHub)" <gi...@apache.org>.
showuon commented on code in PR #20721:
URL: https://github.com/apache/kafka/pull/20721#discussion_r2446831410
##########
core/src/main/scala/kafka/server/ReplicaManager.scala:
##########
@@ -1740,8 +1745,6 @@ class ReplicaManager(val config: KafkaConfig,
// try to complete the request immediately, otherwise put it into the purgatory;
// this is because while the delayed fetch operation is being created, new requests
// may arrive and hence make this operation completable.
- // We only guarantee eventual cleanup via the next FETCH request for the same set of partitions or
- // using reaper-thread.
Review Comment:
Ah, wrong place. Nice find!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Update the delayed remote fetch purgatory cleanup comment [kafka]
Posted by "kamalcph (via GitHub)" <gi...@apache.org>.
kamalcph commented on code in PR #20721:
URL: https://github.com/apache/kafka/pull/20721#discussion_r2449357022
##########
core/src/main/scala/kafka/server/DelayedRemoteFetch.scala:
##########
@@ -109,7 +109,7 @@ class DelayedRemoteFetch(remoteFetchTasks: util.Map[TopicIdPartition, Future[Voi
override def onComplete(): Unit = {
val fetchPartitionData = localReadResults.map { case (tp, result) =>
val remoteFetchResult = remoteFetchResults.get(tp)
- if (remoteFetchInfos.containsKey(tp)
+ if (remoteFetchResults.containsKey(tp)
Review Comment:
This change is minor one and required when enabling single-partition fetch. Let me know if you want to take this up in the next PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org