[jira] [Resolved] (KAFKA-15777) Configurable remote fetch bytes per partition from Consumer

["Kamal Chandraprakash (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/KAFKA-15777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Kamal Chandraprakash resolved KAFKA-15777.
------------------------------------------
    Resolution: Won't Fix

> Configurable remote fetch bytes per partition from Consumer
> -----------------------------------------------------------
>
>                 Key: KAFKA-15777
>                 URL: https://issues.apache.org/jira/browse/KAFKA-15777
>             Project: Kafka
>          Issue Type: Task
>            Reporter: Kamal Chandraprakash
>            Assignee: Kamal Chandraprakash
>            Priority: Major
>              Labels: kip
>
> A consumer can configure the amount of local bytes to read from each partition in the FETCH request.
> {{max.fetch.bytes}} = 50 MB
> {{max.partition.fetch.bytes}} = 1 MB
> Similar to this, the consumer should be able to configure {{max.remote.partition.fetch.bytes}} = 4 MB.
> While handling the {{FETCH}} request, if we encounter a partition to read data from remote storage, then rest of the partitions in the request are ignored. Essentially, we are serving only 1 MB of remote data per FETCH request when all the partitions in the request are to be served from the remote storage.
> Providing one more configuration to the client help the user to tune the values depending on their storage plugin. The user might want to optimise the number of calls to remote storage vs amount of bytes returned back to the client in the FETCH response.
> [https://github.com/apache/kafka/blob/trunk/core/src/main/scala/kafka/server/ReplicaManager.scala#L1454]



--
This message was sent by Atlassian Jira
(v8.20.10#820010)