[jira] [Commented] (KAFKA-5201) Compacted topic could be misused to fill up a disk but deletion policy can't retain legitimate keys

["Michal Borowiecki (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-5201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16002311#comment-16002311 ] 

Michal Borowiecki commented on KAFKA-5201:
------------------------------------------

Hi Edoardo,
From the broker's perspective there is no concept of legitimate vs illegitimate keys AFAIK. What keys are legitimate depends on use-case, so should the broker really care about that?
If this is to prevent disk being filled up by producers you don't trust to produce legitimate keys, perhaps quota + compact,delete cleanup can guard against that? What do you think about that?

> Compacted topic could be misused to fill up a disk but deletion policy can't retain legitimate keys 
> ----------------------------------------------------------------------------------------------------
>
>                 Key: KAFKA-5201
>                 URL: https://issues.apache.org/jira/browse/KAFKA-5201
>             Project: Kafka
>          Issue Type: Improvement
>            Reporter: Edoardo Comar
>
> Misuse of a topic with cleanup policy = compact
> could lead to a disk being filled if a misbehaving producer keeps producing messages with unique keys.
> The mixed cleanup policy compact,delete could be adopted, but would not guarantee that the latest "legitimate" keys will be kept.
> It would be desirable to have a cleanup policy that attempts to preserve messages with 'legitimate' keys
> This issue needs a KIP but I have no proposed solution yet at the time of writing.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)