You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "Sven Ludwig (JIRA)" <ji...@apache.org> on 2017/07/12 13:09:00 UTC
[jira] [Commented] (KAFKA-1499) Broker-side compression
configuration
[ https://issues.apache.org/jira/browse/KAFKA-1499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16083947#comment-16083947 ]
Sven Ludwig commented on KAFKA-1499:
------------------------------------
I have a question regarding this contract.
Assume a broker in its lifetime always had compression.type snappy and contains a number of topics. At some point in time then, it is shutdown, configured to compression.type lz4 (no other changes than that) and started again, just as usual in the same Kafka cluster. What happens to the existing topics?
Following from the discussion above and what has been agreed, implemented and documented, nothing should change in existing topics at all, also not on compaction or replication. The existing topics should remain forever with a LogConfig compression.type snappy. Only new topics should be in lz4, and only on that particular broker.
Is this correct, or is there some room for messing up a topic (maybe through replication, or because other brokers in the cluster have a different default compression.type... eventually leading to a dreaded mixture of compressions)?
> Broker-side compression configuration
> -------------------------------------
>
> Key: KAFKA-1499
> URL: https://issues.apache.org/jira/browse/KAFKA-1499
> Project: Kafka
> Issue Type: New Feature
> Reporter: Joel Koshy
> Assignee: Manikumar
> Labels: newbie++
> Fix For: 0.9.0.0
>
> Attachments: KAFKA-1499_2014-08-15_14:20:27.patch, KAFKA-1499_2014-08-21_21:44:27.patch, KAFKA-1499_2014-09-21_15:57:23.patch, KAFKA-1499_2014-09-23_14:45:38.patch, KAFKA-1499_2014-09-24_14:20:33.patch, KAFKA-1499_2014-09-24_14:24:54.patch, KAFKA-1499_2014-09-25_11:05:57.patch, KAFKA-1499_2014-10-27_13:13:55.patch, KAFKA-1499_2014-12-16_22:39:10.patch, KAFKA-1499_2014-12-26_21:37:51.patch, KAFKA-1499.patch, KAFKA-1499.patch
>
> Original Estimate: 72h
> Remaining Estimate: 72h
>
> A given topic can have messages in mixed compression codecs. i.e., it can
> also have a mix of uncompressed/compressed messages.
> It will be useful to support a broker-side configuration to recompress
> messages to a specific compression codec. i.e., all messages (for all
> topics) on the broker will be compressed to this codec. We could have
> per-topic overrides as well.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)