You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2016/10/13 05:45:20 UTC
[jira] [Commented] (KAFKA-4298) LogCleaner does not convert
compressed message sets properly
[ https://issues.apache.org/jira/browse/KAFKA-4298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15570935#comment-15570935 ]
ASF GitHub Bot commented on KAFKA-4298:
---------------------------------------
GitHub user hachikuji opened a pull request:
https://github.com/apache/kafka/pull/2019
KAFKA-4298: Ensure compressed message sets are converted when cleaning the log
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/hachikuji/kafka KAFKA-4298
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/kafka/pull/2019.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2019
----
commit af3b31b4d94ece1603ac470bfd8d781558987501
Author: Jason Gustafson <ja...@confluent.io>
Date: 2016-10-13T04:56:52Z
KAFKA-4298: Ensure compressed message sets are converted when cleaning the log
----
> LogCleaner does not convert compressed message sets properly
> ------------------------------------------------------------
>
> Key: KAFKA-4298
> URL: https://issues.apache.org/jira/browse/KAFKA-4298
> Project: Kafka
> Issue Type: Bug
> Reporter: Jason Gustafson
> Assignee: Jason Gustafson
> Priority: Critical
> Fix For: 0.10.1.1
>
>
> When cleaning the log, we attempt to write the cleaned messages using the message format configured for the topic, but as far as I can tell, we do not convert the wrapped messages in compressed message sets. For example, if there is an old compressed message set with magic=0 in the log and the topic is configured for magic=1, then after cleaning, the new message set will have a wrapper with magic=1, but the nested messages will still have magic=0. If this happens, there does not seem to be an easy way to recover without manually fixing up the log.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)