You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2016/12/04 22:29:58 UTC
[jira] [Commented] (KAFKA-4451) Recovering empty replica yields
negative offsets in index of compact partitions
[ https://issues.apache.org/jira/browse/KAFKA-4451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15720715#comment-15720715 ]
ASF GitHub Bot commented on KAFKA-4451:
---------------------------------------
GitHub user michaelschiff opened a pull request:
https://github.com/apache/kafka/pull/2210
KAFKA-4451
Fix OffsetIndex overflow when replicating a highly compacted topic.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/michaelschiff/kafka bug/4451
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/kafka/pull/2210.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2210
----
commit d31ae87e47be6c00aca898c05fa5cf993ff127cc
Author: Michael Schiff <sc...@gmail.com>
Date: 2016-12-04T22:26:07Z
Adds a unit test case to LogTest that currently fails due to the
presence of KAFKA-4451
----
> Recovering empty replica yields negative offsets in index of compact partitions
> -------------------------------------------------------------------------------
>
> Key: KAFKA-4451
> URL: https://issues.apache.org/jira/browse/KAFKA-4451
> Project: Kafka
> Issue Type: Bug
> Affects Versions: 0.9.0.1
> Reporter: Michael Schiff
> Labels: reliability
>
> Bringing up an empty broker.
> the partition for a compact topic is not split into multiple log files. All data is written into a single log file, causing offsets to overflow.
> A dump of the affected broker shortly after it started replicating:
> {code}
> michael.schiff@stats-kafka09:~$ sudo /opt/kafka/bin/kafka-run-class.sh kafka.tools.DumpLogSegments --files /kafka/attainment_event-0/00000000000000000000.index | head -n 10
> Dumping /kafka/attainment_event-0/00000000000000000000.index
> offset: 1022071124 position: 1037612
> offset: -1713432120 position: 1348740
> offset: -886291423 position: 2397130
> offset: -644750126 position: 3445630
> offset: -57889876 position: 4493972
> offset: 433950099 position: 5388461
> offset: 1071769472 position: 6436837
> offset: 1746859069 position: 7485367
> offset: 2090359736 position: 8533822
> ...
> {code}
> and the dump of the same log file from the leader of this partition
> {code}
> michael.schiff@stats-kafka12:~$ sudo /opt/kafka/bin/kafka-run-class.sh kafka.tools.DumpLogSegments --files /kafka/attainment_event-0/00000000000000000000.index
> [sudo] password for michael.schiff:
> Dumping /kafka/attainment_event-0/00000000000000000000.index
> offset: 353690666 position: 262054
> offset: 633140428 position: 523785
> offset: 756537951 position: 785815
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)