You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "Ismael Juma (JIRA)" <ji...@apache.org> on 2018/05/09 14:47:00 UTC
[jira] [Commented] (KAFKA-6828) Index files are no longer sparse in
Java 9/10 due to OpenJDK regression
[ https://issues.apache.org/jira/browse/KAFKA-6828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16468920#comment-16468920 ]
Ismael Juma commented on KAFKA-6828:
------------------------------------
[~eolivelli], can you please test the latest JDK 11 build ([http://jdk.java.net/11/)] as it supposedly has a fix for this?
> Index files are no longer sparse in Java 9/10 due to OpenJDK regression
> -----------------------------------------------------------------------
>
> Key: KAFKA-6828
> URL: https://issues.apache.org/jira/browse/KAFKA-6828
> Project: Kafka
> Issue Type: Improvement
> Components: core
> Affects Versions: 1.0.0
> Environment: CentosOS 7 on EXT4 FS
> Reporter: Enrico Olivelli
> Priority: Critical
>
> This is a very strage case. I have a Kafka broker (part of a cluster of 3 brokers) which cannot start upgrading Java from Oracle JDK8 to Oracle JDK 9.0.4 (the same with JDK 10.0.0)
> There are a lot of .index and .timeindex files taking 10MB, they are for empty partiions.
> Running with Java 9 the server seems to rebuild these files and each file takes "really" 10MB.The sum of all the files (calculated using du -sh) is 22GB and the broker crashes during startup, disk becomes full and no log more is written. (I can send an extraction of the logs, but the tell only about 'rebuilding index', the same as on Java 8)
> Reverting the same broker to Java 8 and removing the index files, the broker rebuilds such files, each files take 10MB, but the full sum of sizes (calculated using du -sh) is 38 MB !
> I am running this broker on CentosOS 7 on EXT4 FS.
> I have upgraded the broker to latest and greatest Kafka 1.0.0 (from 0.10.2) without any success.
>
> After checking on JDK nio-dev list it appears a regresion in the behaviour of RandomAccessFile
> Just for reference see this discussion on nio-dev list on OpenJDK
> [http://mail.openjdk.java.net/pipermail/nio-dev/2018-April/005008.html]
> see
> [https://bugs.openjdk.java.net/browse/JDK-8168628]
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)