[jira] [Created] (KAFKA-7729) Mysterious disk increase when switching to RedHat OpenJDK

["Dimitris Mandalidis (JIRA)" <ji...@apache.org>]

Dimitris Mandalidis created KAFKA-7729:
------------------------------------------

             Summary: Mysterious disk increase when switching to RedHat OpenJDK
                 Key: KAFKA-7729
                 URL: https://issues.apache.org/jira/browse/KAFKA-7729
             Project: Kafka
          Issue Type: Bug
    Affects Versions: 1.1.0
            Reporter: Dimitris Mandalidis


(maybe it's not a bug actually, sort of a question)

We run a Kafka cluster 1.1.0 with 3 brokers running in RHEL 7.5 having their logs over an LVM ext4 partition (fsck'd an hour ago). The cluster is using Oracle JDK 8u144 and it uses 45G of disk space under normal circumstances.

We 've started switching to RedHat OpenJDK 1.8.0_191, one broker at a time. Since a broker had to be restarted, we thought that disk space increase was expected during rejoining the cluster, because it will first try to become ISR and it will roll segments (and delete if needed) afterwards (correct me if I 'm wrong). So the first OpenJDK broker ended up using 65G of disk space. For the second broker, because we will be running short of disk space, we decided to take him down and delete all the logs. So it also ended up using 65G of space.

We let the two brokers to operate as such for 8 hours expecting that they will eventually stabilize in the previous 45G but they didn't. The current situation is that we have 2 OpenJDK brokers holding 65G of disk usage, and one Oracle JDK broker which still holds 45G and waits to be migrated.

The actual question is that we cannot explain why we saw 20G disk space increase just switching to OpenJDK. We had performed rolling upgrades in the past (we 've started with 0.10) but we have never seen such a difference. Do you have any idea?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)