You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "vasya b (Jira)" <ji...@apache.org> on 2026/03/23 14:31:00 UTC
[jira] [Commented] (CASSANDRA-21245) Uncompressed size is being used for compressed tables in maintenance operations
[ https://issues.apache.org/jira/browse/CASSANDRA-21245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18067681#comment-18067681 ]
vasya b commented on CASSANDRA-21245:
-------------------------------------
tested in 5.0.7 still reproducible
> Uncompressed size is being used for compressed tables in maintenance operations
> -------------------------------------------------------------------------------
>
> Key: CASSANDRA-21245
> URL: https://issues.apache.org/jira/browse/CASSANDRA-21245
> Project: Apache Cassandra
> Issue Type: Bug
> Reporter: vasya b
> Priority: Normal
> Attachments: cassandra.yaml, go.mod, main.go
>
>
> Using a compressed table can lead to a state where uncompressed table size is bigger
> than the whole volume size, while it shouldn't be a problem it leads to compaction poblems, as other compaction task are being denied.
> f.e. one can easily create a compressed table with
> {code:java}
> CREATE TABLE IF NOT EXISTS ` + keyspace + `.` + table + ` (
> pk bigint,
> data text,
> PRIMARY KEY (pk)
> ) WITH compression = {'class': 'DeflateCompressor'}
> {code}
> (or any other compression)
> Insert a bunch of data which is going to be compressed quite well (f.e. a bunch of zeros or any other single char * 1024 *1024) * N where N is some big number
> one would expect that such table will be compacted without any problems.
> but one will get:
> {code:java}
> WARN [CompactionExecutor:8] 2026-03-20 14:25:17,256 CompactionTask.java:434 - Not enough space for compaction (78c5c7c0-244f-11f1-9e8a-9937f6f9033a) of zerotohero.bulk_data, estimated sstables = 1, expected write size = 2686214
> ERROR [CompactionExecutor:8] 2026-03-20 14:25:17,256 JVMStabilityInspector.java:70 - Exception in thread Thread[CompactionExecutor:8,5,CompactionExecutor]
> java.lang.RuntimeException: Not enough space for compaction (78c5c7c0-244f-11f1-9e8a-9937f6f9033a) of zerotohero.bulk_data, estimated sstables = 1, expected write size = 2686214
> at org.apache.cassandra.db.compaction.CompactionTask.buildCompactionCandidatesForAvailableDiskSpace(CompactionTask.java:436)
> at org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:148)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:26)
> at org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:94)
> at org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:100)
> at org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:374)
> at org.apache.cassandra.concurrent.FutureTask$3.call(FutureTask.java:141)
> at org.apache.cassandra.concurrent.FutureTask.call(FutureTask.java:61)
> at org.apache.cassandra.concurrent.FutureTask.run(FutureTask.java:71)
> at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
> at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
> at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:840)
> WARN [CompactionExecutor:9] 2026-03-20 15:21:27,534 CompactionTask.java:440 - Not enough space for compaction 51834c70-2457-11f1-9e8a-9937f6f9033a, 71.31226MiB estimated. Reducing scope.
> DEBUG [CompactionExecutor:8] 2026-03-20 15:25:27,447 Directories.java:550 - FileStore /mnt/cassandra (/dev/sdb1) has 36221336371 bytes available, checking if we can write 95979501107 bytes
> WARN [CompactionExecutor:8] 2026-03-20 15:25:27,447 Directories.java:553 - FileStore /mnt/cassandra (/dev/sdb1) has only 33.73 GiB available, but 89.39 GiB is needed
> WARN [CompactionExecutor:8] 2026-03-20 15:25:27,447 CompactionTask.java:434 - Not enough space for compaction (e08281c0-2457-11f1-9e8a-9937f6f9033a) of zerotohero.bulk_data_zstd, estimated sstables = 1, expected write size = 2195528
> ERROR [CompactionExecutor:8] 2026-03-20 15:25:27,447 JVMStabilityInspector.java:70 - Exception in thread Thread[CompactionExecutor:8,5,CompactionExecutor]
> java.lang.RuntimeException: Not enough space for compaction (e08281c0-2457-11f1-9e8a-9937f6f9033a) of zerotohero.bulk_data_zstd, estimated sstables = 1, expected write size = 2195528
> at org.apache.cassandra.db.compaction.CompactionTask.buildCompactionCandidatesForAvailableDiskSpace(CompactionTask.java:436)
> at org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:148)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:26)
> at org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:94)
> at org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:100)
> at org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:374)
> at org.apache.cassandra.concurrent.FutureTask$3.call(FutureTask.java:141)
> at org.apache.cassandra.concurrent.FutureTask.call(FutureTask.java:61)
> at org.apache.cassandra.concurrent.FutureTask.run(FutureTask.java:71)
> at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
> at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
> at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:840)
> DEBUG [CompactionExecutor:8] 2026-03-20 15:25:27,447 HeapUtils.java:133 - Heap dump creation on uncaught exceptions is disabled.
> ERROR [CompactionExecutor:12] 2026-03-20 16:09:13,125 JVMStabilityInspector.java:70 - Exception in thread Thread[CompactionExecutor:12,5,CompactionExecutor]
> java.lang.RuntimeException: Not enough space for compaction (fd9ba330-245d-11f1-9e8a-9937f6f9033a) of zerotohero.bulk_data_deflate, estimated sstables = 1, expected write size = 2200455
> at org.apache.cassandra.db.compaction.CompactionTask.buildCompactionCandidatesForAvailableDiskSpace(CompactionTask.java:436)
> at org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:148)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:26)
> at org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:94)
> at org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:100)
> at org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:374)
> at org.apache.cassandra.concurrent.FutureTask$3.call(FutureTask.java:141)
> at org.apache.cassandra.concurrent.FutureTask.call(FutureTask.java:61)
> at org.apache.cassandra.concurrent.FutureTask.run(FutureTask.java:71)
> at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
> at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
> at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:840)
> DEBUG [CompactionExecutor:12] 2026-03-20 16:09:13,126 HeapUtils.java:133 - Heap dump creation on uncaught exceptions is disabled. {code}
> while there is enough space:
> {code:java}
> root@zerotohero:~# df -h
> Filesystem Size Used Avail Use% Mounted on
> /dev/sdb1 49G 4.5G 42G 10% /mnt/cassandra
> root@zerotohero:~# nodetool compactionstats -H
> concurrent compactors 2
> pending tasks 31
> zerotohero bulk_data 31
> compactions completed 5009
> data compacted 1.1 TiB
> compactions aborted 174
> compactions reduced 1
> sstables dropped from compaction 51
> 15 minute rate 5.33/minute
> mean rate 1226.87/hour
> compaction throughput (MiB/s) 64.0
> id compaction type keyspace table completed total unit progress
> c73b9e00-2451-11f1-9e8a-9937f6f9033a Compaction zerotohero bulk_data 58.16 GiB 136.33 GiB bytes 42.66%
> active compaction remaining time 0h20m50s {code}
> simple reproducer in go included
> cassandra version tested: 5.0.6
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org