You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Henrik Ingo (Jira)" <ji...@apache.org> on 2023/04/03 21:13:00 UTC
[jira] [Commented] (CASSANDRA-18260) Add details to Error message: Not enough space for compaction
[ https://issues.apache.org/jira/browse/CASSANDRA-18260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17708131#comment-17708131 ]
Henrik Ingo commented on CASSANDRA-18260:
-----------------------------------------
[~mck] I believe what you are seeing here it that the mock {{FakeFileStore}} class didn't implement a {{toString()}} method. CASSANDRA-18287 seems to have an example of what the same message looks like in production use.
> Add details to Error message: Not enough space for compaction
> --------------------------------------------------------------
>
> Key: CASSANDRA-18260
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18260
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Compaction, Observability/Logging
> Reporter: Brad Schoening
> Assignee: Henrik Ingo
> Priority: Normal
> Fix For: 4.0.x, 4.1.x, 5.x
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> When compaction fails, the log message should list a) the free space available on disk at that point in time and b) perhaps the number and/or size of the source sstables being compacted.
> Free space can change from one moment to the next, so when the below compaction failed because it needed 161GB, upon checking the server a few minutes later, it had 184GB free. Similarly, the error message mentions it was writing out one SSTable on this STCS table, but its not clear if it was combining X -> 1 tables, or something else.
> {noformat}
> [INFO ] [CompactionExecutor:77758] cluster_id=87 ip_address=127.1.1.1 CompactionTask.java:241 - Compacted (8a1cffe0-abb5-11ed-b3fc-8d2ac2c52f0d) 1 sstables to [...] to level=0. 86.997GiB to 86.997GiB (~99% of original) in 1,508,155ms. Read Throughput = 59.069MiB/s, Write Throughput = 59.069MiB/s, Row Throughput = ~20,283/s. 21,375 total partitions merged to 21,375. Partition merge counts were \{1:21375, }
> [ERROR] [CompactionExecutor:77774] cluster_id=87 ip_address=127.1.1.1 CassandraDaemon.java:581 - Exception in thread Thread[CompactionExecutor:77774,1,main]
> java.lang.RuntimeException: Not enough space for compaction, estimated sstables = 1, expected write size = 161228934093
> at org.apache.cassandra.db.compaction.CompactionTask.buildCompactionCandidatesForAvailableDiskSpace(CompactionTask.java:386)
> at org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:126)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:77)
> at org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:100)
> at org.apache.cassandra.db.compaction.CompactionManager$7.execute(CompactionManager.java:613)
> at org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:377)
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:834)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org