You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Stefan Miklosovic (Jira)" <ji...@apache.org> on 2026/03/02 16:02:00 UTC
[jira] [Commented] (CASSANDRA-21188) Race between compaction and dictionary compression training. Status stuck at SAMPLING. ExportImportListCompressionDictionaryTest hangs
[ https://issues.apache.org/jira/browse/CASSANDRA-21188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18062203#comment-18062203 ]
Stefan Miklosovic commented on CASSANDRA-21188:
-----------------------------------------------
https://pre-ci.cassandra.apache.org/job/cassandra/457/#showFailuresLink
> Race between compaction and dictionary compression training. Status stuck at SAMPLING. ExportImportListCompressionDictionaryTest hangs
> --------------------------------------------------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-21188
> URL: https://issues.apache.org/jira/browse/CASSANDRA-21188
> Project: Apache Cassandra
> Issue Type: Bug
> Components: Feature/Compression
> Reporter: Maxim Muzafarov
> Assignee: Stefan Miklosovic
> Priority: Normal
> Fix For: 5.x
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> There is a race beween compaction process and dictionary compression training start:
> # CompressionDictionaryManager: We collect all live sstables
> # ICompressionDictionaryTrainer: start a new traning
> # currentTrainingStatus moved to SAMPLING
> # All SSTables get compacted within concurrent compaction thread
> # SSTableSamplingTask: in the constructor sstable.tryRef return null
> # We run this task on a thread pool and it exists in cancelManualTraining
> # the currentTrainingStatus remains SAMPLING (should be *FAILED* or {*}COMPLETED!{*})
> ExportImportListCompressionDictionaryTest hangs for 10 minutes (configured constant) for now reason.
> The logs:
> {code}
> INFO [PerDiskMemtableFlushWriter_0:1] 2026-02-21T17:07:05,061 Flushing.java:157 - Writing Memtable-table_testexportingspecificdictionary_strateg_18@1268950324(61.523KiB serialized bytes, 1000 ops, 506.836KiB (0%) on-heap, 0B (0%) off-heap), flushed range = [min(-9223372036854775808), max(9223372036854775807))
> INFO [PerDiskMemtableFlushWriter_0:1] 2026-02-21T17:07:05,061 Flushing.java:197 - Completed flushing /Users/maxim.muzafarov/IdeaProjects/cassandra/build/test/cassandra/data/cql_test_keyspace/table_testexportingspecificdictionary_strateg_18-1b255f4def2540a60000000000000056/pa-11-big-Data.db (28.198KiB) for commitlog position CommitLogPosition(segmentId=1771693567140, position=654098), time spent: 0 ms, bytes flushed: 28875 / (rate: 28.198KiB/s), partitions flushed: 1000 / (rate: 1000/s), rows: 1000 / (rate: 1000/s), cpu time: 0 ms, heap allocated: 220.711KiB
> INFO [MemtableFlushWriter:1] 2026-02-21T17:07:05,084 LogTransaction.java:266 - Unfinished transaction log, deleting /Users/maxim.muzafarov/IdeaProjects/cassandra/build/test/cassandra/data/cql_test_keyspace/table_testexportingspecificdictionary_strateg_18-1b255f4def2540a60000000000000056/pa_txn_flush_bf3ac330-0f47-11f1-88d2-574197b4b378.log
> DEBUG [MemtableFlushWriter:1] 2026-02-21T17:07:05,087 ColumnFamilyStore.java:1416 - Flushed to [BigTableReader:big(path='/Users/maxim.muzafarov/IdeaProjects/cassandra/build/test/cassandra/data/cql_test_keyspace/table_testexportingspecificdictionary_strateg_18-1b255f4def2540a60000000000000056/pa-11-big-Data.db')] (1 sstables, 30.889KiB), biggest 30.889KiB, smallest 30.889KiB
> INFO [main] 2026-02-21T17:07:05,091 ColumnFamilyStore.java:1088 - Enqueuing flush of cql_test_keyspace.table_testexportingspecificdictionary_strateg_18, Reason: UNIT_TESTS, Usage: 506.836KiB (0%) on-heap, 0B (0%) off-heap
> INFO [PerDiskMemtableFlushWriter_0:2] 2026-02-21T17:07:05,092 Flushing.java:157 - Writing Memtable-table_testexportingspecificdictionary_strateg_18@957877902(61.523KiB serialized bytes, 1000 ops, 506.836KiB (0%) on-heap, 0B (0%) off-heap), flushed range = [min(-9223372036854775808), max(9223372036854775807))
> INFO [PerDiskMemtableFlushWriter_0:2] 2026-02-21T17:07:05,094 Flushing.java:197 - Completed flushing /Users/maxim.muzafarov/IdeaProjects/cassandra/build/test/cassandra/data/cql_test_keyspace/table_testexportingspecificdictionary_strateg_18-1b255f4def2540a60000000000000056/pa-12-big-Data.db (28.201KiB) for commitlog position CommitLogPosition(segmentId=1771693567140, position=726098), time spent: 0 ms, bytes flushed: 28878 / (rate: 28.201KiB/s), partitions flushed: 1000 / (rate: 1000/s), rows: 1000 / (rate: 1000/s), cpu time: 0 ms, heap allocated: 220.711KiB
> INFO [MemtableFlushWriter:2] 2026-02-21T17:07:05,112 LogTransaction.java:266 - Unfinished transaction log, deleting /Users/maxim.muzafarov/IdeaProjects/cassandra/build/test/cassandra/data/cql_test_keyspace/table_testexportingspecificdictionary_strateg_18-1b255f4def2540a60000000000000056/pa_txn_flush_bf3fa530-0f47-11f1-88d2-574197b4b378.log
> DEBUG [MemtableFlushWriter:2] 2026-02-21T17:07:05,116 ColumnFamilyStore.java:1416 - Flushed to [BigTableReader:big(path='/Users/maxim.muzafarov/IdeaProjects/cassandra/build/test/cassandra/data/cql_test_keyspace/table_testexportingspecificdictionary_strateg_18-1b255f4def2540a60000000000000056/pa-12-big-Data.db')] (1 sstables, 30.887KiB), biggest 30.887KiB, smallest 30.887KiB
> DEBUG [CompactionExecutor:2] 2026-02-21T17:07:05,117 Directories.java:554 - FileStore /System/Volumes/Data (/dev/disk3s5) has 593792975872 bytes available, checking if we can write 103847 bytes
> INFO [CompactionExecutor:2] 2026-02-21T17:07:05,117 CompactionTask.java:229 - Compacting (bf4375c0-0f47-11f1-88d2-574197b4b378) [/Users/maxim.muzafarov/IdeaProjects/cassandra/build/test/cassandra/data/cql_test_keyspace/table_testexportingspecificdictionary_strateg_18-1b255f4def2540a60000000000000056/pa-11-big-Data.db, /Users/maxim.muzafarov/IdeaProjects/cassandra/build/test/cassandra/data/cql_test_keyspace/table_testexportingspecificdictionary_strateg_18-1b255f4def2540a60000000000000056/pa-12-big-Data.db, /Users/maxim.muzafarov/IdeaProjects/cassandra/build/test/cassandra/data/cql_test_keyspace/table_testexportingspecificdictionary_strateg_18-1b255f4def2540a60000000000000056/pa-9-big-Data.db, /Users/maxim.muzafarov/IdeaProjects/cassandra/build/test/cassandra/data/cql_test_keyspace/table_testexportingspecificdictionary_strateg_18-1b255f4def2540a60000000000000056/pa-10-big-Data.db, ]
> DEBUG [CompactionExecutor:2] 2026-02-21T17:07:05,118 CursorCompactor.java:152 - Cursor compaction for table: table_testexportingspecificdictionary_strateg_18 keyspace: cql_test_keyspace is supported.
> INFO [RMI TCP Connection(26)-127.0.0.1] 2026-02-21T17:07:05,146 CommandInvokerService.java:185 - Executing command 'train' with execution ID: a5960218-7f58-41a2-a06f-d627acf20efd
> INFO [RMI TCP Connection(26)-127.0.0.1] 2026-02-21T17:07:05,147 CompressionDictionaryManager.java:237 - Starting SSTable-based training for cql_test_keyspace.table_testexportingspecificdictionary_strateg_18 with 1 SSTables
> INFO [RMI TCP Connection(26)-127.0.0.1] 2026-02-21T17:07:05,150 CompressionDictionaryScheduler.java:101 - Starting SSTable-based dictionary training for cql_test_keyspace.table_testexportingspecificdictionary_strateg_18 from 1 SSTables
> DEBUG [RMI TCP Connection(26)-127.0.0.1] 2026-02-21T17:07:05,150 CompressionDictionaryScheduler.java:198 - Couldn't acquire reference to SSTable /Users/maxim.muzafarov/IdeaProjects/cassandra/build/test/cassandra/data/cql_test_keyspace/table_testexportingspecificdictionary_strateg_18-1b255f4def2540a60000000000000056/pa-13-big. It may have been removed.
> WARN [NonPeriodicTasks:1] 2026-02-21T17:07:05,150 CompressionDictionaryScheduler.java:213 - No SSTables available for sampling in cql_test_keyspace.table_testexportingspecificdictionary_strateg_18
> INFO [RMI TCP Connection(26)-127.0.0.1] 2026-02-21T17:07:05,150 ToolRunner.java:927 - >>>> Polling training status...SAMPLING
> INFO [CompactionExecutor:2] 2026-02-21T17:07:05,152 CursorCompactor.java:1574 - Compaction ended bf4375c0-0f47-11f1-88d2-574197b4b378: { data bytes read = 294620, data bytes written = 297868, input (keys = [1:10000,] = 10000, rows = [1:10000,] = 10000, cells = [1:10000,] = 10000), output (keys = 10000, rows = 10000, cells = 10000)}
> INFO [CompactionExecutor:2] 2026-02-21T17:07:05,153 CompactionTask.java:336 - Compacted (bf4375c0-0f47-11f1-88d2-574197b4b378) 4 sstables to [build/test/cassandra/data/cql_test_keyspace/table_testexportingspecificdictionary_strateg_18-1b255f4def2540a60000000000000056/pa-13-big,] to level=0. 101.413KiB to 91.637KiB (~90% of original) in 35ms. Read Throughput = 2.826MiB/s, Write Throughput = 2.554MiB/s, Row Throughput = ~10,000/s. 10,000 total partitions merged to 10,000. Partition merge counts were {1:10000, }. Time spent writing keys = 10ms
> INFO [NonPeriodicTasks:1] 2026-02-21T17:07:05,153 BigFormat.java:324 - Deleting sstable: /Users/maxim.muzafarov/IdeaProjects/cassandra/build/test/cassandra/data/cql_test_keyspace/table_testexportingspecificdictionary_strateg_18-1b255f4def2540a60000000000000056/pa-11-big
> INFO [NonPeriodicTasks:1] 2026-02-21T17:07:05,154 BigFormat.java:324 - Deleting sstable: /Users/maxim.muzafarov/IdeaProjects/cassandra/build/test/cassandra/data/cql_test_keyspace/table_testexportingspecificdictionary_strateg_18-1b255f4def2540a60000000000000056/pa-12-big
> INFO [NonPeriodicTasks:1] 2026-02-21T17:07:05,154 BigFormat.java:324 - Deleting sstable: /Users/maxim.muzafarov/IdeaProjects/cassandra/build/test/cassandra/data/cql_test_keyspace/table_testexportingspecificdictionary_strateg_18-1b255f4def2540a60000000000000056/pa-9-big
> INFO [NonPeriodicTasks:1] 2026-02-21T17:07:05,155 BigFormat.java:324 - Deleting sstable: /Users/maxim.muzafarov/IdeaProjects/cassandra/build/test/cassandra/data/cql_test_keyspace/table_testexportingspecificdictionary_strateg_18-1b255f4def2540a60000000000000056/pa-10-big
> INFO [NonPeriodicTasks:1] 2026-02-21T17:07:05,155 LogTransaction.java:266 - Unfinished transaction log, deleting /Users/maxim.muzafarov/IdeaProjects/cassandra/build/test/cassandra/data/cql_test_keyspace/table_testexportingspecificdictionary_strateg_18-1b255f4def2540a60000000000000056/pa_txn_compaction_bf4375c0-0f47-11f1-88d2-574197b4b378.log
> INFO [RMI TCP Connection(26)-127.0.0.1] 2026-02-21T17:07:06,155 ToolRunner.java:927 - >>>> Polling training status...SAMPLING
> INFO [RMI TCP Connection(26)-127.0.0.1] 2026-02-21T17:07:07,159 ToolRunner.java:927 - >>>> Polling training status...SAMPLING
> INFO [RMI TCP Connection(26)-127.0.0.1] 2026-02-21T17:07:08,163 ToolRunner.java:927 - >>>> Polling training status...SAMPLING
> INFO [RMI TCP Connection(26)-127.0.0.1] 2026-02-21T17:07:09,168 ToolRunner.java:927 - >>>> Polling training status...SAMPLING
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org