You are viewing a plain text version of this content. The canonical link for it is here.
Posted to pr@cassandra.apache.org by "samueldlightfoot (via GitHub)" <gi...@apache.org> on 2026/08/02 10:43:54 UTC
Re: [PR] Propagate trickle_fsync settings to compressed SSTable writers [cassandra]
samueldlightfoot commented on code in PR #4927:
URL: https://github.com/apache/cassandra/pull/4927#discussion_r3698735308
##########
test/unit/org/apache/cassandra/io/compress/CompressedSequentialWriterTest.java:
##########
@@ -120,6 +121,121 @@ public void testNoopWriter() throws IOException
runTests("Noop");
}
+ @Test
+ public void testWriterOptionsPropagatedToCompressedWriter() throws IOException
+ {
+ CompressionParams params = CompressionParams.lz4();
+ BufferType compressorBufferType = params.getSstableCompressor().preferredBufferType();
+ // caller values chosen to differ from what the writer derives, so an override is provable
+ BufferType callerBufferType = compressorBufferType == BufferType.ON_HEAP ? BufferType.OFF_HEAP
+ : BufferType.ON_HEAP;
+ SequentialWriterOption option = SequentialWriterOption.newBuilder()
+ .bufferSize(params.chunkLength() / 2)
+ .bufferType(callerBufferType)
+ .trickleFsync(true)
+ .trickleFsyncByteInterval(1 << 16)
+ .finishOnClose(true)
+ .build();
+ File f = FileUtils.createTempFile("writerOptionPropagated", "1");
+ File offsets = new File(f.path() + ".metadata");
+ MetadataCollector collector = new MetadataCollector(new ClusteringComparator(UTF8Type.instance));
+ try (ObservableCSW writer = new ObservableCSW(f, offsets, null, option, params, collector))
+ {
+ writer.write(new byte[64]); // give finishOnClose a non-empty file to finalize
+ SequentialWriterOption effective = writer.effectiveOption();
+
+ assertTrue(effective.trickleFsync());
+ assertEquals(1 << 16, effective.trickleFsyncByteInterval());
+ assertTrue(effective.finishOnClose());
+
+ // buffer sizing is derived from the compression layout, not the caller's values
+ assertEquals(params.chunkLength(), effective.bufferSize());
+ assertEquals(compressorBufferType, effective.bufferType());
+ }
+ finally
+ {
+ f.tryDelete();
+ offsets.tryDelete();
+ }
+ }
+
+ @Test
+ public void testTrickleFsyncFiresWhileWritingCompressedData() throws IOException
+ {
+ int chunk = 4096;
+ MetadataCollector collector = new MetadataCollector(new ClusteringComparator(UTF8Type.instance));
+
+ File on = FileUtils.createTempFile("trickleFsyncOn", "1");
+ File onOffsets = new File(on.path() + ".metadata");
+ SequentialWriterOption enabled = SequentialWriterOption.newBuilder()
+ .trickleFsync(true)
+ .trickleFsyncByteInterval(chunk * 4)
+ .build();
+ try (CountingCSW writer = new CountingCSW(on, onOffsets, null, enabled,
+ CompressionParams.lz4(chunk), collector))
+ {
+ writer.write(new byte[chunk * 20]);
+ assertTrue("trickle_fsync must fire during a compressed flush", writer.dataSyncs.get() > 0);
+ }
+ finally
+ {
+ on.tryDelete();
+ onOffsets.tryDelete();
+ }
+
+ File off = FileUtils.createTempFile("trickleFsyncOff", "1");
+ File offOffsets = new File(off.path() + ".metadata");
+ SequentialWriterOption disabled = SequentialWriterOption.newBuilder()
+ .trickleFsync(false)
+ .trickleFsyncByteInterval(chunk * 4)
+ .build();
+ try (CountingCSW writer = new CountingCSW(off, offOffsets, null, disabled,
+ CompressionParams.lz4(chunk), collector))
+ {
+ writer.write(new byte[chunk * 20]);
Review Comment:
It appears pretty straight-forward and worthwhile - I'll raise follow-up Jira.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org