You are viewing a plain text version of this content. The canonical link for it is here.
Posted to pr@cassandra.apache.org by "samueldlightfoot (via GitHub)" <gi...@apache.org> on 2026/05/17 09:15:39 UTC

[PR] Cassandra 21134 - Direct I/O Background Writes [cassandra]

samueldlightfoot opened a new pull request, #4814:
URL: https://github.com/apache/cassandra/pull/4814

   # CASSANDRA-21134: Direct I/O for background SSTable writes
   
   ## Summary
   
   Adds an opt-in `O_DIRECT` write path for background SSTable producers, bypassing the OS
   page cache for data that is unlikely to be re-read soon after being written. Memtable
   flush data stays buffered (it is hot and benefits from the page cache).
   
   Enabled via a new YAML knob:
   
   ```yaml
   background_write_disk_access_mode: direct    # default: standard
   direct_write_buffer_size: 256KiB              # aligned up to FS block size; auto-grows to chunk_length
   ```
   
   The path is gated by:
   
   1. config (`background_write_disk_access_mode == direct`),
   2. table compression being enabled (required — uncompressed writers still use the buffered
      path), and
   3. an `OperationType`-keyed allowlist (`DataComponent#DIRECT_WRITE_SUPPORT`).
   
   Selection happens centrally in `DataComponent.buildWriter`; producers are unchanged.
   
   ## Operations covered (DIO eligible)
   
   | OperationType         | Rationale                                  |
   | --------------------- | ------------------------------------------ |
   | `COMPACTION`          | append-only writer                         |
   | `MAJOR_COMPACTION`    | "                                          |
   | `TOMBSTONE_COMPACTION`| "                                          |
   | `ANTICOMPACTION`      | "                                          |
   | `GARBAGE_COLLECT`     | "                                          |
   | `CLEANUP`             | "                                          |
   | `UPGRADE_SSTABLES`    | "                                          |
   | `WRITE`               | "                                          |
   | `STREAM`              | chunked-receiver path (see ZCS exclusion)  |
   
   The allowlist is exhaustive: any new `OperationType` with `writesData == true` that is not
   classified will fail static initialization (`AssertionError`).
   
   ## Operations NOT covered
   
   | Path                          | Classification            | Reason                                                                                                  |
   | ----------------------------- | ------------------------- | ------------------------------------------------------------------------------------------------------- |
   | `FLUSH` (memtable flush)      | `UNSUPPORTED_POLICY`      | Just-flushed data is hot — keep it in the page cache. Memtable flushes always use buffered I/O.         |
   | `SCRUB`                       | `UNSUPPORTED_CORRECTNESS` | `tryAppend` needs `mark()` / `resetAndTruncate()`, which the DIO writer cannot satisfy.                 |
   | Zero-Copy Streaming (ZCS)     | n/a (path bypass)         | Entire-SSTable streaming does not go through `DataComponent.buildWriter`; the DIO gate never runs.      |
   | Uncompressed writers          | n/a (path bypass)         | Only `CompressedSequentialWriter` has a DIO subclass in this change.                                    |
   
   Removing a `UNSUPPORTED_CORRECTNESS` entry requires code changes; removing
   `UNSUPPORTED_POLICY` is a configuration / policy decision.
   
   ## Key code
   
   - `io/DirectIoSupport.java` — eligibility enum (`SUPPORTED` / `UNSUPPORTED_CORRECTNESS` /
     `UNSUPPORTED_POLICY` / `NOT_APPLICABLE`).
   - `io/sstable/format/DataComponent.java` — central selection + allowlist + exhaustiveness
     check; first activation per op is logged.
   - `io/compress/DirectCompressedSequentialWriter.java` — new writer; aligned buffers, no
     `mark()`/`resetAndTruncate()`.
   - `io/compress/CompressedSequentialWriter.java` — refactored to allow the DIO subclass to
     override the write chunk path; `writeChunk` contract documented and asserted.
   - `config/Config.java`, `config/DatabaseDescriptor.java` — new knobs, validation, and
     startup wiring; buffer size aligned to FS block size and auto-grown to chunk length.
   - `service/StartupChecks.java` — fails fast if `direct` is requested on a platform/FS that
     does not support `O_DIRECT`.
   
   ## Tests introduced
   
   - `DirectCompressedSequentialWriterTest` (unit, 818 lines) — covers the DIO writer in
     isolation: chunk-boundary alignment, buffer auto-expansion to chunk length, abort/close
     paths, checksum + compression-info component correctness, error handling.
   - `DataComponentDirectWriteSelectionTest` (unit) — verifies the selection matrix:
     per-OperationType eligibility, exhaustiveness assertion, compression-enabled gate,
     config-mode gate.
   - `StreamingDirectWriteTest` (in-JVM distributed) — proves chunked streaming
     (`CassandraStreamReader` / `CassandraCompressedStreamReader` →
     `BigTableWriter.openDataWriter` → `OperationType.STREAM`) selects the DIO writer when
     enabled; ZCS is disabled in the test since it bypasses the selection point.
   - `DirectIoTestUtils` — shared helpers (FS block size, alignment) for the suites above.
   - `AntiCompactionTest`, `CompactionsTest` — extended to exercise the DIO path end-to-end
     for the compaction operations in the allowlist.
   - `DatabaseDescriptorTest` — validation of the new knobs (mode parsing, buffer-size
     alignment, defaults).
   
   ## Not in scope
   
   - Direct I/O on the read path.
   - Uncompressed SSTable writers.
   - ZCS streaming.
   - Memtable flush.
   
   
   patch by Sam Lightfoot; reviewed by <Reviewers> for CASSANDRA21134
   
   
   ```
   
   The [Cassandra Jira](https://issues.apache.org/jira/browse/CASSANDRA-21134)
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] Cassandra 21134 - Direct I/O Background Writes [cassandra]

Posted by "samueldlightfoot (via GitHub)" <gi...@apache.org>.
samueldlightfoot closed pull request #4814: Cassandra 21134 - Direct I/O Background Writes
URL: https://github.com/apache/cassandra/pull/4814


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org