[jira] [Commented] (CASSANDRA-21462) Add byte for byte test harness to detect and resolve cursor deviations from iterator path

["Jon Haddad (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CASSANDRA-21462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18099853#comment-18099853 ] 

Jon Haddad commented on CASSANDRA-21462:
----------------------------------------

First patch to address the current cursor path.  No new types added, just fixes for what already exists.  Here's the summary:

*Data-correctness / corruption fixes:*


- Same-timestamp writes to the same cell resolved to the wrong value (tie-break was inverted relative to the reference reconciliation rules) — silent replica divergence
- Tombstone vs. expiring-cell tie-break was dead code (an ambiguous "isExpiring" check meant both sides always looked the same), letting an expiring cell wrongly win over a tombstone at the same timestamp
- Tombstone cells were serialized with a spurious "expiring" flag and a wasted TTL field
- Reversed (DESC) clustering columns compared empty vs. valued values in the wrong order, causing rows to merge out of order and corrupting stats bounds
- NO_DELETION_TIME live markers were misclassified as deleted, poisoning min-timestamp/tombstone stats
- Large (64+ column) table row encoding had two separate corruption bugs: a flipped mode-selection at the encoding boundary, and a wrong loop bound that silently dropped present columns sorted after the last missing one
- Index corruption for partitions larger than 2GiB (an index offset overflowed as a signed int)
- The final output index block excluded the end-of-partition marker byte, and partitions crossing the index-block-size threshold exactly once got no promoted index at all
- Dropped columns weren't filtered from old cell data — a column dropped and later re-added with the same name could resurrect stale data
- Static-row presence was derived from current table metadata instead of the actual input sstables, breaking after ALTER TABLE ... DROP of the last static column
- Row/static-row stats were over-counted for partitions with no static values (an empty placeholder row was counted when it shouldn't have been)
- Disabled bloom filters weren't tolerated by the index writer
- Accord-enabled tables were purging tombstones relative to the wrong time bound instead of gcBefore
- A shared file-rewriting utility (SSTableRewriter) didn't correctly retain original input files in one code path when asked to

*Performance (garbage-free) fixes:*

- Eliminated a per-row heap allocation when decoding sparse rows (rows missing some columns) under 64 columns
- Eliminated the same class of allocation for sparse rows in wide (64+ column) tables, which used a different, previously-unoptimized decode path

> Add byte for byte test harness to detect and resolve cursor deviations from iterator path
> -----------------------------------------------------------------------------------------
>
>                 Key: CASSANDRA-21462
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-21462
>             Project: Apache Cassandra
>          Issue Type: Sub-task
>          Components: Local/Compaction
>            Reporter: Jon Haddad
>            Assignee: Jon Haddad
>            Priority: High
>             Fix For: 6.0
>
>
> This issue addresses various deviations and inconsistencies with the cursor path by introducing tests that verify both the old iterator path and the new cursor path generate the same outputs given the same inputs.  It also fixes various issues found in the process of creating this test system that resulted in corruption or incorrect values being carried into the resulting SSTable.
> It adds a test that uses the JVM's internal memory tracking to assert that we do NOT do unnecessary allocations in the cursor path that should guard against future regressions, and fixes an unnecessary enum allocation.
> It does NOT add support for BTI, multi-cell columns, counters, etc.
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org