[jira] [Created] (PHOENIX-7961) Secondary index diverges from data table after TTL expiry on partial-touch upserts

["Sanjeet Malhotra (Jira)" <ji...@apache.org>]

Sanjeet Malhotra created PHOENIX-7961:
-----------------------------------------

             Summary: Secondary index diverges from data table after TTL expiry on partial-touch upserts
                 Key: PHOENIX-7961
                 URL: https://issues.apache.org/jira/browse/PHOENIX-7961
             Project: Phoenix
          Issue Type: Bug
            Reporter: Sanjeet Malhotra
            Assignee: Sanjeet Malhotra


On a table or view with a literal TTL and a secondary index, an UPSERT that does not re-write any index-referenced column (a "partial touch") can resurrect logically-expired values into the index. Phoenix rebuilds the affected index entry from the current on-disk row, but that internal current-row read is not TTL-masked the way an ordinary client read is. Cells that are logically expired under the TTL but not yet physically removed by compaction get rebuilt into the index, while the data table correctly expires them on read. The data table and its secondary index then diverge.

There is a second facet at the exact TTL boundary. Even once the internal read is masked, \{{TTLRegionScanner}} derives its "current time" from the scan-open wall clock, whereas the index is rebuilt at \{{batchTimestamp}}, which \{{getBatchTimestamp}} bumps slightly forward to avoid timestamp collisions. A cell whose expiry boundary falls in the sub-millisecond \{{(scanOpenTime, batchTimestamp]}} window stays visible to the masked read, is rebuilt into the index, yet is expired on the data side -- the same divergence at the boundary.

Data-integrity / correctness bug: after TTL expiry the secondary index returns a value the data table no longer returns. No error is raised -- the divergence is silent and surfaces only after logical TTL expiry and before major compaction physically purges the expired cells.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)