You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@ignite.apache.org by "alex-plekhanov (via GitHub)" <gi...@apache.org> on 2026/09/07 16:39:50 UTC

[PR] IGNITE-29028 Implement multi-page in-place row update [ignite]

alex-plekhanov opened a new pull request, #13560:
URL: https://github.com/apache/ignite/pull/13560

   Thank you for submitting the pull request to the Apache Ignite.
   
   In order to streamline the review of the contribution 
   we ask you to ensure the following steps have been taken:
   
   ### The Contribution Checklist
   - [ ] There is a single JIRA ticket related to the pull request. 
   - [ ] The web-link to the pull request is attached to the JIRA ticket.
   - [ ] The JIRA ticket has the _Patch Available_ state.
   - [ ] The pull request body describes changes that have been made. 
   The description explains _WHAT_ and _WHY_ was made instead of _HOW_.
   - [ ] The pull request title is treated as the final commit message. 
   The following pattern must be used: `IGNITE-XXXX Change summary` where `XXXX` - number of JIRA issue.
   - [ ] A reviewer has been mentioned through the JIRA comments 
   (see [the Maintainers list](https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute#HowtoContribute-ReviewProcessandMaintainers)) 
   - [ ] The pull request has been checked by the Teamcity Bot and 
   the `green visa` attached to the JIRA ticket (see tab `PR Check` at [TC.Bot - Instance 1](https://tcbot2.sbt-ignite-dev.ru/prs.html) or [TC.Bot - Instance 2](https://mtcga.gridgain.com/prs.html))
   
   ### Notes
   - [How to Contribute](https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute)
   - [Coding abbreviation rules](https://cwiki.apache.org/confluence/display/IGNITE/Abbreviation+Rules)
   - [Coding Guidelines](https://cwiki.apache.org/confluence/display/IGNITE/Coding+Guidelines)
   - [Apache Ignite Teamcity Bot](https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+Teamcity+Bot)
   
   If you need any help, please email dev@ignite.apache.org or ask anу advice on http://asf.slack.com _#ignite_ channel.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29028 Implement multi-page in-place row update [ignite]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #13560:
URL: https://github.com/apache/ignite/pull/13560#issuecomment-5589611425

   <!-- ignite-rolling-upgrade-check -->
   ## Possible compatibility issues. Please, check rolling upgrade cases
   
   This PR modifies protected classes (with **Order** annotation).
   Changes to these classes can break rolling upgrade compatibility.
   
   **Affected files:**
   - `modules/core/src/main/java/org/apache/ignite/internal/processors/rollingupgrade/feature/IgniteNodeFeatureSet.java`
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29028 Implement multi-page in-place row update [ignite]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #13560:
URL: https://github.com/apache/ignite/pull/13560#issuecomment-5710767374

   <!-- ignite-rolling-upgrade-check -->
   ## Possible compatibility issues. Please, check rolling upgrade cases
   
   This PR modifies protected classes (with **Order** annotation).
   Changes to these classes can break rolling upgrade compatibility.
   
   **Affected files:**
   - `modules/core/src/main/java/org/apache/ignite/internal/processors/rollingupgrade/feature/IgniteNodeFeatureSet.java`
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29028 Implement multi-page in-place row update [ignite]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #13560:
URL: https://github.com/apache/ignite/pull/13560#issuecomment-5574417273

   <!-- ignite-rolling-upgrade-check -->
   ## Possible compatibility issues. Please, check rolling upgrade cases
   
   This PR modifies protected classes (with **Order** annotation).
   Changes to these classes can break rolling upgrade compatibility.
   
   **Affected files:**
   - `modules/core/src/main/java/org/apache/ignite/internal/processors/rollingupgrade/feature/IgniteNodeFeatureSet.java`
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29028 Implement multi-page in-place row update [ignite]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #13560:
URL: https://github.com/apache/ignite/pull/13560#issuecomment-5573675693

   <!-- ignite-rolling-upgrade-check -->
   ## Possible compatibility issues. Please, check rolling upgrade cases
   
   This PR modifies protected classes (with **Order** annotation).
   Changes to these classes can break rolling upgrade compatibility.
   
   **Affected files:**
   - `modules/core/src/main/java/org/apache/ignite/internal/processors/rollingupgrade/feature/IgniteNodeFeatureSet.java`
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29028 Implement multi-page in-place row update [ignite]

Posted by "asf-gitbox-commits (via GitHub)" <gi...@apache.org>.
asf-gitbox-commits closed pull request #13560: IGNITE-29028 Implement multi-page in-place row update
URL: https://github.com/apache/ignite/pull/13560


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29028 Implement multi-page in-place row update [ignite]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #13560:
URL: https://github.com/apache/ignite/pull/13560#issuecomment-5581107687

   <!-- ignite-rolling-upgrade-check -->
   ## Possible compatibility issues. Please, check rolling upgrade cases
   
   This PR modifies protected classes (with **Order** annotation).
   Changes to these classes can break rolling upgrade compatibility.
   
   **Affected files:**
   - `modules/core/src/main/java/org/apache/ignite/internal/processors/rollingupgrade/feature/IgniteNodeFeatureSet.java`
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29028 Implement multi-page in-place row update [ignite]

Posted by "alex-plekhanov (via GitHub)" <gi...@apache.org>.
alex-plekhanov commented on PR #13560:
URL: https://github.com/apache/ignite/pull/13560#issuecomment-5668893895

   1. Need more time to investigate, will return later.
   2. Fixed.
   3. Fixed.
   4. Data page scans is broken by design. It has conceptual problems and can't be fixed without compatibility issues. So it's disabled now and no way to enable it. In case it will be fixed someday I have left a comment in https://issues.apache.org/jira/browse/IGNITE-11998 to disable multi-page in-place updates when data page scans is enabled.
   5. Fixed.
   6. storeCacheIdInDataPage - I think it's covered by other tests (since this functionality works as is, no changes here, only refactoring). pageRecordsDisabled - what is expected to be tested here? Ok, I will try to write a test and test something.  Test for features during rolling upgrade - also not a part of this change. I think it's covered (or at least should be) by other tests related to rolling upgrade. Concurrent scan vs. in-place update - ok, will write this one.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29028 Implement multi-page in-place row update [ignite]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #13560:
URL: https://github.com/apache/ignite/pull/13560#issuecomment-5668897550

   <!-- ignite-rolling-upgrade-check -->
   ## Possible compatibility issues. Please, check rolling upgrade cases
   
   This PR modifies protected classes (with **Order** annotation).
   Changes to these classes can break rolling upgrade compatibility.
   
   **Affected files:**
   - `modules/core/src/main/java/org/apache/ignite/internal/processors/rollingupgrade/feature/IgniteNodeFeatureSet.java`
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29028 Implement multi-page in-place row update [ignite]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #13560:
URL: https://github.com/apache/ignite/pull/13560#issuecomment-5585712319

   <!-- ignite-rolling-upgrade-check -->
   ## Possible compatibility issues. Please, check rolling upgrade cases
   
   This PR modifies protected classes (with **Order** annotation).
   Changes to these classes can break rolling upgrade compatibility.
   
   **Affected files:**
   - `modules/core/src/main/java/org/apache/ignite/internal/processors/rollingupgrade/feature/IgniteNodeFeatureSet.java`
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29028 Implement multi-page in-place row update [ignite]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #13560:
URL: https://github.com/apache/ignite/pull/13560#issuecomment-5630915373

   <!-- ignite-rolling-upgrade-check -->
   ## Possible compatibility issues. Please, check rolling upgrade cases
   
   This PR modifies protected classes (with **Order** annotation).
   Changes to these classes can break rolling upgrade compatibility.
   
   **Affected files:**
   - `modules/core/src/main/java/org/apache/ignite/internal/processors/rollingupgrade/feature/IgniteNodeFeatureSet.java`
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29028 Implement multi-page in-place row update [ignite]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #13560:
URL: https://github.com/apache/ignite/pull/13560#issuecomment-5941392363

   <!-- ignite-rolling-upgrade-check -->
   ## Possible compatibility issues. Please, check rolling upgrade cases
   
   This PR modifies protected classes (with **Order** annotation).
   Changes to these classes can break rolling upgrade compatibility.
   
   **Affected files:**
   - `modules/core/src/main/java/org/apache/ignite/internal/processors/rollingupgrade/feature/IgniteNodeFeatureSet.java`
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29028 Implement multi-page in-place row update [ignite]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #13560:
URL: https://github.com/apache/ignite/pull/13560#issuecomment-5608305304

   <!-- ignite-rolling-upgrade-check -->
   ## Possible compatibility issues. Please, check rolling upgrade cases
   
   This PR modifies protected classes (with **Order** annotation).
   Changes to these classes can break rolling upgrade compatibility.
   
   **Affected files:**
   - `modules/core/src/main/java/org/apache/ignite/internal/processors/rollingupgrade/feature/IgniteNodeFeatureSet.java`
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29028 Implement multi-page in-place row update [ignite]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #13560:
URL: https://github.com/apache/ignite/pull/13560#issuecomment-5630576955

   <!-- ignite-rolling-upgrade-check -->
   ## Possible compatibility issues. Please, check rolling upgrade cases
   
   This PR modifies protected classes (with **Order** annotation).
   Changes to these classes can break rolling upgrade compatibility.
   
   **Affected files:**
   - `modules/core/src/main/java/org/apache/ignite/internal/processors/rollingupgrade/feature/IgniteNodeFeatureSet.java`
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29028 Implement multi-page in-place row update [ignite]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #13560:
URL: https://github.com/apache/ignite/pull/13560#issuecomment-5676232973

   <!-- ignite-rolling-upgrade-check -->
   ## Possible compatibility issues. Please, check rolling upgrade cases
   
   This PR modifies protected classes (with **Order** annotation).
   Changes to these classes can break rolling upgrade compatibility.
   
   **Affected files:**
   - `modules/core/src/main/java/org/apache/ignite/internal/processors/rollingupgrade/feature/IgniteNodeFeatureSet.java`
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29028 Implement multi-page in-place row update [ignite]

Posted by "nizhikov (via GitHub)" <gi...@apache.org>.
nizhikov commented on PR #13560:
URL: https://github.com/apache/ignite/pull/13560#issuecomment-5661364383

   Review of the change (head 936fcdbc6f9). The core mechanism (fragment-by-fragment in-place update with compare-before-write, feature-flag gated) looks sound and is well tested, but there is one reproducible regression and a broken test assertion that should be addressed before merge.
   
   ### Blocking
   
   **1. Regression in `IgnitePdsWithTtlTest.testConcurrentPutOpsToCacheWithExpirationCompleteSuccesfully`.**
   Local runs of the full test class:
   
   | Branch | Runs | Failures |
   |---|---|---|
   | PR head 936fcdbc6f9 | 4 | 3 |
   | merge-base d9e785f77fa | 3 | 0 |
   
   Every failure is a `CorruptedTreeException` in the pending tree during `purgeExpired`: a `PendingRow` points to a link whose item was already freed (`AssertionError: itemId=2, directCnt=2` in `CacheDataRowAdapter.doInitFromLink`, called from `PendingEntriesTree.getRow` / `BPlusTree$RemoveRange`). All failures were on the transactional caches (`expirable-cache-tx`, `expirable-cache-near-tx`) with 1 KB entries, so the multi-page code itself is not involved.
   
   The suspect is the change in `GridCacheMapEntry.UpdateClosure.call` (`GridCacheMapEntry.java:4379`): `this.oldRow` is now assigned before `checkRowExpired`, so an expired old row is handed to `finishUpdate`, which now removes its pending entry and frees its link. On master the tx path passed `null` plus `oldRowExpiredFlag`, so the expired row was left in place. I could not pin the exact interleaving by inspection, but the empirical signal is strong. Suggest splitting this change out (or reverting it here) and root-causing it under a separate ticket. Master's behavior looks like a row leak, so a fix is worthwhile, but it should not go in silently with this PR. This also needs a dedicated test.
   
   **2. Broken assertions in the new test.**
   `MultiPageInPlaceUpdateTest` uses `assertNotSame(link, link(ignite, key))` (lines 129 and 302). Both arguments are `long`, autoboxed into distinct `Long` objects, so the assertion always passes. Use `assertNotEquals`. The comment on line 300 claims links change after logical recovery; that is currently unverified.
   
   ### Should fix
   
   **3. `storeCacheId()` flag is set on only one init path.**
   `CacheDataRowAdapter` sets `FLAG_STORE_CACHE_ID` in `doInitFromLink` (line 303) but not in `initFromDataPage` or `initFromPageBuffer`. Rows produced by the data-page scan cursor (`CacheDataTree.java:261`) or the snapshot dump reader report `storeCacheId() == false` and `size()` four bytes short whenever the cache id is stored in pages. Nothing on the update path consumes those rows today, but the method contract is wrong. Set the flag wherever `readCacheId` is consumed (e.g. in `readIncomplete`).
   
   **4. Safety argument in `canUpdateOldRow` does not cover data-page scan.**
   The comment says entries are only read by link from the index tree, pending tree, and data tree under leaf lock. `ScanQuery.setDataPageScanEnabled` (and the JDBC equivalent) bypasses the data tree: `DataPageScanCursor` reads the head fragment under that page's lock and then follows `nextLink` page by page. A concurrent multi-page in-place update can produce a torn row for such readers. Before this PR fragmented rows were replaced, never rewritten. Either document the risk, disable data-page scan for such caches, or gate multi-page in-place update on it.
   
   **5. Unexplained threshold change.**
   `updateValSizeThreshold` moved from `pageSize / 2` to `pageSize * 3 / 4` with no comment. It now only gates the TTL and feature checks. Any row up to `pageSize - MIN_DATA_PAGE_OVERHEAD` is single-page anyway, so 3/4 is safe, but please explain it in the code or keep 1/2.
   
   **6. Missing coverage.**
   No tests for: shared cache group with `storeCacheIdInDataPage` (the format path `storeCacheId()` was introduced for); WAL-disabled group (`pageRecordsDisabled`); feature inactive during rolling upgrade; free-list level test in `CacheFreeListSelfTest` (only a stub method was added there); concurrent scan vs. in-place update.
   
   ### Minor / performance
   
   - `updateRowFragment` allocates a heap `ByteBuffer` per fragment page when WAL is enabled. A 100 KB row is ~25 allocations per update. A thread-local scratch buffer would remove that.
   - With WAL enabled, every fragment page is write-locked even if it ends up unmodified. Under a running checkpoint that triggers copy-on-write into the checkpoint buffer for pages that are never changed. Acceptable, but worth a comment.
   - `isActive(MULTI_PAGE_IN_PLACE_ROW_UPDATE_FEATURE)` is now evaluated in `RowStore.updateRow` for every in-place update, including small rows. It is cheap with the new `coreFeatures` fast path, but it could be evaluated only when `oldRow.size()` exceeds the single-page bound.
   - `oldRowExpiredFlag` is still declared and set in `AtomicCacheUpdateClosure` (line 4558) after its getter was removed from the interface; it is still used once (line 1526), so not dead, but the two closures now handle expiry differently. Consider unifying.
   - `DataPageUpdateResult` has a package-private constructor on a public class; fine, just noting.
   - `IgniteNodeFeatureSet.contains` fast path with `==` on the component-name constant is fine.
   
   ### What looks good
   
   - Fragment walk order matches `addRowFragment` (head first, `rowOff == written`); verified against `DataPageIO.writeFragmentData`.
   - Compare-before-write avoids dirtying and WAL-logging unchanged fragments; the `markDirtyAfterWrite` hook in `PageHandler` is a clean way to skip `setDirty`.
   - `updateRow(byte[])` for WAL replay now handles fragmented entries and asserts the stored length; the feature flag correctly protects downgraded nodes from silently skipping such records.
   - `storeCacheId()` replacing the `cacheId() != 0` heuristic removes the `oldLen -= 4` hack and the `DataRowCacheAware` subclass.
   - Crash/recovery tests (`testApplyInPlaceUpdateDeltaRecordsAfterCrash`, dirty page count test) exercise the snapshot + delta record path end to end.
   
   ### Verification done locally
   
   - Compiled ignite-core and upstream modules from the PR branch.
   - `MultiPageInPlaceUpdateTest`: 8/8 pass. `CacheFreeListSelfTest`: 20/20 pass.
   - `IgnitePdsWithTtlTest`: see the table above.
   - Checkstyle, RAT and `check-test-suites` were not run; the new test is registered in `IgniteBasicTestSuite2`.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29028 Implement multi-page in-place row update [ignite]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #13560:
URL: https://github.com/apache/ignite/pull/13560#issuecomment-5941381772

   <!-- ignite-rolling-upgrade-check -->
   ## Possible compatibility issues. Please, check rolling upgrade cases
   
   This PR modifies protected classes (with **Order** annotation).
   Changes to these classes can break rolling upgrade compatibility.
   
   **Affected files:**
   - `modules/core/src/main/java/org/apache/ignite/internal/processors/rollingupgrade/feature/IgniteNodeFeatureSet.java`
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29028 Implement multi-page in-place row update [ignite]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #13560:
URL: https://github.com/apache/ignite/pull/13560#issuecomment-5941568459

   <!-- ignite-rolling-upgrade-check -->
   ## Possible compatibility issues. Please, check rolling upgrade cases
   
   This PR modifies protected classes (with **Order** annotation).
   Changes to these classes can break rolling upgrade compatibility.
   
   **Affected files:**
   - `modules/core/src/main/java/org/apache/ignite/internal/processors/rollingupgrade/feature/IgniteNodeFeatureSet.java`
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-29028 Implement multi-page in-place row update [ignite]

Posted by "nizhikov (via GitHub)" <gi...@apache.org>.
nizhikov commented on PR #13560:
URL: https://github.com/apache/ignite/pull/13560#issuecomment-6142495967

   Review round 3, head 11ec050310e.
   
   **Verdict: LGTM from my side, pending a green TC bot run.** Both blocking items from round 1 are resolved and verified, the intermediate test-compile breakage is gone, and the design was strengthened with lock coupling. Everything listed below is **non-blocking** (severity noted per item).
   
   ### Status of previous findings
   
   | # | Item | Status |
   |---|---|---|
   | 1 | TTL regression (`IgnitePdsWithTtlTest`) | Fixed by reverting the expired-old-row change. Verified: 2/2 full-class runs clean on the new head (3/4 failed before). |
   | 2 | `assertNotSame` on `long` | Fixed via static import of JUnit `assertNotEquals`. The intermediate `JUnitAssertAware.assertNotEquals(long, long)` helper, which shadowed JUnit's overloads for every test subclass (70 compile errors in 9 files), is gone. |
   | 3 | `storeCacheId()` flag set on only one init path | Fixed on all three entry points, plus `testInPlaceUpdateRowSizeCalculation` for shared group / eviction region. |
   | 4 | Data page scan torn reads | Confirmed: nothing in the codebase ever enables the thread-local; note left in IGNITE-11998. Also moot now that readers use lock coupling. |
   | 5 | 3/4 threshold | Commented. |
   | 6 | Coverage | Mostly closed: concurrent update, concurrent torn-read detection (get + ScanQuery), shared group, checkpoint recovery files, delta records after crash. WAL-disabled group remains untested (minor). |
   
   ### Design changes since round 2 (reviewed, look correct)
   
   - Lock coupling on both sides: `AbstractFreeList.updateFragmentedRow` holds the write lock on fragment N while acquiring N+1; `CacheDataRowAdapter.doInitFromLink` does the same with read locks. I traced lock orders against data-tree / pending-tree / index leaf locks and page eviction and found no cycle: only head pages are shared between rows, and a head page is always the first lock in any chain.
   - TTL restriction for multi-page rows dropped: safe, the pending tree reads the key from the head page only.
   - Compare-before-write also when WAL is off on persistent regions, so checkpoint recovery files don't pick up untouched pages. Covered by `testDirtyPagesCountAfterUpdateWithCheckpoinRecoveryFiles`.
   - `PageHandler.markDirtyAfterWrite` hook removed in favour of manual locking through `DataStructure` helpers. Cleaner.
   
   ### Remaining findings
   
   1. **[Medium] Deadlock freedom rests on an assert-only invariant.** Lock coupling is safe only if non-head fragment pages are never shared. `writeWholePages` / `takePage` guarantee this for single and batch inserts, but the invariant is enforced only by an `assert` in `WriteRowHandler.addRow` at write time. With assertions disabled, data violating it (a future free-list change, a corrupted store) would deadlock instead of failing. A cheap runtime guard in `updateFragmentedRow` (e.g. next page `getDirectCount() == 1`, otherwise fall back to insert or throw a corruption error) would make that a safe path.
   2. **[Low] Lock failure handled by assert only.** `lockFragmentPage` / `readLockFragmentPage` assert a non-zero address; with assertions off a zero address is dereferenced. The pre-existing reader had the same weakness, so this is hardening, not a regression.
   3. **[Low] Page load under a held page lock.** `acquirePage` of the next fragment runs while the current fragment is write-locked, so on a persistent region a page miss does disk I/O under that lock, the entry lock and the checkpoint read lock. B+tree does the same, so acceptable, but worth a sentence in the javadoc.
   4. **[Low] Feature flag evaluated twice.** `canUpdateOldRow` checks `isActive(...)` for large rows, then `RowStore.updateRow` evaluates it again for every in-place update, including small rows. Passing `allowFragmented` down would remove the duplicate.
   5. **[Info] Rolling-upgrade bot warning is a false positive.** The new `coreFeatures` field in `IgniteNodeFeatureSet` carries no `@Order` and is not serialized.
   6. **[Trivial] Typos** in new text: "milti-page" (`IgniteCacheOffheapManagerImpl`), "Checkpoin" (test name), "writting" (recovery test comment).
   7. **[Info]** Merge base is 368f1f66f58, 14 commits behind master; a final rebase before merge is advisable.
   
   ### Verification on head 11ec050310e (local, fork profile, core built with upstream modules in a clean worktree)
   
   | Test class | Result |
   |---|---|
   | `MultiPageInPlaceUpdateTest` | 13/13 pass |
   | `IgnitePdsCheckpointRecoveryTest` | 8/8 pass |
   | `CacheFreeListSelfTest` | 20/20 pass |
   | `IgnitePdsWithTtlTest` (two runs) | 10/10 pass, both runs |
   
   Checkstyle, RAT and `check-test-suites` were not run locally; the new test is registered in `IgniteBasicTestSuite2`.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org