[I] [NIDX-01E] Cut over Property repair search-after [skywalking]

["hanahmily (via GitHub)" <gi...@apache.org>]

hanahmily opened a new issue, #14012:
URL: https://github.com/apache/skywalking/issues/14012

   # [NIDX-01E] Cut over Property repair search-after
   
   Parent workstream: #14002
   Blocked by: #14011
   
   ## Summary
   
   Add the bounded doc-value tuple page required by Property repair and move repair tree construction off Bluge. This final NIDX-01 leaf implements only ascending `(group, name, entityID, timestamp)` ordering, strict search-after, stored SHA projection, and pinned-generation behavior.
   
   ## Blocked by
   
   NIDX-01D (#14011) must be merged before this work starts. Before applying `Backlog`, update inherited visitor identifier names to match main; do not widen the behavior.
   
   ## Pre-agreed test seam
   
   The product seam is one complete Property repair build: provide a committed shard snapshot to the existing repair build operation and observe its persisted repair tree/state output. Tests may exercise the bounded native tuple-page operation through the BanyanDB-owned read-only visitor merged by earlier leaves, but must not target doc-value chunk decoders, heaps, or ICE types.
   
   No generic collector, arbitrary sort expression, offset pagination, or relevance order is part of the seam.
   
   ## Independent fixture
   
   The checked-in legacy Property shard contains these visible repair rows in declared ascending order:
   
   1. `(g-a, n-a, e-1, 10, sha-a)`
   2. `(g-a, n-a, e-1, 20, sha-b)`
   3. `(g-a, n-b, e-2, 5, sha-c)`
   4. `(g-b, n-a, e-3, 7, sha-d)`
   
   With page size `2`, page one is rows `1–2`; the strict cursor is `(g-a,n-a,e-1,20)`; page two is rows `3–4`. The completed repair tree maps `g-a/n-a/e-1 → sha-b`, `g-a/n-b/e-2 → sha-c`, and `g-b/n-a/e-3 → sha-d`.
   
   ## Requirements
   
   R1. The native page orders encoded doc values by all four ascending components and resumes strictly after the complete prior tuple on the same pinned generation. It returns no duplicate or skipped row.
   
   R2. Each page holds at most the configured page size plus bounded comparison state; it never performs a growing offset scan or materializes the shard.
   
   R3. The existing repair build uses native pages and produces exactly the declared three entity/SHA leaves and its normal generation state.
   
   R4. A generation published after page one does not enter page two. Missing sort values, malformed doc-value chunks, invalid cursors, cancellation, and limit overflow return typed bounded errors and publish no partial repair state. CRC32 remains ignored.
   
   ## RED and end-to-end contract
   
   - Boundary RED: request the two declared pages through the native tuple-page seam; earlier leaves do not support doc values or search-after.
   - E2E RED: run a complete Property repair build at page size `2`, publish another legacy generation between pages, and verify the tree contains only the pinned fixture's three declared leaves and SHA values.
   
   ## Acceptance criteria
   
   - `go test ./pkg/index/inverted/... ./banyand/property/db -run 'Test(NativeRepairTuplePage|E2EPropertyRepairNative)' -count=1` passes, including under `-race`.
   - A call-graph/dependency assertion proves Property repair no longer opens a Bluge reader or constructs a Bluge `TopNSearch`.
   - Pagination, concurrent publication, corrupt doc values, invalid cursor, cancellation, and bounded-memory cases pass.
   - `make test-ci PKG='./pkg/index/inverted/... ./banyand/property/db'`, `make build`, and `make pre-push` pass.
   
   ## Scope
   
   Packages: `pkg/index/inverted`, its private doc-value reader, and `banyand/property/db` repair.
   
   Out of scope: arbitrary sort, descending repair order, range/prefix/wildcard/MATCH queries, online Property writer/query cutover, snapshots written by native code, merge, expiry, and GC.
   
   ## Compatibility and rollback
   
   This slice emits only the existing repair state/tree artifacts, not index bytes. Roll back repair reads to the retained Bluge path; the source ICE/snapshot directory remains unchanged.
   
   ## Design
   
   [BDB-NIDX-SPEC-001 revision 0.2 — NIDX-01](https://github.com/apache/skywalking-banyandb/blob/main/docs/design/archive/0.12.0/native-inverted-index/delivery-review.html#nidx-01)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [NIDX-01E] Cut over Property repair search-after [skywalking]

["hanahmily (via GitHub)" <gi...@apache.org>]

hanahmily closed issue #14012: [NIDX-01E] Cut over Property repair search-after
URL: https://github.com/apache/skywalking/issues/14012


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org