[XL] Change in asterixdb[master]: [NO ISSUE][STO] WIP: speed up columnar random sampling

[AsterixDB Code Review <do...@asterix-gerrit.ics.uci.edu>]

From Ritik Raj <ri...@couchbase.com>:

Attention is currently required from: Ritik Raj.

Hello Anon. E. Moose #1000171, Jenkins, 

I'd like you to reexamine a change. Please visit

    https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/21579?usp=email

to look at the new patch set (#3).

The following approvals got outdated and were removed:
Integration-Tests+1 by Jenkins


Change subject: [NO ISSUE][STO] WIP: speed up columnar random sampling
......................................................................

[NO ISSUE][STO] WIP: speed up columnar random sampling

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- Work in progress, pushed for visibility rather than review. It builds
  on the two ASTERIXDB-3802 changes below it in this stack.
- Random ANALYZE on a mutated columnar collection was far slower than a
  full scan. This collapses several passes of that work into one change:
  a benchmark harness, the cursor changes it justified, and the
  uniformity guards that keep the sample honest while the cursor gets
  faster.
- ColumnBtreeSampleCursor: fix a spatially biased sample caused by
  truncating a sorted draw batch at the target, draw intra-page slots
  without replacement, and share one primary key pass across all draws
  that land on the same page.
- Make the newer-component liveness probe existence-only: a zero-column
  projection through a new ColumnBTreeExistencePointSearchCursor and
  LSMColumnSampleLivenessSearchCursor, wired only where the sample
  collector builds its cursor so the query path is untouched. The cloud
  read context pins only what that projection needs.
- SampleCursorStats records a disjoint per-phase cost breakdown
  (liveness, primary key seek, materialization) so a change can be
  attributed rather than guessed at.
- Clamp columnar samples-per-page to 1: drawing several tuples from one
  page clusters the sample and moves chi-square well past its critical
  value.

Tests:
- ColumnSampleBenchHarness is the first in-tree column LSM sampling
  harness. It enforces its own cache-residency and disk-budget
  contracts, so a run that silently stopped measuring disk cost fails
  instead of reporting a fast number.
- Chi-square uniformity guards for both column and row sampling, each
  demonstrated to fail when the corresponding bias is reintroduced.
- Cost baseline, liveness wiring and shape, cloud pinning, and
  samples-per-page clamp tests.

Ext-ref: MB-73172

Co-Authored-By: Claude Opus 4.8 <no...@anthropic.com>
Change-Id: I531e3a4be1b5d91379319130463846b9ec8ee1bf
---
M asterixdb/asterix-column/pom.xml
M asterixdb/asterix-column/src/main/java/org/apache/asterix/column/operation/lsm/merge/MergeColumnReadMetadata.java
A asterixdb/asterix-column/src/test/java/org/apache/asterix/column/test/sample/CloudExistenceProjectionPinningTest.java
A asterixdb/asterix-column/src/test/java/org/apache/asterix/column/test/sample/ColumnSampleBenchHarness.java
A asterixdb/asterix-column/src/test/java/org/apache/asterix/column/test/sample/ColumnSampleBenchHarnessResidencyTest.java
A asterixdb/asterix-column/src/test/java/org/apache/asterix/column/test/sample/ColumnSampleBenchHarnessTest.java
A asterixdb/asterix-column/src/test/java/org/apache/asterix/column/test/sample/ColumnSampleCostBaselineTest.java
A asterixdb/asterix-column/src/test/java/org/apache/asterix/column/test/sample/ColumnSampleLivenessWiringTest.java
A asterixdb/asterix-column/src/test/java/org/apache/asterix/column/test/sample/ColumnSampleUniformityTest.java
M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/config/StorageProperties.java
A asterixdb/asterix-common/src/test/java/org/apache/asterix/common/config/ColumnSamplesPerPageClampTest.java
M hyracks-fullstack/hyracks/hyracks-storage-am-btree/src/main/java/org/apache/hyracks/storage/am/btree/impls/DiskBTreeSampleCursor.java
M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree-column/src/main/java/org/apache/hyracks/storage/am/lsm/btree/column/api/projection/ColumnProjectorType.java
M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree-column/src/main/java/org/apache/hyracks/storage/am/lsm/btree/column/api/projection/IColumnProjectionInfo.java
M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree-column/src/main/java/org/apache/hyracks/storage/am/lsm/btree/column/cloud/CloudColumnIndexDiskCacheManager.java
M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree-column/src/main/java/org/apache/hyracks/storage/am/lsm/btree/column/cloud/buffercache/read/CloudColumnReadContext.java
M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree-column/src/main/java/org/apache/hyracks/storage/am/lsm/btree/column/cloud/buffercache/read/CloudMegaPageReadContext.java
M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree-column/src/main/java/org/apache/hyracks/storage/am/lsm/btree/column/impls/btree/ColumnBTree.java
A hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree-column/src/main/java/org/apache/hyracks/storage/am/lsm/btree/column/impls/btree/ColumnBTreeExistencePointSearchCursor.java
M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree-column/src/main/java/org/apache/hyracks/storage/am/lsm/btree/column/impls/btree/ColumnBtreeSampleCursor.java
A hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree-column/src/main/java/org/apache/hyracks/storage/am/lsm/btree/column/impls/btree/SampleCursorStats.java
M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree-column/src/main/java/org/apache/hyracks/storage/am/lsm/btree/column/impls/lsm/LSMColumnBTree.java
A hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree-column/src/main/java/org/apache/hyracks/storage/am/lsm/btree/column/impls/lsm/LSMColumnSampleLivenessSearchCursor.java
M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree/src/main/java/org/apache/hyracks/storage/am/lsm/btree/impls/LSMBTree.java
M hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-btree-test/pom.xml
A hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-btree-test/src/test/java/org/apache/hyracks/storage/am/lsm/btree/DiskBTreeSampleUniformityTest.java
A hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-btree-test/src/test/java/org/apache/hyracks/storage/am/lsm/btree/SampleCursorStatsTest.java
27 files changed, 3,628 insertions(+), 227 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/79/21579/3
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/21579?usp=email
To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings?usp=email

Gerrit-MessageType: newpatchset
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Change-Id: I531e3a4be1b5d91379319130463846b9ec8ee1bf
Gerrit-Change-Number: 21579
Gerrit-PatchSet: 3
Gerrit-Owner: Ritik Raj <ri...@couchbase.com>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Attention: Ritik Raj <ri...@couchbase.com>