You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@paimon.apache.org by "JingsongLi (via GitHub)" <gi...@apache.org> on 2026/09/20 02:48:09 UTC

[PR] [core] Add cost guards for data evolution scalar indexes [paimon]

JingsongLi opened a new pull request, #9986:
URL: https://github.com/apache/paimon/pull/9986

   ### Purpose
   
   Prevent BTree scalar-index lookups on data-evolution tables from making broad or fragmented queries slower than a regular data scan.
   
   - Add a query-wide decoded RowID budget shared by BTree readers and abandon the complete lookup when exceeded.
   - Fall back to a regular data scan when index candidates exceed the configured selection ratio or range-count limit.
   - Apply the guards to FAST, FULL, and DETAIL modes; document that FAST fallback may include matches outside current index coverage.
   - Convert Roaring bitmaps to ranges with an early fragmentation limit.
   - Cache null-bitmap cardinality safely for concurrent readers.
   
   ### Tests
   
   - BtreeGlobalIndexTableTest and DataEvolutionBatchScanTest: 34 tests passed.
   - Relevant paimon-common global-index suites: 350 tests passed.
   - mvn -q -pl paimon-api,paimon-common,paimon-core -am -DskipTests compile
   - git diff --check


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [core] Add cost guards for data evolution scalar indexes [paimon]

Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.

JingsongLi closed pull request #9986: [core] Add cost guards for data evolution scalar indexes
URL: https://github.com/apache/paimon/pull/9986


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org