You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "Jackie-Jiang (via GitHub)" <gi...@apache.org> on 2026/09/27 06:48:37 UTC
[PR] Normalize empty filter results to EmptyDocIdSet when they are built [pinot]
Jackie-Jiang opened a new pull request, #19684:
URL: https://github.com/apache/pinot/pull/19684
## Summary
`BlockDocIdSet#getOptimizedDocIdSet` let a parent ask whether a child was empty, but only `AND` / `OR` `getTrues`, `CombinedFilterOperator` and `deriveNulls` asked. The not-false / false paths used by `NOT` and null handling checked `instanceof EmptyDocIdSet` directly and never saw an empty bitmap as empty. This PR normalizes emptiness when a doc id set is built instead, so an empty result is always an `EmptyDocIdSet` and every parent can recognize it by type:
- `BitmapDocIdSet`, `RangelessBitmapDocIdSet` and `SortedDocIdSet` get `create` factories that return an `EmptyDocIdSet` for an empty input. The index-based filter operators and `BlockDocIdSet#toNonScanDocIdSet` build their results through them.
- `ShortCircuitingDocIdSet` is merged into `EmptyDocIdSet`, which now keeps the number of entries scanned before the result was found empty. `EmptyDocIdSet.unscanned()` (was `getInstance()`) is the shared instance for a result found without scanning. It is no longer the only instance, so emptiness is tested with `instanceof`, not `==`.
- `BitmapDocIdSet` takes an optional scan count, replacing the anonymous subclasses in `H3IndexFilterOperator`, `H3InclusionIndexFilterOperator` and the legacy path of `RangeIndexBasedFilterOperator`.
- `BaseColumnFilterOperator` returns an empty match as is from `excludeNulls`, and returns the null bitmap directly from `getNotFalses` when nothing matches. The result stays recognizable as empty, and `AndDocIdSet` / `OrDocIdSet` keep merging their bitmap children instead of falling back to iterator intersection.
- `TestFilterOperator` moves from main to test sources.
## Behavior
Query results do not change: an `EmptyDocIdSet` iterates exactly like an empty bitmap. Two things do:
- More short-circuits fire. `NOT` over an index-based predicate that matches nothing becomes a match-all instead of iterating the complement of an empty set, and with null handling enabled an empty column predicate no longer hides inside an `AndDocIdSet`, so a parent `AND` stops early.
- `numEntriesScannedInFilter` can be lower on the `NOT` path. When a filter that scans eagerly (H3, legacy range index) matches nothing, `NOT` short-circuits to a match-all, which carries no scan count. This matches how `OR` already drops the count when it short-circuits on a match-all child.
## API changes
For code outside Pinot that implements or builds these classes:
- `BlockDocIdSet#getOptimizedDocIdSet` is removed.
- `ShortCircuitingDocIdSet` is removed; use `new EmptyDocIdSet(numEntriesScannedInFilter)`.
- `EmptyDocIdSet.getInstance()` is renamed to `EmptyDocIdSet.unscanned()`.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Normalize empty filter results to EmptyDocIdSet when they are built [pinot]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #19684:
URL: https://github.com/apache/pinot/pull/19684#issuecomment-5853877719
## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/19684?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:white_check_mark: All modified and coverable lines are covered by tests.
:white_check_mark: Project coverage is 100.00%. Comparing base ([`ef7b1b7`](https://app.codecov.io/gh/apache/pinot/commit/ef7b1b7a733bd85d7d49bf46bd3a4d5e7217d3ab?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`4879355`](https://app.codecov.io/gh/apache/pinot/commit/4879355eb45b8072594dcb69569495b1258a11a2?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #19684 +/- ##
==============================================
+ Coverage 68.07% 100.00% +31.92%
+ Complexity 1450 6 -1444
==============================================
Files 3516 3 -3513
Lines 228011 6 -228005
Branches 36077 0 -36077
==============================================
- Hits 155213 6 -155207
+ Misses 60613 0 -60613
+ Partials 12185 0 -12185
```
| [Flag](https://app.codecov.io/gh/apache/pinot/pull/19684/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [integration](https://app.codecov.io/gh/apache/pinot/pull/19684/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `100.00% <ø> (ø)` | |
| [integration1](https://app.codecov.io/gh/apache/pinot/pull/19684/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `100.00% <ø> (ø)` | |
| [integration2](https://app.codecov.io/gh/apache/pinot/pull/19684/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [java-25](https://app.codecov.io/gh/apache/pinot/pull/19684/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `100.00% <ø> (+31.92%)` | :arrow_up: |
| [lane-a](https://app.codecov.io/gh/apache/pinot/pull/19684/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `100.00% <ø> (ø)` | |
| [lane-b](https://app.codecov.io/gh/apache/pinot/pull/19684/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `0.00% <ø> (ø)` | |
| [temurin](https://app.codecov.io/gh/apache/pinot/pull/19684/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `100.00% <ø> (+31.92%)` | :arrow_up: |
| [unittests](https://app.codecov.io/gh/apache/pinot/pull/19684/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [unittests1](https://app.codecov.io/gh/apache/pinot/pull/19684/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [unittests2](https://app.codecov.io/gh/apache/pinot/pull/19684/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
</details>
[:umbrella: View full report in Codecov by Harness](https://app.codecov.io/gh/apache/pinot/pull/19684?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Normalize empty filter results to EmptyDocIdSet when they are built [pinot]
Posted by "xiangfu0 (via GitHub)" <gi...@apache.org>.
xiangfu0 merged PR #19684:
URL: https://github.com/apache/pinot/pull/19684
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org