You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "yashmayya (via GitHub)" <gi...@apache.org> on 2025/12/16 23:13:57 UTC
[PR] Guard against NPE for MSE filtered aggregations [pinot]
yashmayya opened a new pull request, #17386:
URL: https://github.com/apache/pinot/pull/17386
```
java.lang.NullPointerException: Cannot invoke "java.lang.Integer.intValue()" because "java.util.List.get(int)[filterArgId]" is null
at org.apache.pinot.query.runtime.operator.AggregateOperator.getMatchedBitmap(AggregateOperator.java:352)
at org.apache.pinot.query.runtime.operator.MultistageGroupByExecutor.processAggregateWithFilter(MultistageGroupByExecutor.java:347)
at org.apache.pinot.query.runtime.operator.MultistageGroupByExecutor.processAggregate(MultistageGroupByExecutor.java:316)
at org.apache.pinot.query.runtime.operator.MultistageGroupByExecutor.processBlock(MultistageGroupByExecutor.java:186)
at org.apache.pinot.query.runtime.operator.AggregateOperator.consumeGroupBy(AggregateOperator.java:270)
at org.apache.pinot.query.runtime.operator.AggregateOperator.getNextBlock(AggregateOperator.java:205)
at org.apache.pinot.query.runtime.operator.MultiStageOperator.nextBlock(MultiStageOperator.java:121)
at org.apache.pinot.query.runtime.operator.MailboxSendOperator.getNextBlock(MailboxSendOperator.java:209)
at org.apache.pinot.query.runtime.operator.MultiStageOperator.nextBlock(MultiStageOperator.java:121)
at org.apache.pinot.query.runtime.executor.OpChainSchedulerService$1.runJob(OpChainSchedulerService.java:131)
```
- This is an edge case issue because Calcite usually inserts an `IS_TRUE` filter on top of the filtered aggregate here - https://github.com/apache/calcite/blob/f19e854c39e5a43aea2323e455229bd8dfc6dcdb/core/src/main/java/org/apache/calcite/tools/RelBuilder.java#L4533-L4535. `IS_TRUE` returns `false` for `null` values.
- However, if a column has actual null values and query time null handling is enabled using `enableNullHandling`, but the column is not marked as nullable in the schema (see https://docs.pinot.apache.org/developers/advanced/null-value-support#column-based-null-storing), then the Calcite type is inferred as non-nullable and the `IS_TRUE` filter isn't inserted on top. So the filter in the filtered aggregate can generate actual null values which would then trigger the above NPE.
- This patch adds an additional guardrail to protect against such NPEs. The serialized block path doesn't need any changes because it will only ever return null value placeholders (`false` / `0` for boolean) and not actual nulls.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Guard against NPE for MSE filtered aggregations [pinot]
Posted by "gortiz (via GitHub)" <gi...@apache.org>.
gortiz merged PR #17386:
URL: https://github.com/apache/pinot/pull/17386
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Guard against NPE for MSE filtered aggregations [pinot]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #17386:
URL: https://github.com/apache/pinot/pull/17386#issuecomment-3662972845
## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/17386?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `50.00000%` with `1 line` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 55.60%. Comparing base ([`42715fc`](https://app.codecov.io/gh/apache/pinot/commit/42715fcc4af2ce61bf79982a16e39b55045ba09b?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`6dbd3dc`](https://app.codecov.io/gh/apache/pinot/commit/6dbd3dc8b5aae8a6b85b01668c96d8a90adee924?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 5 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/pinot/pull/17386?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...inot/query/runtime/operator/AggregateOperator.java](https://app.codecov.io/gh/apache/pinot/pull/17386?src=pr&el=tree&filepath=pinot-query-runtime%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fpinot%2Fquery%2Fruntime%2Foperator%2FAggregateOperator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-cGlub3QtcXVlcnktcnVudGltZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvcXVlcnkvcnVudGltZS9vcGVyYXRvci9BZ2dyZWdhdGVPcGVyYXRvci5qYXZh) | 50.00% | [0 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/pinot/pull/17386?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (42715fc) and HEAD (6dbd3dc). Click for more details.
>
> <details><summary>HEAD has 36 uploads less than BASE</summary>
>
>| Flag | BASE (42715fc) | HEAD (6dbd3dc) |
>|------|------|------|
>|java-21|5|1|
>|unittests1|2|1|
>|unittests|4|1|
>|temurin|10|1|
>|java-11|5|0|
>|unittests2|2|0|
>|integration|6|0|
>|integration2|2|0|
>|integration1|2|0|
>|custom-integration1|2|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #17386 +/- ##
============================================
- Coverage 63.21% 55.60% -7.61%
+ Complexity 1474 702 -772
============================================
Files 3147 2459 -688
Lines 187562 138857 -48705
Branches 28712 22139 -6573
============================================
- Hits 118560 77215 -41345
+ Misses 59808 55128 -4680
+ Partials 9194 6514 -2680
```
| [Flag](https://app.codecov.io/gh/apache/pinot/pull/17386/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [custom-integration1](https://app.codecov.io/gh/apache/pinot/pull/17386/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [integration](https://app.codecov.io/gh/apache/pinot/pull/17386/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [integration1](https://app.codecov.io/gh/apache/pinot/pull/17386/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [integration2](https://app.codecov.io/gh/apache/pinot/pull/17386/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [java-11](https://app.codecov.io/gh/apache/pinot/pull/17386/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [java-21](https://app.codecov.io/gh/apache/pinot/pull/17386/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `55.60% <50.00%> (-7.58%)` | :arrow_down: |
| [temurin](https://app.codecov.io/gh/apache/pinot/pull/17386/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `55.60% <50.00%> (-7.61%)` | :arrow_down: |
| [unittests](https://app.codecov.io/gh/apache/pinot/pull/17386/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `55.60% <50.00%> (-7.61%)` | :arrow_down: |
| [unittests1](https://app.codecov.io/gh/apache/pinot/pull/17386/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `55.60% <50.00%> (-0.02%)` | :arrow_down: |
| [unittests2](https://app.codecov.io/gh/apache/pinot/pull/17386/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
</details>
[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/pinot/pull/17386?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org