You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "yashmayya (via GitHub)" <gi...@apache.org> on 2025/04/11 13:17:39 UTC
[PR] Fix MIN_MAX_RANGE aggregation function behavior when all input values are null [pinot]
yashmayya opened a new pull request, #15525:
URL: https://github.com/apache/pinot/pull/15525
- Currently, the `MIN_MAX_RANGE` aggregation function returns `-Infinity` when null handling is enabled and all input values are null.
- This is incorrect and the function should return `null` in such cases. This patch fixes the issue.
- Note that the bug only applies to pure aggregations and not group by / keyed aggregations (which already returned `null` in such scenarios).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Fix MIN_MAX_RANGE aggregation function behavior when all input values are null [pinot]
Posted by "gortiz (via GitHub)" <gi...@apache.org>.
gortiz commented on code in PR #15525:
URL: https://github.com/apache/pinot/pull/15525#discussion_r2044442718
##########
pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/MinMaxRangeAggregationFunction.java:
##########
@@ -81,10 +84,13 @@ public void aggregate(int length, AggregationResultHolder aggregationResultHolde
for (int i = from; i < to; i++) {
MinMaxRangePair minMaxRangePair = ObjectSerDeUtils.MIN_MAX_RANGE_PAIR_SER_DE.deserialize(bytesValues[i]);
minMax.apply(minMaxRangePair);
+ empty.set(false);
}
});
}
- setAggregationResult(aggregationResultHolder, minMax.getMin(), minMax.getMax());
+ if (!empty.get()) {
Review Comment:
The code may be clearer, but I don't think the suggested version will be faster than the current code. It probably is depreciable, but I would say this code could be even faster than the one in #15545. I'm fine with both versions, and I would even say I prefer the new one because it is easier to read, but I don't see how that can affect performance.
The difference between reading an atomic boolean modified by a single thread and comparing two primitives (through a reference) should always favor the atomic boolean. But even if it isn't, the cost of all other instructions in this method are going to hide any tiny difference between one and the other version
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Fix MIN_MAX_RANGE aggregation function behavior when all input values are null [pinot]
Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang commented on code in PR #15525:
URL: https://github.com/apache/pinot/pull/15525#discussion_r2045201020
##########
pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/MinMaxRangeAggregationFunction.java:
##########
@@ -81,10 +84,13 @@ public void aggregate(int length, AggregationResultHolder aggregationResultHolde
for (int i = from; i < to; i++) {
MinMaxRangePair minMaxRangePair = ObjectSerDeUtils.MIN_MAX_RANGE_PAIR_SER_DE.deserialize(bytesValues[i]);
minMax.apply(minMaxRangePair);
+ empty.set(false);
}
});
}
- setAggregationResult(aggregationResultHolder, minMax.getMin(), minMax.getMax());
+ if (!empty.get()) {
Review Comment:
My conclusion is mainly based on how many times the method is invoked. With current approach, we need to update the atomic boolean for each doc, which could also potentially introduce memory barrier (not sure about the overhead in single thread scenario).
The reading part should be more or less the same, but we can skip the update
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Fix MIN_MAX_RANGE aggregation function behavior when all input values are null [pinot]
Posted by "yashmayya (via GitHub)" <gi...@apache.org>.
yashmayya commented on code in PR #15525:
URL: https://github.com/apache/pinot/pull/15525#discussion_r2046438796
##########
pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/MinMaxRangeAggregationFunction.java:
##########
@@ -81,10 +84,13 @@ public void aggregate(int length, AggregationResultHolder aggregationResultHolde
for (int i = from; i < to; i++) {
MinMaxRangePair minMaxRangePair = ObjectSerDeUtils.MIN_MAX_RANGE_PAIR_SER_DE.deserialize(bytesValues[i]);
minMax.apply(minMaxRangePair);
+ empty.set(false);
}
});
}
- setAggregationResult(aggregationResultHolder, minMax.getMin(), minMax.getMax());
+ if (!empty.get()) {
Review Comment:
Yeah the read part should definitely not be an issue, but it makes sense to avoid updating the atomic in the tight loop
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Fix MIN_MAX_RANGE aggregation function behavior when all input values are null [pinot]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #15525:
URL: https://github.com/apache/pinot/pull/15525#issuecomment-2797000761
## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/15525?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
All modified and coverable lines are covered by tests :white_check_mark:
> Project coverage is 55.84%. Comparing base [(`9145e92`)](https://app.codecov.io/gh/apache/pinot/commit/9145e92f655ed481d9ceac6e92ba1d0671b1d1c4?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) to head [(`7774416`)](https://app.codecov.io/gh/apache/pinot/commit/7774416c1149b2e4cefac803d25866bca51e38d4?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
> Report is 5 commits behind head on master.
> :exclamation: There is a different number of reports uploaded between BASE (9145e92) and HEAD (7774416). Click for more details.
>
> <details><summary>HEAD has 67 uploads less than BASE</summary>
>
>| Flag | BASE (9145e92) | HEAD (7774416) |
>|------|------|------|
>|unittests|6|1|
>|unittests1|3|1|
>|java-21|10|1|
>|skip-bytebuffers-false|8|1|
>|temurin|15|1|
>|skip-bytebuffers-true|4|0|
>|unittests2|3|0|
>|java-11|5|0|
>|integration|9|0|
>|integration2|3|0|
>|integration1|3|0|
>|custom-integration1|3|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #15525 +/- ##
============================================
- Coverage 62.95% 55.84% -7.12%
+ Complexity 1381 681 -700
============================================
Files 2848 2223 -625
Lines 161547 120091 -41456
Branches 24752 19188 -5564
============================================
- Hits 101700 67060 -34640
+ Misses 52167 47498 -4669
+ Partials 7680 5533 -2147
```
| [Flag](https://app.codecov.io/gh/apache/pinot/pull/15525/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [custom-integration1](https://app.codecov.io/gh/apache/pinot/pull/15525/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [integration](https://app.codecov.io/gh/apache/pinot/pull/15525/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [integration1](https://app.codecov.io/gh/apache/pinot/pull/15525/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [integration2](https://app.codecov.io/gh/apache/pinot/pull/15525/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [java-11](https://app.codecov.io/gh/apache/pinot/pull/15525/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [java-21](https://app.codecov.io/gh/apache/pinot/pull/15525/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `55.84% <100.00%> (-7.11%)` | :arrow_down: |
| [skip-bytebuffers-false](https://app.codecov.io/gh/apache/pinot/pull/15525/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `55.84% <100.00%> (-7.11%)` | :arrow_down: |
| [skip-bytebuffers-true](https://app.codecov.io/gh/apache/pinot/pull/15525/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [temurin](https://app.codecov.io/gh/apache/pinot/pull/15525/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `55.84% <100.00%> (-7.12%)` | :arrow_down: |
| [unittests](https://app.codecov.io/gh/apache/pinot/pull/15525/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `55.84% <100.00%> (-7.11%)` | :arrow_down: |
| [unittests1](https://app.codecov.io/gh/apache/pinot/pull/15525/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `55.84% <100.00%> (-0.08%)` | :arrow_down: |
| [unittests2](https://app.codecov.io/gh/apache/pinot/pull/15525/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
</details>
[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/pinot/pull/15525?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Fix MIN_MAX_RANGE aggregation function behavior when all input values are null [pinot]
Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang commented on code in PR #15525:
URL: https://github.com/apache/pinot/pull/15525#discussion_r2043248743
##########
pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/MinMaxRangeAggregationFunction.java:
##########
@@ -81,10 +84,13 @@ public void aggregate(int length, AggregationResultHolder aggregationResultHolde
for (int i = from; i < to; i++) {
MinMaxRangePair minMaxRangePair = ObjectSerDeUtils.MIN_MAX_RANGE_PAIR_SER_DE.deserialize(bytesValues[i]);
minMax.apply(minMaxRangePair);
+ empty.set(false);
}
});
}
- setAggregationResult(aggregationResultHolder, minMax.getMin(), minMax.getMax());
+ if (!empty.get()) {
Review Comment:
To have better performance, we can check if `max >= min`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Fix MIN_MAX_RANGE aggregation function behavior when all input values are null [pinot]
Posted by "yashmayya (via GitHub)" <gi...@apache.org>.
yashmayya commented on code in PR #15525:
URL: https://github.com/apache/pinot/pull/15525#discussion_r2043619684
##########
pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/MinMaxRangeAggregationFunction.java:
##########
@@ -81,10 +84,13 @@ public void aggregate(int length, AggregationResultHolder aggregationResultHolde
for (int i = from; i < to; i++) {
MinMaxRangePair minMaxRangePair = ObjectSerDeUtils.MIN_MAX_RANGE_PAIR_SER_DE.deserialize(bytesValues[i]);
minMax.apply(minMaxRangePair);
+ empty.set(false);
}
});
}
- setAggregationResult(aggregationResultHolder, minMax.getMin(), minMax.getMax());
+ if (!empty.get()) {
Review Comment:
https://github.com/apache/pinot/pull/15545
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Fix MIN_MAX_RANGE aggregation function behavior when all input values are null [pinot]
Posted by "gortiz (via GitHub)" <gi...@apache.org>.
gortiz merged PR #15525:
URL: https://github.com/apache/pinot/pull/15525
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org