You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@iotdb.apache.org by Yuan Tian <ja...@gmail.com> on 2026/07/24 09:03:48 UTC
[Bug] Table-model grouped MAX returns NULL for non-positive floating-point values
Hi all,
I found an issue in the table-model grouped MAX aggregation for FLOAT
and DOUBLE values.
Bug description
---------------
When all values in a group are zero, MAX returns NULL instead of 0.
MIN and AVG return the expected result, and MAX_BY(value, value) also
works correctly.
The issue is not limited to zero. It affects any FLOAT or DOUBLE group
whose maximum value is non-positive, including groups containing only
negative values.
For example:
SELECT device_id, MAX(value), MIN(value), AVG(value)
FROM table1
GROUP BY device_id;
If all value entries for a device are 0, the result is:
MAX(value) = NULL
MIN(value) = 0
AVG(value) = 0
Root cause
----------
GroupedMaxAccumulator used the following initial values:
new FloatBigArray(Float.MIN_VALUE)
new DoubleBigArray(Double.MIN_VALUE)
In Java, Float.MIN_VALUE and Double.MIN_VALUE are the smallest
positive non-zero values, rather than the most negative representable
values.
The grouped accumulator updates its state using logic equivalent to:
if (value >= currentMax) {
inits.set(groupId, true);
currentMax = value;
}
Therefore, when every input value is zero or negative, the condition
is never satisfied. The group's initialization flag remains false, and
evaluateFinal() treats the group as empty and emits NULL.
This explains why positive values work and why MAX_BY is unaffected.
MAX_BY explicitly checks whether the group has been initialized before
comparing values.
Affected scope
--------------
The issue affects table-model grouped MAX(FLOAT) and MAX(DOUBLE) when
the query uses GroupedMaxAccumulator, such as the hash or
streaming-hash aggregation path.
The non-grouped MaxAccumulator and the tree-model MAX_VALUE
implementation use separate initialization handling and are not
affected.
Proposed fix
------------
Initialize the floating-point maximum state with negative infinity:
new FloatBigArray(Float.NEGATIVE_INFINITY)
new DoubleBigArray(Double.NEGATIVE_INFINITY)
This preserves the existing sentinel-based update path while correctly
handling zero, negative values, and negative infinity. Truly empty
groups still return NULL because their initialization flags remain
false.
Validation
----------
The fix includes:
- Unit tests for FLOAT and DOUBLE values containing zero, negative
values, and negative infinity.
- Coverage for both raw input and intermediate aggregation input.
- Verification that groups without input still return NULL.
- A table-model integration test covering the non-stream grouped
aggregation path.
The targeted unit and integration tests passed, along with a clean
37-module reactor build.
I opened a draft PR here:
https://github.com/apache/iotdb/pull/18300
Feedback is welcome.
Best regards,
Yuan Tian
Re: [Bug] Table-model grouped MAX returns NULL for non-positive floating-point values
Posted by Zh D <dz...@gmail.com>.
Hi Yuan, Xuan,
+1 on including this in 2.0.11. Here is some verification data from a
downstream integration, in case it is useful for the release decision.
I could reproduce the wrong result on the released standalone images of
both 2.0.8 and 2.0.10, and through the time-bucketed downsampling shape
rather than a tag GROUP BY:
SELECT date_bin(1000ms, time, 1000) AS bucket_ts,
MAX(COALESCE(double_v, CAST(long_v AS DOUBLE))) AS agg_num,
...
FROM telemetry
WHERE <identity predicates> AND time >= 1000 AND time < 3000
GROUP BY 1
With two rows of -5.0 and -3.0 in one bucket, agg_num comes back NULL on
both builds. The same window returns the correct -3.0 when the query is
issued without GROUP BY. An all-zero bucket also returns NULL, and a
bucket of positive values is correct. MIN over the same expression, and
AVG, SUM and COUNT, are all correct, and MAX(long_v) is correct, which
matches the FLOAT/DOUBLE-only sentinel analysis in your report.
On affected versions: the Float.MIN_VALUE / Double.MIN_VALUE
initialization is present in every release tag I checked (v2.0.4, v2.0.6,
v2.0.8 and v2.0.10), and the commit that introduced it is already
contained in v2.0.2, so this does not look like a recent regression.
On severity, the reason I would argue for 2.0.11: the failure is silent.
In the ThingsBoard Table Mode integration in iotdb-extras, the aggregation
DAO treats a NULL aggregate for a bucket as "this bucket has no data" and
skips it. So a MAX downsampling query over, for example, a sub-zero
temperature sensor does not return a wrong number that someone would
notice -- it returns no points at all for those buckets, and nothing
reports an error anywhere. I only found it because of this thread, and I
have opened a client-side workaround for it (iotdb-extras#116) so that the
integration stays correct on current releases.
One more data point on workarounds, in case anyone hits this before
upgrading: projecting the raw column instead of an expression is not
reliable. In my runs MAX(double_v) returned the correct value for a group
where every row had a non-null double_v, but returned NULL for a group
where double_v was a mix of NULLs and negative values, which is the normal
case for a schema that stores one typed value column per row. Computing
-MIN(-x) did work on both builds, since the grouped MIN accumulator seeds
with MAX_VALUE, which is a correct sentinel for every finite double.
Thanks Yuan for the quick root cause and fix.
Best regards,
Zihan Dai
On Fri, Jul 24, 2026 07:10 PM, Yuan Tian <ja...@gmail.com> wrote:
> I think it's a critical bug, we need to include that into the 2.0.11
> version, what do other people think?
>
Re: [Bug] Table-model grouped MAX returns NULL for non-positive floating-point values
Posted by Yuan Tian <ja...@gmail.com>.
I think it's a critical bug, we need to include that into the 2.0.11
version, what do other people think?
Re: [Bug] Table-model grouped MAX returns NULL for non-positive floating-point values
Posted by 王旋 <cr...@gmail.com>.
Nice catch! Could we cherry-pick this into the rc/2.0.11 branch?
Xuan Wang
Yuan Tian <ja...@gmail.com> 于2026年7月24日周五 17:04写道：
> Hi all,
>
>
> I found an issue in the table-model grouped MAX aggregation for FLOAT
> and DOUBLE values.
>
> Bug description
> ---------------
>
> When all values in a group are zero, MAX returns NULL instead of 0.
> MIN and AVG return the expected result, and MAX_BY(value, value) also
> works correctly.
>
> The issue is not limited to zero. It affects any FLOAT or DOUBLE group
> whose maximum value is non-positive, including groups containing only
> negative values.
>
> For example:
>
> SELECT device_id, MAX(value), MIN(value), AVG(value)
> FROM table1
> GROUP BY device_id;
>
> If all value entries for a device are 0, the result is:
>
> MAX(value) = NULL
> MIN(value) = 0
> AVG(value) = 0
>
> Root cause
> ----------
>
> GroupedMaxAccumulator used the following initial values:
>
> new FloatBigArray(Float.MIN_VALUE)
> new DoubleBigArray(Double.MIN_VALUE)
>
> In Java, Float.MIN_VALUE and Double.MIN_VALUE are the smallest
> positive non-zero values, rather than the most negative representable
> values.
>
> The grouped accumulator updates its state using logic equivalent to:
>
> if (value >= currentMax) {
> inits.set(groupId, true);
> currentMax = value;
> }
>
> Therefore, when every input value is zero or negative, the condition
> is never satisfied. The group's initialization flag remains false, and
> evaluateFinal() treats the group as empty and emits NULL.
>
> This explains why positive values work and why MAX_BY is unaffected.
> MAX_BY explicitly checks whether the group has been initialized before
> comparing values.
>
> Affected scope
> --------------
>
> The issue affects table-model grouped MAX(FLOAT) and MAX(DOUBLE) when
> the query uses GroupedMaxAccumulator, such as the hash or
> streaming-hash aggregation path.
>
> The non-grouped MaxAccumulator and the tree-model MAX_VALUE
> implementation use separate initialization handling and are not
> affected.
>
> Proposed fix
> ------------
>
> Initialize the floating-point maximum state with negative infinity:
>
> new FloatBigArray(Float.NEGATIVE_INFINITY)
> new DoubleBigArray(Double.NEGATIVE_INFINITY)
>
> This preserves the existing sentinel-based update path while correctly
> handling zero, negative values, and negative infinity. Truly empty
> groups still return NULL because their initialization flags remain
> false.
>
> Validation
> ----------
>
> The fix includes:
>
> - Unit tests for FLOAT and DOUBLE values containing zero, negative
> values, and negative infinity.
> - Coverage for both raw input and intermediate aggregation input.
> - Verification that groups without input still return NULL.
> - A table-model integration test covering the non-stream grouped
> aggregation path.
>
> The targeted unit and integration tests passed, along with a clean
> 37-module reactor build.
>
> I opened a draft PR here:
> https://github.com/apache/iotdb/pull/18300
>
> Feedback is welcome.
>
> Best regards,
> Yuan Tian
>