You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "hanke580 (via GitHub)" <gi...@apache.org> on 2026/10/04 05:25:52 UTC
[I] [Bug] INVERTED index on a FLOAT/DOUBLE column treats -0.0 as less than 0, so the same predicate returns different rows with and without the index [doris]
hanke580 opened a new issue, #68715:
URL: https://github.com/apache/doris/issues/68715
### Search before asking
- [x] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues.
### Version
- `master` @ `0c29961f45ee976ef61c6a5dfe7b38b40a6ec131` (2026-09-30), built from source
- 4.1.4 (`doris-4.1.4-rc04-ad35a140c7f`, official `apache/doris:fe-4.1.4` / `be-4.1.4` images)
Single FE + single BE, default configuration (`inverted_index_storage_format` = V3).
### What's Wrong?
When a `DOUBLE` or `FLOAT` column has an `INVERTED` index and contains `-0.0`, predicates
answered by the index treat `-0.0` as strictly less than `0.0`:
- `v = 0` and `v >= 0` leave out the `-0.0` row
- `v < 0` and `v != 0` include it
Without the index, or with `enable_inverted_index_query = false`, the same table and query
give the IEEE-754 answer (`-0.0 = 0.0`), so whether the index is used changes the result.
| predicate | table without index | same rows, `INVERTED` index |
|---|---|---|
| `v = 0` | k = 0, 1 | **k = 1** |
| `v < 0` | k = 3 | **k = 0, 3** |
| `v >= 0` | k = 0, 1, 2 | **k = 1, 2** |
| `count(*) WHERE v != 0` | 2 | **3** |
| `v > 0`, `v <= 0` | same | same |
(k = 0 holds `-0.0`, k = 1 holds `0.0`.) `FLOAT` gives the same results.
### What You Expected?
The index should give the same answer as the column scan. The scan, Doris's own
comparison, and IEEE-754 all treat `-0.0` and `0.0` as equal, so with the index:
- `v = 0` should return k = 0, 1
- `v < 0` should return only k = 3
- `v >= 0` should return k = 0, 1, 2
- `v != 0` should count 2
### How to Reproduce?
```sql
CREATE DATABASE IF NOT EXISTS b065;
USE b065;
CREATE TABLE plain (k INT, v DOUBLE)
DUPLICATE KEY(k) DISTRIBUTED BY HASH(k) BUCKETS 1
PROPERTIES ('replication_num' = '1');
CREATE TABLE ixed (k INT, v DOUBLE, INDEX iv (v) USING INVERTED)
DUPLICATE KEY(k) DISTRIBUTED BY HASH(k) BUCKETS 1
PROPERTIES ('replication_num' = '1');
INSERT INTO plain VALUES (0, CAST('-0.0' AS DOUBLE)), (1, 0.0), (2, 1.5), (3, -1.5);
INSERT INTO ixed VALUES (0, CAST('-0.0' AS DOUBLE)), (1, 0.0), (2, 1.5), (3, -1.5);
SELECT k, v, CAST(v AS STRING) AS v_str FROM ixed ORDER BY k;
SELECT 'plain' AS t, k FROM plain WHERE v = 0 ORDER BY k;
SELECT 'ixed' AS t, k FROM ixed WHERE v = 0 ORDER BY k;
SELECT 'plain' AS t, k FROM plain WHERE v < 0 ORDER BY k;
SELECT 'ixed' AS t, k FROM ixed WHERE v < 0 ORDER BY k;
SELECT 'plain' AS t, k FROM plain WHERE v >= 0 ORDER BY k;
SELECT 'ixed' AS t, k FROM ixed WHERE v >= 0 ORDER BY k;
SELECT 'plain' AS t, count(*) FROM plain WHERE v != 0;
SELECT 'ixed' AS t, count(*) FROM ixed WHERE v != 0;
SET enable_inverted_index_query = false;
SELECT k FROM ixed WHERE v = 0 ORDER BY k;
SELECT k FROM ixed WHERE v < 0 ORDER BY k;
```
Actual output on master (abridged from the verify log):
```
SELECT k, v, CAST(v AS STRING) AS v_str FROM ixed ORDER BY k
+------+------+-------+
| k | v | v_str |
+------+------+-------+
| 0 | -0 | -0 |
| 1 | 0 | 0 |
| 2 | 1.5 | 1.5 |
| 3 | -1.5 | -1.5 |
+------+------+-------+
v = 0 plain: 0, 1 ixed: 1 <-- k=0 (-0.0) missing
v < 0 plain: 3 ixed: 0, 3 <-- k=0 (-0.0) included
v >= 0 plain: 0, 1, 2 ixed: 1, 2 <-- k=0 (-0.0) missing
count(*) WHERE v != 0 plain: 2 ixed: 3
SET enable_inverted_index_query = false;
v = 0 ixed: 0, 1 <-- correct again
v < 0 ixed: 3 <-- correct again
```
Expected: the `ixed` results equal the `plain` results for every predicate.
`FLOAT` instead of `DOUBLE` gives the same results for `=`, `<`, `>=` and `!=`.
### Anything Else?
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [I] [Bug] INVERTED index on a FLOAT/DOUBLE column treats -0.0 as less than 0, so the same predicate returns different rows with and without the index [doris]
Posted by "Doris-Breakwater (via GitHub)" <gi...@apache.org>.
Doris-Breakwater commented on issue #68715:
URL: https://github.com/apache/doris/issues/68715#issuecomment-5976951547
Breakwater-GitHub-Analysis-Slot: slot_5ab6a687cb2f
**Triage: reproducible correctness issue in floating-point inverted-index comparisons (high priority).** The supplied four-row reproducer and the scan/index toggle are sufficient to isolate the symptom to index evaluation. I have reviewed the code at `master` commit `0c29961f45e` and the local `4.1.4-rc04` tag; I have not independently run a Doris cluster.
**Code evidence / likely cause:** The FLOAT/DOUBLE key coder deliberately gives the two zeros different sortable keys: FLOAT `-0.0` is `7FFFFFFF` and `+0.0` is `80000000` (DOUBLE `7FFFFFFFFFFFFFFF` and `8000000000000000`). See [key encoding](https://github.com/apache/doris/blob/0c29961f45ee976ef61c6a5dfe7b38b40a6ec131/be/src/storage/key_coder.h#L359-L440) and [the existing special-value test](https://github.com/apache/doris/blob/0c29961f45ee976ef61c6a5dfe7b38b40a6ec131/be/test/storage/key_coder_test.cpp#L751-L817). The [inverted-index writer](https://github.com/apache/doris/blob/0c29961f45ee976ef61c6a5dfe7b38b40a6ec131/be/src/storage/index/inverted/inverted_index_writer.cpp#L555-L584) stores that encoding in BKD. The [BKD reader](https://github.com/apache/doris/blob/0c29961f45ee976ef61c6a5dfe7b38b40a6ec131/be/src/storage/index/inverted/inverted_index_reader.cpp#L668-L691) encodes the predicate value into its search bounds, and its [visitor compares the encoded bytes](https://githu
b.com/apache/doris/blob/0c29961f45ee976ef61c6a5dfe7b38b40a6ec131/be/src/storage/index/inverted/inverted_index_reader.cpp#L946-L1087). Thus an equality probe for `+0.0` misses the stored `-0.0`, `< +0.0` includes it, and `>= +0.0` excludes it. The [comparison predicate](https://github.com/apache/doris/blob/0c29961f45ee976ef61c6a5dfe7b38b40a6ec131/be/src/storage/predicate/comparison_predicate.h#L74-L123) implements `!=` by subtracting equality matches, so the same mismatch also explains its extra row. These are source-confirmed mechanisms consistent with the reported output; the exact execution path of each supplied SQL query still warrants an `EXPLAIN`/profile check.
**Immediate mitigation:** For queries on affected FLOAT/DOUBLE indexed columns, use `SET enable_inverted_index_query = false` in the querying session, as the issue's control query demonstrates, until a correctness fix is available. This trades index performance for scan semantics.
**Maintainer next steps:** Add a regression test with both signed zeros and both FLOAT and DOUBLE, covering `=`, `!=`, `<`, `<=`, `>`, `>=` with `+0.0` and `-0.0` predicate values. Make index comparisons treat the two zeros as one SQL equivalence class. Please account for *existing* index files, which already hold distinct encodings: changing only newly written keys would leave old data incorrect; a key-coder-wide format change also needs compatibility review. Verify the chosen fix on both newly built and pre-existing indexes in the affected release lines.
**Useful confirmation from the reporter:** The posted actual results are identified as master output. Please attach the same query results from 4.1.4, plus `EXPLAIN` (or a query profile) for one indexed and one index-disabled query on each version, to confirm which index path each build used. The SQL and master output already provide enough detail for initial root-cause triage; no additional logs are needed to start work.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org