[I] Equality delete with a composite key containing a uuid is never attached to FileScanTask.deletes() [iceberg]

["Neuw84 (via GitHub)" <gi...@apache.org>]

Neuw84 opened a new issue, #18619:
URL: https://github.com/apache/iceberg/issues/18619

   ### Apache Iceberg version
   
   1.12.0 (iceberg-spark-runtime-4.1_2.13, Spark 4.1.3)
   
   ### Query engine
   
   Spark
   
   ### Please describe the bug 🐞
   
   An equality delete whose key has more than one field, one of them a `uuid`, is never applied: no data file's `FileScanTask.deletes()` contains it, so zero rows are deleted.
   
   **Repro**
   
   1. v2 table `(id long, u uuid, n int, a int, s string)`, `write.delete.mode=merge-on-read`, 1000 rows, `u` deterministic per `id`, `n = id % 5`.
   2. Write one Parquet equality delete file with equality field ids `(u, n)` and keys `(uuid_of(i), i % 5)` for `i` = 0, 11, 22, ...; commit with `newRowDelta().addDeletes(...)`.
   3. `SELECT id FROM t` keeps all 1000 rows (expected 909).
   
   Controls on an identically shaped table, written by the same helper:
   - composite key `(a, n)` (int, int): 909 rows kept, correct;
   - composite key `(s, n)` (string, int): 909 rows kept, correct;
   - single-field key `(u)`: correct.
   
   The loaded delete set itself is correct (`BaseDeleteLoader.loadEqualityDeletes` gives the right distinct count, elements are `(java.util.UUID, Integer)`, and `contains` returns `true` for a key that should match). But instrumenting the scan shows the `(u, n)` delete file is **not attached to any data file task**: no `FileScanTask.deletes()` of that table lists it, while the `(a, n)` and `(s, n)` deletes are listed for their tables. So the reader never receives the delete.
   
   **Where to look**
   
   Scan planning (`DeleteFileIndex` / `ManifestGroup`) when matching an equality delete whose `equality_ids` include a `uuid` field together with other fields -- e.g. how the delete file's lower/upper bounds for the uuid column are compared with the data file's bounds when deciding whether the delete can apply.
   
   Found while building a native Iceberg scan for Spark; since the delete never reaches `FileScanTask.deletes()`, no reader built on Iceberg's planning can apply it.
   
   ### Willingness to contribute
   
   - [ ] I can contribute a fix for this bug independently
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org

Re: [I] Equality delete with a composite key containing a uuid is never attached to FileScanTask.deletes() [iceberg]

["Neuw84 (via GitHub)" <gi...@apache.org>]

Neuw84 commented on issue #18619:
URL: https://github.com/apache/iceberg/issues/18619#issuecomment-6358363071

   Opened #18622 with the fix: `UUIDType` is now ordered as an unsigned 128-bit value (`Long.compareUnsigned` on MSB then LSB), matching the 16-byte big-endian order the bounds use. The signed `UUID.compareTo` was inverting a data file's stored [lower, upper] uuid bounds when they straddle the signed boundary, so `DeleteFileIndex.canContainEqDeletesForFile` dropped the delete. Includes a `TestComparators` boundary-crossing case and an end-to-end `DeleteFileIndexTestBase` test asserting a `(uuid, int)` equality delete is attached.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org