You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "jaideeppyne (via GitHub)" <gi...@apache.org> on 2026/08/31 12:28:57 UTC
[I] truncate transform rejects binary values in transform_literal and LargeBinary arrays [iceberg-rust]
jaideeppyne opened a new issue, #3119:
URL: https://github.com/apache/iceberg-rust/issues/3119
### Apache Iceberg Rust version
main (0.10.0)
### Describe the bug
The spec lists `binary` as a valid source type for `truncate[W]` ("Partition Transforms": `truncate[W]` source types are `int`, `long`, `decimal`, `string`, `binary`; "Truncate Transform Details": `binary` / `L`, length / `v.subarray(0, L)`). `Transform::result_type` agrees and accepts `PrimitiveType::Binary`, so a `truncate[W]` partition field on a binary column is a legal spec.
But both halves of `TransformFunction for Truncate` reject it:
1. `transform_literal` has no `PrimitiveLiteral::Binary` arm, so every predicate projection on such a field returns `FeatureUnsupported`. `InclusiveProjection` (used in `scan/cache.rs`) propagates that error, so any scan with a filter on that column fails instead of pruning.
2. `transform` handles `DataType::Binary` but not `DataType::LargeBinary`, and `schema_to_arrow_schema` maps Iceberg `binary` to `LargeBinary` (`crates/iceberg/src/arrow/schema.rs`, `PrimitiveType::Binary => DataType::LargeBinary`). So the array path fails on exactly the arrow type this crate produces for a binary column.
`Truncate::truncate_binary` already exists and is correct; it is just not wired to either entry point. pyiceberg and iceberg-go both implement truncate on binary.
### To Reproduce
```rust
let t = Transform::Truncate(3);
// 1. literal / projection path
let pred = /* binary column `b`, `b <= x'0102030405'` */;
t.project("pb", &pred);
// Err(FeatureUnsupported => Unsupported data type for truncate transform: Binary)
t.strict_project("pb", &pred);
// Err(FeatureUnsupported => Unsupported data type for truncate transform: Binary)
// 2. array path
let f = create_transform_function(&t).unwrap();
f.transform(Arc::new(BinaryArray::from_iter_values([vec![1u8,2,3,4,5]])));
// Ok([1, 2, 3])
f.transform(Arc::new(LargeBinaryArray::from_iter_values([vec![1u8,2,3,4,5]])));
// Err(FeatureUnsupported => Unsupported data type for truncate transform: LargeBinary)
```
### Expected behavior
`truncate[W]` on a binary column truncates to the first `W` bytes, in both the array and the literal path, so partition values can be computed and predicates can be projected.
### Willingness to contribute
I can contribute a fix for this bug independently
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org