[I] Partition paths format timestamp and binary values differently from iceberg-java [iceberg-rust]

["JeonDaehong (via GitHub)" <gi...@apache.org>]

JeonDaehong opened a new issue, #3349:
URL: https://github.com/apache/iceberg-rust/issues/3349

   ### Apache Iceberg Rust version
   
   v0.10.1
   
   ### Describe the bug
   
   Like #3326 for float/double, `PartitionSpec::partition_to_path` formats identity-partitioned timestamp and binary values differently from iceberg-java. A table written by both gets two directories for one partition value, and the `partitions.<path>` keys in the snapshot summary differ.
   
   | Type | Value | iceberg-rust | iceberg-java |
   | --- | --- | --- | --- |
   | `timestamp` | `2017-11-16T22:31:08.123456` | `2017-11-16 22:31:08.123456` | `2017-11-16T22:31:08.123456` |
   | `timestamp` | `2017-11-16T22:31:00.12` | `2017-11-16 22:31:00.120` | `2017-11-16T22:31:00.12` |
   | `timestamptz` | same instant | `2017-11-16 22:31:08.123456 UTC` | `2017-11-16T22:31:08.123456+00:00` |
   | `timestamp_ns` / `timestamptz_ns` | `...08.123456789` | same as above (space, ` UTC`) | `T`, `+00:00` |
   | `binary` / `fixed` | `01 02 FF` | `0102FF` | `AQL/` |
   
   There are four differences:
   - date/time separator: `T` in Java, a space in Rust
   - zone suffix: `+00:00` in Java, ` UTC` in Rust
   - fraction: Java trims trailing zeros, chrono pads to 3/6/9 digits
   - binary: Java uses standard Base64 with padding, Rust uses uppercase hex
   
   `time`, `date`, `decimal`, `uuid`, `string` (including URL escaping) and `boolean` already match.
   
   The path goes through `Transform::to_human_string` -> `Datum::to_human_string` -> `Display for Datum`.
   
   ### To Reproduce
   
   ```rust
   use iceberg::spec::{Literal, PrimitiveType, Transform, Type};
   
   let human = |t: PrimitiveType, v: Literal| {
       Transform::Identity.to_human_string(&Type::Primitive(t), Some(&v))
   };
   
   assert_eq!(human(PrimitiveType::Timestamp, Literal::timestamp(1510871468123456)), "2017-11-16 22:31:08.123456");
   assert_eq!(human(PrimitiveType::Timestamptz, Literal::timestamptz(1510871468123456)), "2017-11-16 22:31:08.123456 UTC");
   assert_eq!(human(PrimitiveType::Binary, Literal::binary(vec![1, 2, 0xFF])), "0102FF");
   ```
   
   ### Expected behavior
   
   Match iceberg-java: `yyyy-MM-ddTHH:mm:ss` with trailing zeros of the fraction trimmed, `+00:00` for timestamptz, and standard Base64 for binary and fixed. The Java values above come from `PartitionSpec.partitionToPath` in iceberg-api on `main`.
   
   ### Willingness to contribute
   
   I can contribute a fix for this bug independently


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org