[PR] fix(variant): render DATE and TIMESTAMP in to_json and string casts like Java [paimon-rust]

["jackylee-ch (via GitHub)" <gi...@apache.org>]

jackylee-ch opened a new pull request, #1037:
URL: https://github.com/apache/paimon-rust/pull/1037

   Variant `DATE`, `TIMESTAMP` and `TIMESTAMP_NTZ` values were serialized as the
   raw backing integer. `GenericVariant::to_json` emitted a date as `"19723"` and
   a timestamp as `"1704067200000000"`, and the variant→VARCHAR extraction
   (`cast_variant_to_string`) and the DataFusion `CAST(variant AS STRING)` /
   `variant_get(.., 'string')` path did the same. The output is silently wrong
   rather than an error, so a Variant column written by Spark/Flink/Paimon (or
   assembled internally) round-trips to JSON/string as a bare integer. The three
   paths also would have diverged the moment one was fixed.
   
   Java `GenericVariantUtil` renders `DATE` via `LocalDate.ofEpochDay(..)` and
   `TIMESTAMP`/`TIMESTAMP_NTZ` via an ISO formatter (`ISO_LOCAL_DATE` + `' '` +
   `ISO_LOCAL_TIME`), anchoring NTZ to UTC and the zoned `TIMESTAMP` to the
   caller's zone with a `+HH:MM` offset. This adds shared `format_variant_date` /
   `format_variant_timestamp` helpers and routes all three paths through them, so
   they can no longer disagree. `DATE` and `TIMESTAMP_NTZ` match Java exactly; the
   fractional part uses 0/3/6 digits like Java's ISO time formatter, consistent
   with the existing `partition_utils` timestamp formatting.
   
   One deliberate divergence: `GenericVariant::to_json` takes no `ZoneId`, so a
   local-zoned `TIMESTAMP` is rendered in UTC rather than a session zone. Matching
   Java's zone-aware rendering would require threading a zone through the public
   API and is left as a follow-up. Non-finite float rendering is already handled
   and is unchanged here.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] fix(variant): render DATE and TIMESTAMP in to_json and string casts like Java [paimon-rust]

["jackylee-ch (via GitHub)" <gi...@apache.org>]

jackylee-ch commented on PR #1037:
URL: https://github.com/apache/paimon-rust/pull/1037#issuecomment-5995206896

   Withdrawing this PR. It's a parity-driven change with no backing issue or demonstrated impact; I'd rather keep the review queue focused on changes with a concrete use case. Will re-propose via an issue if a real need comes up.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] fix(variant): render DATE and TIMESTAMP in to_json and string casts like Java [paimon-rust]

["jackylee-ch (via GitHub)" <gi...@apache.org>]

jackylee-ch closed pull request #1037: fix(variant): render DATE and TIMESTAMP in to_json and string casts like Java
URL: https://github.com/apache/paimon-rust/pull/1037


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org