[PR] fix(arrow): scale INTEGER/BIGINT epoch seconds evolved to TIMESTAMP [paimon-rust]

["jackylee-ch (via GitHub)" <gi...@apache.org>]

jackylee-ch opened a new pull request, #924:
URL: https://github.com/apache/paimon-rust/pull/924

   When a column is evolved to `TIMESTAMP` (e.g. `ALTER TABLE ... ALTER COLUMN c ...`), old data files are reconciled through `evolve_column`, which fell through to Arrow's `cast(Int64, Timestamp(unit))`. That reinterprets the raw integer in the column's unit, but Java `NumericPrimitiveToTimestamp` reads an INTEGER/BIGINT column as epoch **seconds** (`value * 1000` millis, UTC). So a Java-written BIGINT epoch column evolved to `TIMESTAMP(6)` was read 10^6× too small (~1970), silently.
   
   The fix scales the value into the target unit before reinterpreting. Scoped to INTEGER/BIGINT → TIMESTAMP without local time zone, matching Java's rule: its `create()` returns null for the other numeric sources, and its local-zone target uses `ZoneId.systemDefault()` (environment dependent) — both stay on the existing Arrow path.
   
   Added a unit test asserting per-precision scaling (0/3/6) and null preservation; it fails on the pre-fix raw reinterpret.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] fix(arrow): scale INTEGER/BIGINT epoch seconds evolved to TIMESTAMP [paimon-rust]

["JingsongLi (via GitHub)" <gi...@apache.org>]

JingsongLi merged PR #924:
URL: https://github.com/apache/paimon-rust/pull/924


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org