You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@paimon.apache.org by "QuakeWang (via GitHub)" <gi...@apache.org> on 2026/05/20 16:07:30 UTC
[PR] feat: expose timestamp nanos as Arrow timestamp(ns) [paimon-mosaic]
QuakeWang opened a new pull request, #21:
URL: https://github.com/apache/paimon-mosaic/pull/21
## Problem
Mosaic exposes TIMESTAMP precision > 6 as a legacy Arrow Struct (`millis`, `nanos_of_milli`) instead of standard Arrow timestamp(ns). This leaks the internal 12-byte storage representation into the Arrow API and loses timezone metadata for TIMESTAMP_LTZ precision > 6.
## Fix
Keep the Mosaic file format unchanged, but expose high-precision timestamps through standard Arrow timestamp(ns):
- Deserialize precision > 6 timestamps as `Timestamp(Nanosecond, timezone)`.
- Read and write `TimestampNanosecondArray` using the existing `millis + nanos_of_milli` storage encoding.
- Preserve TIMESTAMP_LTZ timezone metadata.
- Keep legacy Struct input compatibility with validation.
- Add Rust, Python, C++, and Java timestamp(ns) roundtrip coverage.
Precision 7/8 is normalized to Arrow timestamp(ns), since Arrow does not preserve decimal timestamp precision separately.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat: expose timestamp nanos as Arrow timestamp(ns) [paimon-mosaic]
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi merged PR #21:
URL: https://github.com/apache/paimon-mosaic/pull/21
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat: expose timestamp nanos as Arrow timestamp(ns) [paimon-mosaic]
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi commented on PR #21:
URL: https://github.com/apache/paimon-mosaic/pull/21#issuecomment-4503928633
+1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org