You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "stevomitric (via GitHub)" <gi...@apache.org> on 2026/09/07 15:00:26 UTC
[PR] [SPARK-59300][SQL] Support nanosecond-precision timestamps in from_utc_timestamp and to_utc_timestamp [spark]
stevomitric opened a new pull request, #58576:
URL: https://github.com/apache/spark/pull/58576
### What changes were proposed in this pull request?
Extend from_utc_timestamp / to_utc_timestamp (the shared UTCTimestamp trait) to accept nanosecond-precision timestamps (TIMESTAMP_LTZ(p) / TIMESTAMP_NTZ(p), p in [7, 9]) alongside microsecond TimestampType. The result keeps the source's exact nanosecond type/family; a zone conversion shifts only epochMicros (offsets are whole seconds), so the sub-microsecond nanosWithinMicro remainder is preserved. Micros path unchanged. Follows date_trunc (SPARK-57821) and convert_timezone (SPARK-57818). Part of SPARK-56822.
### Why are the changes needed?
These functions typed their argument as TimestampType only, narrowing a nanosecond input to microseconds and dropping the sub-microsecond digits. Nanosecond support is a P1 expression item.
### Does this PR introduce _any_ user-facing change?
Yes, behind the preview flag spark.sql.timestampNanosTypes.enabled (unreleased feature; no released version affected). The functions now return the same nanosecond type with the remainder preserved.
### How was this patch tested?
New DateExpressionsSuite case (both functions x NTZ/LTZ x p in {7,8,9}; interpreted + both codegen branches
### Was this patch authored or co-authored using generative AI tooling?
Co-authored-by: Claude Code 4.8
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-59300][SQL] Support nanosecond-precision timestamps in from_utc_timestamp and to_utc_timestamp [spark]
Posted by "uros-b (via GitHub)" <gi...@apache.org>.
uros-b commented on PR #58576:
URL: https://github.com/apache/spark/pull/58576#issuecomment-5617411243
Thank you @stevomitric!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-59300][SQL] Support nanosecond-precision timestamps in from_utc_timestamp and to_utc_timestamp [spark]
Posted by "uros-b (via GitHub)" <gi...@apache.org>.
uros-b closed pull request #58576: [SPARK-59300][SQL] Support nanosecond-precision timestamps in from_utc_timestamp and to_utc_timestamp
URL: https://github.com/apache/spark/pull/58576
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-59300][SQL] Support nanosecond-precision timestamps in from_utc_timestamp and to_utc_timestamp [spark]
Posted by "uros-b (via GitHub)" <gi...@apache.org>.
uros-b commented on PR #58576:
URL: https://github.com/apache/spark/pull/58576#issuecomment-5617433177
**Merge Summary:**
- merged into master https://github.com/apache/spark/commit/1db290ce3678fac9e160ab51e2fc6776f2ddf15d
- merged into branch-4.x https://github.com/apache/spark/commit/863df7e32945d8a9a90afb6a6855b2fd9f39c4f1
*Posted by `merge_spark_pr.py`*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-59300][SQL] Support nanosecond-precision timestamps in from_utc_timestamp and to_utc_timestamp [spark]
Posted by "stevomitric (via GitHub)" <gi...@apache.org>.
stevomitric commented on PR #58576:
URL: https://github.com/apache/spark/pull/58576#issuecomment-5584787855
cc @uros-b PTAL.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org