You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Max Gekk (Jira)" <ji...@apache.org> on 2026/07/01 14:37:00 UTC
[jira] [Created] (SPARK-57833) Support nanosecond-precision timestamps in the timestampadd and timestampdiff functions
Max Gekk created SPARK-57833:
--------------------------------
Summary: Support nanosecond-precision timestamps in the timestampadd and timestampdiff functions
Key: SPARK-57833
URL: https://issues.apache.org/jira/browse/SPARK-57833
Project: Spark
Issue Type: Sub-task
Components: SQL
Affects Versions: 4.3.0
Reporter: Max Gekk
This sub-task is part of the umbrella SPARK-56822 (timestamps with nanosecond precision).
h2. Problem
{{TimestampAdd}} (datetimeExpressions.scala ~L4278-4288) types its timestamp arg as {{AnyTimestampType}} (excludes nanosecond) and evaluates {{DateTimeUtils.timestampAdd(unit, Long, ...)}}; {{TimestampDiff}} (~L4367-4380) types both args as {{TimestampType}} and operates on {{Long}}. Nanosecond args raise {{UNEXPECTED_INPUT_TYPE}}. This is distinct from SPARK-57501, which added the {{+}} / {{-}} day-time-interval OPERATOR ({{TimestampAddInterval}}), not these named functions.
h2. Goal
{{timestampadd(unit, qty, ts_nanos)}} returns a nanosecond timestamp (same type/precision) with {{nanosWithinMicro}} preserved for unit >= MICROSECOND (and correctly rescaled for a NANOSECOND unit if added); {{timestampdiff(unit, start, end)}} accepts nanosecond endpoints and, for sub-microsecond units, computes the difference using the nanosecond remainder.
h2. Scope
Extend both {{inputTypes}} to accept {{AnyTimestampNanoType}}; add {{TimestampNanosVal}} eval/codegen; add {{DateTimeUtils}} helpers for nanosecond add/diff; decide and document NANOSECOND-unit handling.
h2. Acceptance criteria
* add/diff over NTZ/LTZ nanosecond timestamps preserve/consume the nanosecond remainder; microsecond behavior unchanged.
h2. Testing
{{DateExpressionsSuite}}; nanos golden files.
h2. Dependencies
None hard - coordinate with the overflow/range-hardening sub-task (overflow-safe add/diff helpers).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org