You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Max Gekk (Jira)" <ji...@apache.org> on 2026/07/01 14:37:00 UTC

[jira] [Created] (SPARK-57833) Support nanosecond-precision timestamps in the timestampadd and timestampdiff functions

Max Gekk created SPARK-57833:
--------------------------------

             Summary: Support nanosecond-precision timestamps in the timestampadd and timestampdiff functions
                 Key: SPARK-57833
                 URL: https://issues.apache.org/jira/browse/SPARK-57833
             Project: Spark
          Issue Type: Sub-task
          Components: SQL
    Affects Versions: 4.3.0
            Reporter: Max Gekk


This sub-task is part of the umbrella SPARK-56822 (timestamps with nanosecond precision).

h2. Problem
{{TimestampAdd}} (datetimeExpressions.scala ~L4278-4288) types its timestamp arg as {{AnyTimestampType}} (excludes nanosecond) and evaluates {{DateTimeUtils.timestampAdd(unit, Long, ...)}}; {{TimestampDiff}} (~L4367-4380) types both args as {{TimestampType}} and operates on {{Long}}. Nanosecond args raise {{UNEXPECTED_INPUT_TYPE}}. This is distinct from SPARK-57501, which added the {{+}} / {{-}} day-time-interval OPERATOR ({{TimestampAddInterval}}), not these named functions.

h2. Goal
{{timestampadd(unit, qty, ts_nanos)}} returns a nanosecond timestamp (same type/precision) with {{nanosWithinMicro}} preserved for unit >= MICROSECOND (and correctly rescaled for a NANOSECOND unit if added); {{timestampdiff(unit, start, end)}} accepts nanosecond endpoints and, for sub-microsecond units, computes the difference using the nanosecond remainder.

h2. Scope
Extend both {{inputTypes}} to accept {{AnyTimestampNanoType}}; add {{TimestampNanosVal}} eval/codegen; add {{DateTimeUtils}} helpers for nanosecond add/diff; decide and document NANOSECOND-unit handling.

h2. Acceptance criteria
* add/diff over NTZ/LTZ nanosecond timestamps preserve/consume the nanosecond remainder; microsecond behavior unchanged.

h2. Testing
{{DateExpressionsSuite}}; nanos golden files.

h2. Dependencies
None hard - coordinate with the overflow/range-hardening sub-task (overflow-safe add/diff helpers).




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org