[jira] [Created] (SPARK-57812) Support catalog column-statistics serialization for nanosecond-precision timestamps

["Max Gekk (Jira)" <ji...@apache.org>]

Max Gekk created SPARK-57812:
--------------------------------

             Summary: Support catalog column-statistics serialization for nanosecond-precision timestamps
                 Key: SPARK-57812
                 URL: https://issues.apache.org/jira/browse/SPARK-57812
             Project: Spark
          Issue Type: Sub-task
          Components: SQL
    Affects Versions: 4.3.0
            Reporter: Max Gekk


This sub-task is part of the umbrella SPARK-56822 (timestamps with nanosecond precision).

h2. Problem
{{CatalogColumnStat.toExternalString}} and {{fromExternalString}} (catalyst/catalog/interface.scala ~L1040-1081) handle only {{TimestampType}} / {{TimestampNTZType}}; nanosecond min/max hit {{columnStatisticsSerializationNotSupportedError}} / {{columnStatisticsDeserializationNotSupportedError}}. In-query stat aggregation already works ({{CommandUtils.statExprs}} uses {{case _: DatetimeType}}); the failure is on persisting/reading catalog stats.

h2. Goal
Serialize and deserialize nanosecond timestamp min/max column statistics to/from the catalog at full precision.

h2. Scope
Add nanosecond arms to {{toExternalString}} / {{fromExternalString}} using the nanos-aware formatter/parser (a round-trippable text form including the fractional digits).

h2. Acceptance criteria
* {{ANALYZE TABLE ... FOR COLUMNS <nanos_col>}} persists and reloads min/max without error; values round-trip.

h2. Testing
{{StatisticsCollectionSuite}} / {{StatisticsSuite}}.

h2. Dependencies
None - independent. Enables the CBO FilterEstimation sub-task (which needs persisted nanosecond min/max).




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org