[jira] [Assigned] (SPARK-57457) Support nanosecond-precision timestamp types in the CSV datasource (v1 and v2)

["Max Gekk (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-57457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Max Gekk reassigned SPARK-57457:
--------------------------------

    Assignee: Vinod KC

> Support nanosecond-precision timestamp types in the CSV datasource (v1 and v2)
> ------------------------------------------------------------------------------
>
>                 Key: SPARK-57457
>                 URL: https://issues.apache.org/jira/browse/SPARK-57457
>             Project: Spark
>          Issue Type: Sub-task
>          Components: SQL
>    Affects Versions: 4.3.0
>            Reporter: Max Gekk
>            Assignee: Vinod KC
>            Priority: Major
>              Labels: pull-request-available
>
> Umbrella: SPARK-56822 (Timestamps with nanosecond precision).
> Add read and write support for the nanosecond-capable timestamp types TIMESTAMP_NTZ(p) and TIMESTAMP_LTZ(p) (p in 7-9) so this datasource reaches parity with the microsecond TimestampType / TimestampNTZType. Remove the SPARK-57166 rejection guardrail (supportDataType / supportsDataType) once read and write are implemented and tested, and update FileBasedDataSourceSuite accordingly. Cover precisions 7-9 for both NTZ and LTZ.
> Scope:
> - Read: UnivocityParser nanos cases via TimestampFormatter parseNanos / parseWithoutTimeZoneNanos with the column precision.
> - Write: UnivocityGenerator via formatNanos / formatWithoutTimeZoneNanos.
> - Schema inference: CSVInferSchema (keep inferring microsecond by default).
> - Guardrails: CSVFileFormat (v1), CSVTable (v2).



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org