[jira] [Created] (SPARK-57555) Support the TIME data type in the built-in JDBC data source

["Max Gekk (Jira)" <ji...@apache.org>]

Max Gekk created SPARK-57555:
--------------------------------

             Summary: Support the TIME data type in the built-in JDBC data source
                 Key: SPARK-57555
                 URL: https://issues.apache.org/jira/browse/SPARK-57555
             Project: Spark
          Issue Type: Sub-task
          Components: SQL
    Affects Versions: 4.3.0
            Reporter: Max Gekk


h2. What

Add native {{TIME}} support to the built-in JDBC data source so that SQL {{TIME}} columns
in external databases map to Spark's {{TimeType}} on read, and {{TimeType}} columns map to
SQL {{TIME}} on write.

h2. Why

This is the core motivation of the SPIP (SPARK-51162, Q5): simplify migration from
PostgreSQL, Snowflake, Redshift, Teradata, DB2, etc. Today JDBC cannot represent TIME:
* On read, {{JdbcUtils.getCatalystType}} maps {{java.sql.Types.TIME}} to a *timestamp*
  type, not {{TimeType}}.
* On write, {{JdbcUtils.getJdbcType}} has no {{case TimeType}}.
* No dialect maps TIME.

h2. Scope

* {{JdbcUtils.getCatalystType}}: map {{java.sql.Types.TIME}} to {{TimeType}} (gated by the
  TIME feature flag and a JDBC option for back-compat).
* {{JdbcUtils.getJdbcType}}: add {{case TimeType => JdbcType("TIME", java.sql.Types.TIME)}}.
* Value conversion in {{JdbcUtils}} (read {{ResultSet.getTime}}/getObject -> nanos-of-day;
  write nanos-of-day -> {{java.sql.Time}}/{{LocalTime}}), preserving sub-second precision
  where the driver allows.
* Per-dialect overrides where the SQL type name/precision differs
  (PostgreSQL, MySQL, MS SQL Server, Oracle, DB2, H2) in {{JdbcDialects}} and dialect
  classes. Consider splitting per-dialect work into sub-tasks.

h2. Compatibility

* A back-compat switch so existing pipelines that read TIME-as-timestamp do not break;
  document the new behavior.

h2. Acceptance criteria

* Read/write round-trip of TIME columns against the JDBC integration-test databases
  (Postgres/MySQL/MS SQL/Oracle/DB2) preserves values.
* Schema inference reports {{TimeType}} for TIME columns.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org