[jira] [Created] (SPARK-57744) Types Framework - migrate vectorized column population (ColumnVectorUtils.appendValue and Columnar row getters)

["Max Gekk (Jira)" <ji...@apache.org>]

Max Gekk created SPARK-57744:
--------------------------------

             Summary: Types Framework - migrate vectorized column population (ColumnVectorUtils.appendValue and Columnar row getters)
                 Key: SPARK-57744
                 URL: https://issues.apache.org/jira/browse/SPARK-57744
             Project: Spark
          Issue Type: Sub-task
          Components: SQL
    Affects Versions: 4.3.0
            Reporter: Max Gekk


h2. What
Migrate the vectorized-reader column-population code paths to dispatch through the Types Framework (TypeOps / PhysicalDataType) instead of hand-rolled per-type {{instanceof}} chains, so any framework-registered type (TimeType today, future types tomorrow) is handled without per-type edits.

h2. Where
* {{sql/core .../execution/vectorized/ColumnVectorUtils.java}} -- {{appendValue(WritableColumnVector, DataType, Object)}} (the {{toBatch}} path): route the external -> internal conversion through {{TypeOps.toCatalyst}} and append according to {{getPhysicalType}}, replacing the tail of the explicit {{instanceof}} chain.
* {{sql/catalyst .../vectorized/ColumnarRow.java}} and {{ColumnarBatchRow.java}} -- {{get(int, DataType)}}: delegate to {{SpecializedGettersReader.read(...)}} (which dispatches on {{PhysicalDataType}}), mirroring {{ColumnarArray.get}}, which already does this.

h2. Why
The populate path and the {{copy()}} methods already dispatch on {{PhysicalDataType}} and therefore support framework types for free, and {{ColumnarArray.get}} already delegates to {{SpecializedGettersReader}}. {{appendValue}} and the two row getters are the only remaining hand-maintained {{instanceof}} chains in this layer. Bringing them onto the framework removes duplicated dispatch and means new framework types need no edits here.

h2. Relationship
* Parent: SPARK-53504 (Types Framework umbrella).
* Related: SPARK-56076 (Phase 1b - Generic Row Access) overlaps with the {{get(int, DataType)}} delegation.
* Follows: SPARK-57570, which added an explicit TimeType case to {{appendValue}} as a focused fix; this task generalizes that case through the framework.

h2. Acceptance criteria
* {{appendValue}} and the columnar row getters no longer special-case individual framework types; TIME (and other framework types) flow through automatically via {{PhysicalDataType}} / {{TypeOps}}.
* Behavior preserved for existing types. Call out in review: delegating the getters changes the unsupported-type error code ({{_LEGACY_ERROR_TEMP_3152}} / {{_LEGACY_ERROR_TEMP_3155}} -> {{_LEGACY_ERROR_TEMP_3131}}), and {{ColumnarRow.get}} gains {{CalendarIntervalType}} support it currently lacks.
* {{ColumnVectorUtilsSuite}} / {{ColumnarBatchSuite}} continue to pass; the per-precision TIME {{testVector}} in {{ColumnarBatchSuite}} can then assert via {{batchRow.get(0, dt)}}.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org