You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "L. C. Hsieh (Jira)" <ji...@apache.org> on 2021/07/20 20:35:00 UTC

[jira] [Updated] (SPARK-36210) Preserve column insertion order in Dataset.withColumns

     [ https://issues.apache.org/jira/browse/SPARK-36210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

L. C. Hsieh updated SPARK-36210:
--------------------------------
    Affects Version/s: 3.2.0
                       3.0.3

> Preserve column insertion order in Dataset.withColumns
> ------------------------------------------------------
>
>                 Key: SPARK-36210
>                 URL: https://issues.apache.org/jira/browse/SPARK-36210
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 3.0.3, 3.1.2, 3.2.0
>            Reporter: koert kuipers
>            Assignee: koert kuipers
>            Priority: Minor
>             Fix For: 3.2.0, 3.1.3, 3.0.4
>
>
> Dataset.withColumns uses a Map (columnMap) to store the mapping of column name to column. however this loses the order of the columns. also none of the operations used on the Map (find and filter) benefits from the map's lookup features. so it seems simpler to use a Seq instead, which also preserves the insertion order.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org