[jira] [Created] (SPARK-30781) Missing SortedMap type in pyspark

["Mateusz User (Jira)" <ji...@apache.org>]

Mateusz User created SPARK-30781:
------------------------------------

             Summary: Missing SortedMap type in pyspark
                 Key: SPARK-30781
                 URL: https://issues.apache.org/jira/browse/SPARK-30781
             Project: Spark
          Issue Type: Improvement
          Components: PySpark
    Affects Versions: 2.4.5
            Reporter: Mateusz User


Currently there is only MapType in pyspark API, which does not keep order of key-value map.

 

*SortedMapType* would fill this gap: a map with sorted key-value pairs (like TreeMap in Java).

 

For example:

*SortedMapType* would be very useful when user wants to persist row from DataFrame into Mongo-db.

row which consist a column with values of an ordered map type:

col : [1 -> 22, 2 -> 16, 3 -> 25]

will be persisted as following json:

{

  "1": 22,

  "2": 16,

  "3": 25

}

 

instead of MapType which results in currently:

{

  "2": 16,

  "1": 22,

  "3": 25

}

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org