[jira] [Created] (SPARK-12467) Get rid of sorting in Row's constructor in pyspark

["Irakli Machabeli (JIRA)" <ji...@apache.org>]

Irakli Machabeli created SPARK-12467:
----------------------------------------

             Summary: Get rid of sorting in Row's constructor in pyspark
                 Key: SPARK-12467
                 URL: https://issues.apache.org/jira/browse/SPARK-12467
             Project: Spark
          Issue Type: Bug
          Components: PySpark, SQL
    Affects Versions: 1.5.2
            Reporter: Irakli Machabeli
            Priority: Minor


Current implementation of Row's __new__ sorts columns by name
First of all there is no obvious reason to sort, second, if one converts dataframe to rdd and than back to dataframe, order of column changes. While this is not  a bug, nevetheless it makes looking at the data really inconvenient.



    def __new__(self, *args, **kwargs):
        if args and kwargs:
            raise ValueError("Can not use both args "
                             "and kwargs to create Row")
        if args:
            # create row class or objects
            return tuple.__new__(self, args)

        elif kwargs:
            # create row objects
            names = sorted(kwargs.keys()) # just get rid of sorting here!!!
            row = tuple.__new__(self, [kwargs[n] for n in names])
            row.__fields__ = names
            return row

        else:
            raise ValueError("No args or kwargs")




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org