You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Louis Yang (JIRA)" <ji...@apache.org> on 2019/05/17 06:05:00 UTC

[jira] [Created] (SPARK-27756) Add a shape property to DataFrame in pyspark

Louis Yang created SPARK-27756:
----------------------------------

             Summary: Add a shape property to DataFrame in pyspark
                 Key: SPARK-27756
                 URL: https://issues.apache.org/jira/browse/SPARK-27756
             Project: Spark
          Issue Type: Wish
          Components: PySpark
    Affects Versions: 2.4.3
            Reporter: Louis Yang


It will be great if PySpark DataFrame can support simple shape attribute which returns the number of rows and columns similar to what [pandas|[https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.shape.html]] has.

We can add the following to the DataFrame class

 
{code:java}
@property
def shape(self):
    return (self.count(), len(self.columns)){code}
Then user in python can simply do

 
{code:java}
>>> df.shape
(10000, 20){code}
to know the most fundamental information of a dataframe when working interactively.

 

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org