You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Arunkumar Pillai <ar...@gmail.com> on 2016/02/04 10:22:33 UTC
Need to user univariate summary stats
Hi
I'm currently using query
sqlContext.sql("SELECT MAX(variablesArray) FROM " + tableName)
to extract mean max min.
is there any better optimized way ?
In the example i saw df.groupBy("key").agg(skewness("a"), kurtosis("a"))
But i don't have key anywhere in the data.
How to extract the univariate summary stats from df. please help
--
Thanks and Regards
Arun
RE: Need to user univariate summary stats
Posted by Lohith Samaga M <Lo...@mphasis.com>.
Hi Arun,
You can do df.agg(max(,,), min(..)).
Best regards / Mit freundlichen Grüßen / Sincères salutations
M. Lohith Samaga
From: Arunkumar Pillai [mailto:arunkumar1111@gmail.com]
Sent: Thursday, February 04, 2016 14.53
To: user@spark.apache.org
Subject: Need to user univariate summary stats
Hi
I'm currently using query
sqlContext.sql("SELECT MAX(variablesArray) FROM " + tableName)
to extract mean max min.
is there any better optimized way ?
In the example i saw df.groupBy("key").agg(skewness("a"), kurtosis("a"))
But i don't have key anywhere in the data.
How to extract the univariate summary stats from df. please help
--
Thanks and Regards
Arun
Information transmitted by this e-mail is proprietary to Mphasis, its associated companies and/ or its customers and is intended
for use only by the individual or entity to which it is addressed, and may contain information that is privileged, confidential or
exempt from disclosure under applicable law. If you are not the intended recipient or it appears that this mail has been forwarded
to you without proper authority, you are notified that any use or dissemination of this information in any manner is strictly
prohibited. In such cases, please notify us immediately at mailmaster@mphasis.com and delete this mail from your records.