You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by Reynold Xin <rx...@databricks.com> on 2015/04/02 02:11:10 UTC
Re: Can I call aggregate UDF in DataFrame?
You totally can.
https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/DataFrame.scala#L792
There is also an attempt at adding stddev here already:
https://github.com/apache/spark/pull/5228
On Thu, Mar 26, 2015 at 12:37 AM, Haopu Wang <HW...@qilinsoft.com> wrote:
> Specifically there are only 5 aggregate functions in class
> org.apache.spark.sql.GroupedData: sum/max/min/mean/count.
>
> Can I plugin a function to calculate stddev?
>
> Thank you!
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
>
RE: Can I call aggregate UDF in DataFrame?
Posted by Haopu Wang <HW...@qilinsoft.com>.
Great! Thank you!
________________________________
From: Reynold Xin [mailto:rxin@databricks.com]
Sent: Thursday, April 02, 2015 8:11 AM
To: Haopu Wang
Cc: user; dev@spark.apache.org
Subject: Re: Can I call aggregate UDF in DataFrame?
You totally can.
https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/
apache/spark/sql/DataFrame.scala#L792
There is also an attempt at adding stddev here already:
https://github.com/apache/spark/pull/5228
On Thu, Mar 26, 2015 at 12:37 AM, Haopu Wang <HW...@qilinsoft.com>
wrote:
Specifically there are only 5 aggregate functions in class
org.apache.spark.sql.GroupedData: sum/max/min/mean/count.
Can I plugin a function to calculate stddev?
Thank you!
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org
RE: Can I call aggregate UDF in DataFrame?
Posted by Haopu Wang <HW...@qilinsoft.com>.
Great! Thank you!
________________________________
From: Reynold Xin [mailto:rxin@databricks.com]
Sent: Thursday, April 02, 2015 8:11 AM
To: Haopu Wang
Cc: user; dev@spark.apache.org
Subject: Re: Can I call aggregate UDF in DataFrame?
You totally can.
https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/
apache/spark/sql/DataFrame.scala#L792
There is also an attempt at adding stddev here already:
https://github.com/apache/spark/pull/5228
On Thu, Mar 26, 2015 at 12:37 AM, Haopu Wang <HW...@qilinsoft.com>
wrote:
Specifically there are only 5 aggregate functions in class
org.apache.spark.sql.GroupedData: sum/max/min/mean/count.
Can I plugin a function to calculate stddev?
Thank you!
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org