You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by sachin Singh <sa...@gmail.com> on 2014/12/23 10:47:41 UTC
JavaRDD (Data Aggregation) based on key
Hi,
I have a csv file having fields as a,b,c .
I want to do aggregation(sum,average..) based on any field(a,b or c) as per
user input,
using Apache Spark Java API,Please Help Urgent!
Thanks in advance,
Regards
Sachin
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/JavaRDD-Data-Aggregation-based-on-key-tp20828.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org
Re: JavaRDD (Data Aggregation) based on key
Posted by Rishi Yadav <ri...@infoobjects.com>.
One approach is to first transform this RDD into a PairRDD by taking the
field you are going to do aggregation on as key
On Tue, Dec 23, 2014 at 1:47 AM, sachin Singh <sa...@gmail.com>
wrote:
> Hi,
> I have a csv file having fields as a,b,c .
> I want to do aggregation(sum,average..) based on any field(a,b or c) as per
> user input,
> using Apache Spark Java API,Please Help Urgent!
>
> Thanks in advance,
>
> Regards
> Sachin
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/JavaRDD-Data-Aggregation-based-on-key-tp20828.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
>
Re: JavaRDD (Data Aggregation) based on key
Posted by Jon Chase <jo...@gmail.com>.
Have a look at RDD.groupBy(...) and reduceByKey(...)
On Tue, Dec 23, 2014 at 4:47 AM, sachin Singh <sa...@gmail.com>
wrote:
> Hi,
> I have a csv file having fields as a,b,c .
> I want to do aggregation(sum,average..) based on any field(a,b or c) as per
> user input,
> using Apache Spark Java API,Please Help Urgent!
>
> Thanks in advance,
>
> Regards
> Sachin
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/JavaRDD-Data-Aggregation-based-on-key-tp20828.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
>