You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by David Thomas <dt...@gmail.com> on 2014/03/03 04:04:37 UTC
Help with groupByKey
I have an RDD of (K, Array[V]) pairs.
For example: ((key1, (1,2,3)), (key2, (3,2,4)), (key1, (4,3,2)))
How can I do a groupByKey such that I get back an RDD of the form (K,
Array[V]) pairs.
Ex: ((key1, (1,2,3,4,3,2)), (key2, (3,2,4)))
Re: Help with groupByKey
Posted by Cheng Lian <rh...@gmail.com>.
Actually it should be rdd.reduceByKey(_ ++ _)
On Mar 3, 2014, at 11:56, Andrew Ash <an...@andrewash.com> wrote:
> rdd.reduceByKey(_+_) using list concatenation?
>
> Sent from my mobile phone
>
> On Mar 2, 2014 7:05 PM, "David Thomas" <dt...@gmail.com> wrote:
> I have an RDD of (K, Array[V]) pairs.
>
> For example: ((key1, (1,2,3)), (key2, (3,2,4)), (key1, (4,3,2)))
>
> How can I do a groupByKey such that I get back an RDD of the form (K, Array[V]) pairs.
>
> Ex: ((key1, (1,2,3,4,3,2)), (key2, (3,2,4)))
Re: Help with groupByKey
Posted by Cheng Lian <rh...@gmail.com>.
Actually it should be rdd.reduceByKey(_ ++ _)
On Mar 3, 2014, at 11:56, Andrew Ash <an...@andrewash.com> wrote:
> rdd.reduceByKey(_+_) using list concatenation?
>
> Sent from my mobile phone
>
> On Mar 2, 2014 7:05 PM, "David Thomas" <dt...@gmail.com> wrote:
> I have an RDD of (K, Array[V]) pairs.
>
> For example: ((key1, (1,2,3)), (key2, (3,2,4)), (key1, (4,3,2)))
>
> How can I do a groupByKey such that I get back an RDD of the form (K, Array[V]) pairs.
>
> Ex: ((key1, (1,2,3,4,3,2)), (key2, (3,2,4)))
Re: Help with groupByKey
Posted by Andrew Ash <an...@andrewash.com>.
rdd.reduceByKey(_+_) using list concatenation?
Sent from my mobile phone
On Mar 2, 2014 7:05 PM, "David Thomas" <dt...@gmail.com> wrote:
> I have an RDD of (K, Array[V]) pairs.
>
> For example: ((key1, (1,2,3)), (key2, (3,2,4)), (key1, (4,3,2)))
>
> How can I do a groupByKey such that I get back an RDD of the form (K,
> Array[V]) pairs.
>
> Ex: ((key1, (1,2,3,4,3,2)), (key2, (3,2,4)))
>