how to convert RDD to PairRDDFunctions ?

[Konstantin Kudryavtsev <ku...@gmail.com>]

Hi all,

sorry for fooly question, but how can I get PairRDDFunctions RDD? I'm doing
it to perform leftOuterJoin aftewards

currently I do in this was (it seems incorrect):
val parRDD = new PairRDDFunctions( oldRdd.map(i => (i.key, i)) )

I guess this constructor is definitely wrong...


Thank you,
Konstantin Kudryavtsev

Re: how to convert RDD to PairRDDFunctions ?

[Mark Hamstra <ma...@clearstorydata.com>]

See Working with Key-Value Pairs
<http://spark.apache.org/docs/latest/programming-guide.html>.  In
particular: "In Scala, these operations are automatically available on RDDs
containing Tuple2 objects (the built-in tuples in the language, created by
simply writing (a, b)), as long as you import
org.apache.spark.SparkContext._ in your program to enable Spark’s implicit
conversions."


On Tue, Jul 8, 2014 at 10:25 AM, Konstantin Kudryavtsev <
kudryavtsev.konstantin@gmail.com> wrote:

> Hi all,
>
> sorry for fooly question, but how can I get PairRDDFunctions RDD? I'm
> doing it to perform leftOuterJoin aftewards
>
> currently I do in this was (it seems incorrect):
> val parRDD = new PairRDDFunctions( oldRdd.map(i => (i.key, i)) )
>
> I guess this constructor is definitely wrong...
>
>
> Thank you,
> Konstantin Kudryavtsev
>

Re: how to convert RDD to PairRDDFunctions ?

[Sean Owen <so...@cloudera.com>]

If your RDD contains pairs, like an RDD[(String,Integer)] or something,
then you get to use the functions in PairRDDFunctions as if they were
declared on RDD.


On Tue, Jul 8, 2014 at 6:25 PM, Konstantin Kudryavtsev <
kudryavtsev.konstantin@gmail.com> wrote:

> Hi all,
>
> sorry for fooly question, but how can I get PairRDDFunctions RDD? I'm
> doing it to perform leftOuterJoin aftewards
>
> currently I do in this was (it seems incorrect):
> val parRDD = new PairRDDFunctions( oldRdd.map(i => (i.key, i)) )
>
> I guess this constructor is definitely wrong...
>
>
> Thank you,
> Konstantin Kudryavtsev
>