You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Deep Pradhan <pr...@gmail.com> on 2014/08/26 06:55:40 UTC
Pair RDD
Hi,
I have an input file of a graph in the format <source_node dest_node>
When I use sc.textFile, it will change the entire text file into an RDD.
How can I transform the file into key, value pair and then eventually into
paired RDDs.
Thank You
Re: Pair RDD
Posted by Yanbo Liang <ya...@gmail.com>.
val node = textFile.map(line => {
val fileds = line.split("\\s+")
(fileds(1),fileds(2))
})
then you can manipulate node RDD with PairRDD function.
2014-08-26 12:55 GMT+08:00 Deep Pradhan <pr...@gmail.com>:
> Hi,
> I have an input file of a graph in the format <source_node dest_node>
> When I use sc.textFile, it will change the entire text file into an RDD.
> How can I transform the file into key, value pair and then eventually into
> paired RDDs.
> Thank You
>