You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Tom <th...@gmail.com> on 2014/09/10 15:01:23 UTC
JavaPairRDD to JavaPairRDD based
on key
Is it possible to generate a JavaPairRDD<String, Integer> from a
JavaPairRDD<String, String>, where I can also use the key values? I have
looked at for instance mapToPair, but this generates a new K/V pair based on
the original value, and does not give me information about the key.
I need this in the initialization phase, where I have two RDD's with similar
keys, but with different types of values. Generating these is computational
intensive, and if I could use the first list to generate the second, it
would save me a big map/reduce phase.
Thanks!
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/JavaPairRDD-String-Integer-to-JavaPairRDD-String-String-based-on-key-tp13875.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org
Re: JavaPairRDD to JavaPairRDD based
on key
Posted by Sean Owen <so...@cloudera.com>.
So, each key-value pair gets a new value for the original key? you
want mapValues().
On Wed, Sep 10, 2014 at 2:01 PM, Tom <th...@gmail.com> wrote:
> Is it possible to generate a JavaPairRDD<String, Integer> from a
> JavaPairRDD<String, String>, where I can also use the key values? I have
> looked at for instance mapToPair, but this generates a new K/V pair based on
> the original value, and does not give me information about the key.
>
> I need this in the initialization phase, where I have two RDD's with similar
> keys, but with different types of values. Generating these is computational
> intensive, and if I could use the first list to generate the second, it
> would save me a big map/reduce phase.
>
> Thanks!
>
>
>
> --
> View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/JavaPairRDD-String-Integer-to-JavaPairRDD-String-String-based-on-key-tp13875.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org