You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Yasemin Kaya <go...@gmail.com> on 2015/05/14 10:15:01 UTC
swap tuple
Hi,
I have *JavaPairRDD<String, String> *and I want to *swap tuple._1() to
tuple._2()*. I use *tuple.swap() *but it can't be changed JavaPairRDD in
real. When I print JavaPairRDD, the values are same.
Anyone can help me for that?
Thank you.
Have nice day.
yasemin
--
hiç ender hiç
Re: swap tuple
Posted by Yasemin Kaya <go...@gmail.com>.
I solved my problem right this way.
JavaPairRDD<String, String> swappedPair = pair.mapToPair(
new PairFunction<Tuple2<String, String>, String, String>() {
@Override
public Tuple2<String, String> call(
Tuple2<String, String> item)
throws Exception {
return item.swap();
}
});
2015-05-14 20:42 GMT+03:00 Stephen Carman <sc...@coldlight.com>:
> Yea, I wouldn't try and modify the current since RDDs are suppose to be
> immutable, just create a new one...
>
> val newRdd = oldRdd.map(r => (r._2(), r._1()))
>
> or something of that nature...
>
> Steve
> ------------------------------
> *From:* Evo Eftimov [evo.eftimov@isecc.com]
> *Sent:* Thursday, May 14, 2015 1:24 PM
> *To:* 'Holden Karau'; 'Yasemin Kaya'
> *Cc:* user@spark.apache.org
> *Subject:* RE: swap tuple
>
> Where is the “Tuple” supposed to be in <String, String> - you can
> refer to a “Tuple” if it was e.g. <String, Tuple2<String, String>>
>
>
>
> *From:* holden.karau@gmail.com [mailto:holden.karau@gmail.com] *On Behalf
> Of *Holden Karau
> *Sent:* Thursday, May 14, 2015 5:56 PM
> *To:* Yasemin Kaya
> *Cc:* user@spark.apache.org
> *Subject:* Re: swap tuple
>
>
>
> Can you paste your code? transformations return a new RDD rather than
> modifying an existing one, so if you were to swap the values of the tuple
> using a map you would get back a new RDD and then you would want to try and
> print this new RDD instead of the original one.
>
> On Thursday, May 14, 2015, Yasemin Kaya <go...@gmail.com> wrote:
>
> Hi,
>
>
>
> I have *JavaPairRDD<String, String> *and I want to *swap tuple._1() to
> tuple._2()*. I use *tuple.swap() *but it can't be changed JavaPairRDD in
> real. When I print JavaPairRDD, the values are same.
>
>
>
> Anyone can help me for that?
>
>
>
> Thank you.
>
> Have nice day.
>
>
>
> yasemin
>
>
>
> --
>
> hiç ender hiç
>
>
>
> --
>
> Cell : 425-233-8271
>
> Twitter: https://twitter.com/holdenkarau
>
> Linked In: https://www.linkedin.com/in/holdenkarau
>
>
> This e-mail is intended solely for the above-mentioned recipient and
> it may contain confidential or privileged information. If you have received
> it in error, please notify us immediately and delete the e-mail. You must
> not copy, distribute, disclose or take any action in reliance on it. In
> addition, the contents of an attachment to this e-mail may contain software
> viruses which could damage your own computer system. While ColdLight
> Solutions, LLC has taken every reasonable precaution to minimize this risk,
> we cannot accept liability for any damage which you sustain as a result of
> software viruses. You should perform your own virus checks before opening
> the attachment.
>
--
hiç ender hiç
RE: swap tuple
Posted by Stephen Carman <sc...@coldlight.com>.
Yea, I wouldn't try and modify the current since RDDs are suppose to be immutable, just create a new one...
val newRdd = oldRdd.map(r => (r._2(), r._1()))
or something of that nature...
Steve
________________________________
From: Evo Eftimov [evo.eftimov@isecc.com]
Sent: Thursday, May 14, 2015 1:24 PM
To: 'Holden Karau'; 'Yasemin Kaya'
Cc: user@spark.apache.org
Subject: RE: swap tuple
Where is the “Tuple” supposed to be in <String, String> - you can refer to a “Tuple” if it was e.g. <String, Tuple2<String, String>>
From: holden.karau@gmail.com [mailto:holden.karau@gmail.com] On Behalf Of Holden Karau
Sent: Thursday, May 14, 2015 5:56 PM
To: Yasemin Kaya
Cc: user@spark.apache.org
Subject: Re: swap tuple
Can you paste your code? transformations return a new RDD rather than modifying an existing one, so if you were to swap the values of the tuple using a map you would get back a new RDD and then you would want to try and print this new RDD instead of the original one.
On Thursday, May 14, 2015, Yasemin Kaya <go...@gmail.com>> wrote:
Hi,
I have JavaPairRDD<String, String> and I want to swap tuple._1() to tuple._2(). I use tuple.swap() but it can't be changed JavaPairRDD in real. When I print JavaPairRDD, the values are same.
Anyone can help me for that?
Thank you.
Have nice day.
yasemin
--
hiç ender hiç
--
Cell : 425-233-8271
Twitter: https://twitter.com/holdenkarau
Linked In: https://www.linkedin.com/in/holdenkarau
This e-mail is intended solely for the above-mentioned recipient and it may contain confidential or privileged information. If you have received it in error, please notify us immediately and delete the e-mail. You must not copy, distribute, disclose or take any action in reliance on it. In addition, the contents of an attachment to this e-mail may contain software viruses which could damage your own computer system. While ColdLight Solutions, LLC has taken every reasonable precaution to minimize this risk, we cannot accept liability for any damage which you sustain as a result of software viruses. You should perform your own virus checks before opening the attachment.
RE: swap tuple
Posted by Evo Eftimov <ev...@isecc.com>.
Where is the “Tuple” supposed to be in <String, String> - you can refer to a “Tuple” if it was e.g. <String, Tuple2<String, String>>
From: holden.karau@gmail.com [mailto:holden.karau@gmail.com] On Behalf Of Holden Karau
Sent: Thursday, May 14, 2015 5:56 PM
To: Yasemin Kaya
Cc: user@spark.apache.org
Subject: Re: swap tuple
Can you paste your code? transformations return a new RDD rather than modifying an existing one, so if you were to swap the values of the tuple using a map you would get back a new RDD and then you would want to try and print this new RDD instead of the original one.
On Thursday, May 14, 2015, Yasemin Kaya <go...@gmail.com> wrote:
Hi,
I have JavaPairRDD<String, String> and I want to swap tuple._1() to tuple._2(). I use tuple.swap() but it can't be changed JavaPairRDD in real. When I print JavaPairRDD, the values are same.
Anyone can help me for that?
Thank you.
Have nice day.
yasemin
--
hiç ender hiç
--
Cell : 425-233-8271
Twitter: https://twitter.com/holdenkarau
Linked In: https://www.linkedin.com/in/holdenkarau
Re: swap tuple
Posted by Holden Karau <ho...@pigscanfly.ca>.
Can you paste your code? transformations return a new RDD rather than
modifying an existing one, so if you were to swap the values of the tuple
using a map you would get back a new RDD and then you would want to try and
print this new RDD instead of the original one.
On Thursday, May 14, 2015, Yasemin Kaya <go...@gmail.com> wrote:
> Hi,
>
> I have *JavaPairRDD<String, String> *and I want to *swap tuple._1() to
> tuple._2()*. I use *tuple.swap() *but it can't be changed JavaPairRDD in
> real. When I print JavaPairRDD, the values are same.
>
> Anyone can help me for that?
>
> Thank you.
> Have nice day.
>
> yasemin
>
> --
> hiç ender hiç
>
--
Cell : 425-233-8271
Twitter: https://twitter.com/holdenkarau
Linked In: https://www.linkedin.com/in/holdenkarau