You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Yasemin Kaya <go...@gmail.com> on 2015/10/26 17:40:02 UTC
rdd conversion
Hi,
I have *JavaRDD<List<Map<Integer, ArrayList<String>>>>* and I want to
convert every map to pairrdd, i mean
* JavaPairRDD<Integer,ArrayList<String>>. *
There is a loop in list to get the indexed map, when I write code below, it
returns me only one rdd.
JavaPairRDD<Integer,ArrayList<String>> mapToRDD =
IdMapValues.mapToPair(new
PairFunction<List<Map<Integer,ArrayList<String>>>, Integer,
ArrayList<String>>() {
@Override
public Tuple2<Integer, ArrayList<String>> call(
List<Map<Integer, ArrayList<String>>> arg0)
throws Exception {
Tuple2<Integer, ArrayList<String>> t = null;
for(int i=0; i<arg0.size(); ++i){
for (Map.Entry<Integer, ArrayList<String>> entry :arg0.get(i).entrySet()) {
t = new Tuple2 <Integer, ArrayList<String>> (entry.getKey(),
entry.getValue());
}
}
return t;
}
});
As you can see i am using java. Give me some clue .. Thanks.
Best,
yasemin
--
hiç ender hiç
Re: rdd conversion
Posted by Yasemin Kaya <go...@gmail.com>.
But if I put the return inside loop, method still wants me a return
statement.
2015-10-26 19:09 GMT+02:00 Ted Yu <yu...@gmail.com>:
> bq. t = new Tuple2 <Integer, ArrayList<String>> (entry.getKey(),
> entry.getValue());
>
> The return statement is outside the loop.
> That was why you got one RDD.
>
> On Mon, Oct 26, 2015 at 9:40 AM, Yasemin Kaya <go...@gmail.com> wrote:
>
>> Hi,
>>
>> I have *JavaRDD<List<Map<Integer, ArrayList<String>>>>* and I want to
>> convert every map to pairrdd, i mean
>> * JavaPairRDD<Integer,ArrayList<String>>. *
>>
>> There is a loop in list to get the indexed map, when I write code below,
>> it returns me only one rdd.
>>
>> JavaPairRDD<Integer,ArrayList<String>> mapToRDD =
>> IdMapValues.mapToPair(new
>> PairFunction<List<Map<Integer,ArrayList<String>>>, Integer,
>> ArrayList<String>>() {
>>
>> @Override
>> public Tuple2<Integer, ArrayList<String>> call(
>> List<Map<Integer, ArrayList<String>>> arg0)
>> throws Exception {
>> Tuple2<Integer, ArrayList<String>> t = null;
>> for(int i=0; i<arg0.size(); ++i){
>> for (Map.Entry<Integer, ArrayList<String>> entry :arg0.get(i).entrySet())
>> {
>> t = new Tuple2 <Integer, ArrayList<String>> (entry.getKey(),
>> entry.getValue());
>> }
>> }
>>
>> return t;
>> }
>> });
>>
>> As you can see i am using java. Give me some clue .. Thanks.
>>
>> Best,
>> yasemin
>>
>> --
>> hiç ender hiç
>>
>
>
--
hiç ender hiç
Re: rdd conversion
Posted by Ted Yu <yu...@gmail.com>.
bq. t = new Tuple2 <Integer, ArrayList<String>> (entry.getKey(),
entry.getValue());
The return statement is outside the loop.
That was why you got one RDD.
On Mon, Oct 26, 2015 at 9:40 AM, Yasemin Kaya <go...@gmail.com> wrote:
> Hi,
>
> I have *JavaRDD<List<Map<Integer, ArrayList<String>>>>* and I want to
> convert every map to pairrdd, i mean
> * JavaPairRDD<Integer,ArrayList<String>>. *
>
> There is a loop in list to get the indexed map, when I write code below,
> it returns me only one rdd.
>
> JavaPairRDD<Integer,ArrayList<String>> mapToRDD =
> IdMapValues.mapToPair(new
> PairFunction<List<Map<Integer,ArrayList<String>>>, Integer,
> ArrayList<String>>() {
>
> @Override
> public Tuple2<Integer, ArrayList<String>> call(
> List<Map<Integer, ArrayList<String>>> arg0)
> throws Exception {
> Tuple2<Integer, ArrayList<String>> t = null;
> for(int i=0; i<arg0.size(); ++i){
> for (Map.Entry<Integer, ArrayList<String>> entry :arg0.get(i).entrySet()) {
> t = new Tuple2 <Integer, ArrayList<String>> (entry.getKey(),
> entry.getValue());
> }
> }
>
> return t;
> }
> });
>
> As you can see i am using java. Give me some clue .. Thanks.
>
> Best,
> yasemin
>
> --
> hiç ender hiç
>