You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@pig.apache.org by Walter Chang <we...@gmail.com> on 2011/10/03 09:22:56 UTC

pig commands help

Hi ,

assuming i have  table with the following sample (two column separated by
space):

a 1,2,3
b 4,5,6

I would like to covert it to

a 1
a 2
a 3
b 4
b 5
b 6

basically split the second column and use the elements to generate separate
rows but keep the first column intact.

What's the best way i should achieve this ?

Thanks,

Weide

RE: pig commands help

Posted by Marek Miglinski <mm...@seven.com>.

Hey,

First UNION, then SORT.



Sincerely,
Marek M.
________________________________________
From: Walter Chang [weidezhang2007@gmail.com]
Sent: Monday, October 03, 2011 10:22 AM
To: user@pig.apache.org
Cc: weidezhang2007@gmail.com
Subject: pig commands help

Hi ,

assuming i have  table with the following sample (two column separated by
space):

a 1,2,3
b 4,5,6

I would like to covert it to

a 1
a 2
a 3
b 4
b 5
b 6

basically split the second column and use the elements to generate separate
rows but keep the first column intact.

What's the best way i should achieve this ?

Thanks,

Weide

Re: pig commands help

Posted by Ayon Sinha <ay...@yahoo.com>.

I tried the same thing a few days ago:My data looks like
6202445(2284,11096,2931,11168)
6202446(83258,738,10215,12987)
6202447(83258,738,10215,12987)
6202448(1001,1284,11550)
6202449(1560,752,13505,12876,2906)
6202450(1162,11010,13018)
6202451(2784,1729,12987)
6202452(383,12912)
6202453(1201,12987,1202)
6202454(1632,1633,297,10254)

I'm trying to flatten the list

a = load '/home/ayon/test' using PigStorage('\u0001') as (location_id:long, cats:tuple());
a = foreach a generate location_id, flatten(cats);
dump a;

 
Apparently, it works on Pig 0.8.1 and pig 0.9. I have pig 0.8 and it didn't work for me. I had to make the text file look like a bag of tuples.
 

-Ayon
See My Photos on Flickr
Also check out my Blog for answers to commonly asked questions.



________________________________
From: Walter Chang <we...@gmail.com>
To: user@pig.apache.org
Cc: weidezhang2007@gmail.com
Sent: Monday, October 3, 2011 12:22 AM
Subject: pig commands help

Hi ,

assuming i have  table with the following sample (two column separated by
space):

a 1,2,3
b 4,5,6

I would like to covert it to

a 1
a 2
a 3
b 4
b 5
b 6

basically split the second column and use the elements to generate separate
rows but keep the first column intact.

What's the best way i should achieve this ?

Thanks,

Weide