You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by Irfan Mohammed <ir...@gmail.com> on 2009/08/16 16:41:50 UTC
Question on Traversing Tuple of Tuples
Hi,
Thanks for the excellent tool.
I want to traverse a tuple of tuples in pig.
1<tab>((t1,1),(t2,2),(t3,3))
2<tab>((t3,3),(t4,4))
I defined the load schema as follows r1 = LOAD
'file:///home/irfan/qw/sql/pig/tuple_test.txt' using PigStorage() as
(c1:chararray, t01:tuple(t11:tuple(k1:chararray, v1:chararray)));
I want to be able output something like
t1,1
t2,1
t3,2
t4,1
Everytime I only get the first inner tuple (t11) of the outer tuple (t01)
Please see the attached input data and the sample pig script trying to
traverse the tuple of tuples.
Thanks,
Irfan
Re: Question on Traversing Tuple of Tuples
Posted by Irfan Mohammed <ir...@gmail.com>.
Thank you. That helped.
zhang jianfeng wrote:
> Hi Irfan,
>
> Flatten is used for bag, you can change your input data to :
>
>
> 1 {(t1,1),(t2,2),(t3,3)}
> 2 {(t3,3),(t4,4)}
>
> ------------------------------------------------------------------------------------
> And change your script to :
>
>
> r1 = LOAD 'data/input/a.txt' using PigStorage() as (c1:chararray,
> t01:bag{t11:tuple(k1:chararray, v1:chararray)});
>
> --dump r1;
> --describe r1;
>
> r2 = foreach r1 generate flatten(t01);
>
> dump r2;
> describe r2;
>
> ------------------------------------------------------------------------------------
>
> Then you will get what you expected :
>
> (t1,1)
> (t2,2)
> (t3,3)
> (t3,3)
> (t4,4)
>
>
>
> Jeff zhang
>
>
> On Sun, Aug 16, 2009 at 7:41 AM, Irfan Mohammed <ir...@gmail.com> wrote:
>
>
>> Hi,
>> Thanks for the excellent tool.
>>
>> I want to traverse a tuple of tuples in pig.
>>
>> 1<tab>((t1,1),(t2,2),(t3,3))
>> 2<tab>((t3,3),(t4,4))
>>
>> I defined the load schema as follows r1 = LOAD
>> 'file:///home/irfan/qw/sql/pig/tuple_test.txt' using PigStorage() as
>> (c1:chararray, t01:tuple(t11:tuple(k1:chararray, v1:chararray)));
>>
>> I want to be able output something like
>>
>> t1,1
>> t2,1
>> t3,2
>> t4,1
>>
>> Everytime I only get the first inner tuple (t11) of the outer tuple (t01)
>>
>> Please see the attached input data and the sample pig script trying to
>> traverse the tuple of tuples.
>>
>> Thanks,
>> Irfan
>>
>> 1 ((t1,1),(t2,2),(t3,3))
>> 2 ((t3,3),(t4,4))
>>
>> r1 = LOAD 'file:///home/irfan/qw/sql/pig/tuple_test.txt' using PigStorage()
>> as (c1:chararray, t01:tuple(t11:tuple(k1:chararray, v1:chararray)));
>>
>> dump r1;
>> describe r1;
>>
>> r2 = foreach r1 generate flatten(t01.t11);
>>
>> dump r2;
>> describe r2;
>>
>> /*
>> r3 = foreach r2 generate group.t11;
>>
>> dump r3;
>> describe r3;
>>
>> r4 = group r3 by t02;
>>
>> dump r4;
>> describe r4;
>>
>> r5 = foreach r4 {
>> generate $0;
>> }
>>
>> dump r5;
>> describe r5;
>> */
>>
>> -- store r3 into 'file:///home/irfan/qw/sql/pig/tuple_test_output.txt'
>> using PigStorage(',');
>>
>>
>>
>>
>
>
Re: Question on Traversing Tuple of Tuples
Posted by zhang jianfeng <zj...@gmail.com>.
Hi Irfan,
Flatten is used for bag, you can change your input data to :
1 {(t1,1),(t2,2),(t3,3)}
2 {(t3,3),(t4,4)}
------------------------------------------------------------------------------------
And change your script to :
r1 = LOAD 'data/input/a.txt' using PigStorage() as (c1:chararray,
t01:bag{t11:tuple(k1:chararray, v1:chararray)});
--dump r1;
--describe r1;
r2 = foreach r1 generate flatten(t01);
dump r2;
describe r2;
------------------------------------------------------------------------------------
Then you will get what you expected :
(t1,1)
(t2,2)
(t3,3)
(t3,3)
(t4,4)
Jeff zhang
On Sun, Aug 16, 2009 at 7:41 AM, Irfan Mohammed <ir...@gmail.com> wrote:
> Hi,
> Thanks for the excellent tool.
>
> I want to traverse a tuple of tuples in pig.
>
> 1<tab>((t1,1),(t2,2),(t3,3))
> 2<tab>((t3,3),(t4,4))
>
> I defined the load schema as follows r1 = LOAD
> 'file:///home/irfan/qw/sql/pig/tuple_test.txt' using PigStorage() as
> (c1:chararray, t01:tuple(t11:tuple(k1:chararray, v1:chararray)));
>
> I want to be able output something like
>
> t1,1
> t2,1
> t3,2
> t4,1
>
> Everytime I only get the first inner tuple (t11) of the outer tuple (t01)
>
> Please see the attached input data and the sample pig script trying to
> traverse the tuple of tuples.
>
> Thanks,
> Irfan
>
> 1 ((t1,1),(t2,2),(t3,3))
> 2 ((t3,3),(t4,4))
>
> r1 = LOAD 'file:///home/irfan/qw/sql/pig/tuple_test.txt' using PigStorage()
> as (c1:chararray, t01:tuple(t11:tuple(k1:chararray, v1:chararray)));
>
> dump r1;
> describe r1;
>
> r2 = foreach r1 generate flatten(t01.t11);
>
> dump r2;
> describe r2;
>
> /*
> r3 = foreach r2 generate group.t11;
>
> dump r3;
> describe r3;
>
> r4 = group r3 by t02;
>
> dump r4;
> describe r4;
>
> r5 = foreach r4 {
> generate $0;
> }
>
> dump r5;
> describe r5;
> */
>
> -- store r3 into 'file:///home/irfan/qw/sql/pig/tuple_test_output.txt'
> using PigStorage(',');
>
>
>