using a generate to collect fields into tuples

[Mat Kelcey <ma...@gmail.com>]

Hi all,

I want to use a generate to munge my data into some tuples.

(This is a contrived example to illustrate my problem)

grunt> dump test
(a,1,8)
(b,2,4)
(c,3,1)

i can use generate to reorder and duplicate fields ok
grunt> test2 = foreach test generate $0,$2,$1,$0;
grunt> dump test2
(a,8,1,a)
(b,4,2,b)
(c,1,3,c)

how can i change the structure to make it the following? (each records
is a tuple with two tuples)
((a,8),(1,a))
((b,4),(2,b))
((c,1),(3,c))

is there is nothing like??
grunt> test2 = foreach test generate TUPLE($0,$2),TUPLE($1,$0);

i must be missing something really simple...

cheers!
mat

Re: using a generate to collect fields into tuples

[Jeff Zhang <zj...@gmail.com>]

Hi Mat,

I don't think Pig has its built-in UDF like TUPLE, but you can write it by
your own, and it's very easy to write an UDF, you can look at here for
reference http://hadoop.apache.org/pig/docs/r0.5.0/udf.html


On Thu, Jan 21, 2010 at 4:58 AM, Mat Kelcey <ma...@gmail.com>wrote:

> Hi all,
>
> I want to use a generate to munge my data into some tuples.
>
> (This is a contrived example to illustrate my problem)
>
> grunt> dump test
> (a,1,8)
> (b,2,4)
> (c,3,1)
>
> i can use generate to reorder and duplicate fields ok
> grunt> test2 = foreach test generate $0,$2,$1,$0;
> grunt> dump test2
> (a,8,1,a)
> (b,4,2,b)
> (c,1,3,c)
>
> how can i change the structure to make it the following? (each records
> is a tuple with two tuples)
> ((a,8),(1,a))
> ((b,4),(2,b))
> ((c,1),(3,c))
>
> is there is nothing like??
> grunt> test2 = foreach test generate TUPLE($0,$2),TUPLE($1,$0);
>
> i must be missing something really simple...
>
> cheers!
> mat
>



-- 
Best Regards

Jeff Zhang

Re: using a generate to collect fields into tuples

[Rekha Joshi <re...@yahoo-inc.com>]

Well, the easiest way would be to pass your input set to your custom UDF exec() and set your own flavor of outputSchema() there.

Cheers,
/R

On 1/21/10 6:28 PM, "Mat Kelcey" <ma...@gmail.com> wrote:

Hi all,

I want to use a generate to munge my data into some tuples.

(This is a contrived example to illustrate my problem)

grunt> dump test
(a,1,8)
(b,2,4)
(c,3,1)

i can use generate to reorder and duplicate fields ok
grunt> test2 = foreach test generate $0,$2,$1,$0;
grunt> dump test2
(a,8,1,a)
(b,4,2,b)
(c,1,3,c)

how can i change the structure to make it the following? (each records
is a tuple with two tuples)
((a,8),(1,a))
((b,4),(2,b))
((c,1),(3,c))

is there is nothing like??
grunt> test2 = foreach test generate TUPLE($0,$2),TUPLE($1,$0);

i must be missing something really simple...

cheers!
mat