You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by Josh Devins <in...@joshdevins.net> on 2010/10/08 18:13:05 UTC

Converting an inner bag

I have a simple schema that contains an inner bag. What I need to
essentially do is that for each tuple in the inner bag, I need to create a
new tuple in a new outer bag. This is easier shown than explained! Consider
the following schema and data:

DESCRIBE A;
A: {id: chararray, items: {item: chararray}}

DUMP A;
(a1, {(i1), (i2), (i3)})
(a2, {(i1), (i2), (i3)})

This needs to turn into some relation, I that looks like:

DESCRIBE I;
I: {id: chararray, item: chararray}

DUMP I;
(a1, i1)
(a1, i2)
(a1, i3)
(a2, i1)
(a2, i2)
(a2, i3)

...so that I can then do a join with another relation X against the item
field from I.

Any ideas? Hope that description made sense.

Many thanks,

Josh

Re: Converting an inner bag

Posted by Josh Devins <in...@joshdevins.net>.
Crap, of course :)

Many thanks, that did the job.

Cheers,

Josh


On 8 October 2010 19:12, Mehmet Tepedelenlioglu <me...@sbcglobal.net>wrote:

> I= foreach A generate group, flatten(items);
>
> I believe that should do it.
>
> On Oct 8, 2010, at 9:13 AM, Josh Devins wrote:
>
> > I have a simple schema that contains an inner bag. What I need to
> > essentially do is that for each tuple in the inner bag, I need to create
> a
> > new tuple in a new outer bag. This is easier shown than explained!
> Consider
> > the following schema and data:
> >
> > DESCRIBE A;
> > A: {id: chararray, items: {item: chararray}}
> >
> > DUMP A;
> > (a1, {(i1), (i2), (i3)})
> > (a2, {(i1), (i2), (i3)})
> >
> > This needs to turn into some relation, I that looks like:
> >
> > DESCRIBE I;
> > I: {id: chararray, item: chararray}
> >
> > DUMP I;
> > (a1, i1)
> > (a1, i2)
> > (a1, i3)
> > (a2, i1)
> > (a2, i2)
> > (a2, i3)
> >
> > ...so that I can then do a join with another relation X against the item
> > field from I.
> >
> > Any ideas? Hope that description made sense.
> >
> > Many thanks,
> >
> > Josh
>
>

Re: Converting an inner bag

Posted by Mehmet Tepedelenlioglu <me...@sbcglobal.net>.
I= foreach A generate group, flatten(items);

I believe that should do it.

On Oct 8, 2010, at 9:13 AM, Josh Devins wrote:

> I have a simple schema that contains an inner bag. What I need to
> essentially do is that for each tuple in the inner bag, I need to create a
> new tuple in a new outer bag. This is easier shown than explained! Consider
> the following schema and data:
> 
> DESCRIBE A;
> A: {id: chararray, items: {item: chararray}}
> 
> DUMP A;
> (a1, {(i1), (i2), (i3)})
> (a2, {(i1), (i2), (i3)})
> 
> This needs to turn into some relation, I that looks like:
> 
> DESCRIBE I;
> I: {id: chararray, item: chararray}
> 
> DUMP I;
> (a1, i1)
> (a1, i2)
> (a1, i3)
> (a2, i1)
> (a2, i2)
> (a2, i3)
> 
> ...so that I can then do a join with another relation X against the item
> field from I.
> 
> Any ideas? Hope that description made sense.
> 
> Many thanks,
> 
> Josh