You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by Simha G <ha...@gmail.com> on 2015/08/27 23:35:12 UTC
Flatten keyword in Pig
Hi Team,
I have a source data like below,
{(03-07-2011),(03-07-2011)},{(Jackson),(Jackson)},{(Mississippi),(Mississippi)},{(Team
Sports),(Team Sports)},{(credit),(credit)}
{(11-14-2011),(11-14-2011)},{(Des Moines),(Des
Moines)},{(Iowa),(Iowa)},{(Outdoor Play Equipment),(Outdoor Play
Equipment)},{(credit),(credit)}
And i want the data in tupples, I tried like below
mydata = load '/user/flatendata' USING PigStorage(',') as (id:chararray,
name:chararray, lastname:chararray, hobby:chararray, city:chararray);
flatted = foreach mydata genereate flatten(id), flatten(name),
flatten(lastname), flatten(hobby);
but i am getting below output ,
({(03-07-2011),(03-07-2011)},{(Jackson),(Jackson)})
({(11-14-2011),(11-14-2011)},{(Des Moines),(Des Moines)})
Could you please suggest me how to get the below flatten data.
(03-07-2011,03-07-2011,Jackson,Jackson,Mississippi,Mississippi,Team
Sports,Team Sports,credit,credit)
(11-14-2011,11-14-2011,Des Moines,Des Moines,Iowa,Iowa,Outdoor Play
Equipment,Outdoor Play Equipment,credit,credit)
Thanks,
Narasimha
Re: Flatten keyword in Pig
Posted by Rohini Palaniswamy <ro...@gmail.com>.
You can do FLATTEN + TOTUPLE() UDF
On Fri, Aug 28, 2015 at 11:20 AM, Arvind S <ar...@gmail.com> wrote:
> read as a singe string field and use REPLACE .. you will have to use it 4
> times ..one for each of (,),{ & } ..
>
> *Cheers !!*
> Arvind
>
> On Fri, Aug 28, 2015 at 7:29 PM, Simha G <ha...@gmail.com> wrote:
>
> > Hi Aravind,
> >
> > Thanks for your prompt reply. can we do it without using UDF's?
> >
> > Thanks,
> > Narasimha
> >
> > On Fri, Aug 28, 2015 at 10:53 AM, Arvind S <ar...@gmail.com>
> wrote:
> >
> > > see if "BagToString" helps you
> > >
> > >
> >
> https://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/builtin/BagToString.java
> > >
> > >
> > > *Cheers !!*
> > > Arvind
> > >
> > > On Fri, Aug 28, 2015 at 3:05 AM, Simha G <ha...@gmail.com>
> wrote:
> > >
> > > > Hi Team,
> > > >
> > > > I have a source data like below,
> > > >
> > > >
> > > >
> > >
> >
> {(03-07-2011),(03-07-2011)},{(Jackson),(Jackson)},{(Mississippi),(Mississippi)},{(Team
> > > > Sports),(Team Sports)},{(credit),(credit)}
> > > > {(11-14-2011),(11-14-2011)},{(Des Moines),(Des
> > > > Moines)},{(Iowa),(Iowa)},{(Outdoor Play Equipment),(Outdoor Play
> > > > Equipment)},{(credit),(credit)}
> > > >
> > > > And i want the data in tupples, I tried like below
> > > >
> > > > mydata = load '/user/flatendata' USING PigStorage(',') as
> > (id:chararray,
> > > > name:chararray, lastname:chararray, hobby:chararray, city:chararray);
> > > >
> > > > flatted = foreach mydata genereate flatten(id), flatten(name),
> > > > flatten(lastname), flatten(hobby);
> > > >
> > > > but i am getting below output ,
> > > >
> > > > ({(03-07-2011),(03-07-2011)},{(Jackson),(Jackson)})
> > > > ({(11-14-2011),(11-14-2011)},{(Des Moines),(Des Moines)})
> > > >
> > > >
> > > > Could you please suggest me how to get the below flatten data.
> > > >
> > > > (03-07-2011,03-07-2011,Jackson,Jackson,Mississippi,Mississippi,Team
> > > > Sports,Team Sports,credit,credit)
> > > > (11-14-2011,11-14-2011,Des Moines,Des Moines,Iowa,Iowa,Outdoor Play
> > > > Equipment,Outdoor Play Equipment,credit,credit)
> > > >
> > > >
> > > > Thanks,
> > > > Narasimha
> > > >
> > >
> >
>
Re: Flatten keyword in Pig
Posted by Arvind S <ar...@gmail.com>.
read as a singe string field and use REPLACE .. you will have to use it 4
times ..one for each of (,),{ & } ..
*Cheers !!*
Arvind
On Fri, Aug 28, 2015 at 7:29 PM, Simha G <ha...@gmail.com> wrote:
> Hi Aravind,
>
> Thanks for your prompt reply. can we do it without using UDF's?
>
> Thanks,
> Narasimha
>
> On Fri, Aug 28, 2015 at 10:53 AM, Arvind S <ar...@gmail.com> wrote:
>
> > see if "BagToString" helps you
> >
> >
> https://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/builtin/BagToString.java
> >
> >
> > *Cheers !!*
> > Arvind
> >
> > On Fri, Aug 28, 2015 at 3:05 AM, Simha G <ha...@gmail.com> wrote:
> >
> > > Hi Team,
> > >
> > > I have a source data like below,
> > >
> > >
> > >
> >
> {(03-07-2011),(03-07-2011)},{(Jackson),(Jackson)},{(Mississippi),(Mississippi)},{(Team
> > > Sports),(Team Sports)},{(credit),(credit)}
> > > {(11-14-2011),(11-14-2011)},{(Des Moines),(Des
> > > Moines)},{(Iowa),(Iowa)},{(Outdoor Play Equipment),(Outdoor Play
> > > Equipment)},{(credit),(credit)}
> > >
> > > And i want the data in tupples, I tried like below
> > >
> > > mydata = load '/user/flatendata' USING PigStorage(',') as
> (id:chararray,
> > > name:chararray, lastname:chararray, hobby:chararray, city:chararray);
> > >
> > > flatted = foreach mydata genereate flatten(id), flatten(name),
> > > flatten(lastname), flatten(hobby);
> > >
> > > but i am getting below output ,
> > >
> > > ({(03-07-2011),(03-07-2011)},{(Jackson),(Jackson)})
> > > ({(11-14-2011),(11-14-2011)},{(Des Moines),(Des Moines)})
> > >
> > >
> > > Could you please suggest me how to get the below flatten data.
> > >
> > > (03-07-2011,03-07-2011,Jackson,Jackson,Mississippi,Mississippi,Team
> > > Sports,Team Sports,credit,credit)
> > > (11-14-2011,11-14-2011,Des Moines,Des Moines,Iowa,Iowa,Outdoor Play
> > > Equipment,Outdoor Play Equipment,credit,credit)
> > >
> > >
> > > Thanks,
> > > Narasimha
> > >
> >
>
Re: Flatten keyword in Pig
Posted by Simha G <ha...@gmail.com>.
Hi Aravind,
Thanks for your prompt reply. can we do it without using UDF's?
Thanks,
Narasimha
On Fri, Aug 28, 2015 at 10:53 AM, Arvind S <ar...@gmail.com> wrote:
> see if "BagToString" helps you
>
> https://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/builtin/BagToString.java
>
>
> *Cheers !!*
> Arvind
>
> On Fri, Aug 28, 2015 at 3:05 AM, Simha G <ha...@gmail.com> wrote:
>
> > Hi Team,
> >
> > I have a source data like below,
> >
> >
> >
> {(03-07-2011),(03-07-2011)},{(Jackson),(Jackson)},{(Mississippi),(Mississippi)},{(Team
> > Sports),(Team Sports)},{(credit),(credit)}
> > {(11-14-2011),(11-14-2011)},{(Des Moines),(Des
> > Moines)},{(Iowa),(Iowa)},{(Outdoor Play Equipment),(Outdoor Play
> > Equipment)},{(credit),(credit)}
> >
> > And i want the data in tupples, I tried like below
> >
> > mydata = load '/user/flatendata' USING PigStorage(',') as (id:chararray,
> > name:chararray, lastname:chararray, hobby:chararray, city:chararray);
> >
> > flatted = foreach mydata genereate flatten(id), flatten(name),
> > flatten(lastname), flatten(hobby);
> >
> > but i am getting below output ,
> >
> > ({(03-07-2011),(03-07-2011)},{(Jackson),(Jackson)})
> > ({(11-14-2011),(11-14-2011)},{(Des Moines),(Des Moines)})
> >
> >
> > Could you please suggest me how to get the below flatten data.
> >
> > (03-07-2011,03-07-2011,Jackson,Jackson,Mississippi,Mississippi,Team
> > Sports,Team Sports,credit,credit)
> > (11-14-2011,11-14-2011,Des Moines,Des Moines,Iowa,Iowa,Outdoor Play
> > Equipment,Outdoor Play Equipment,credit,credit)
> >
> >
> > Thanks,
> > Narasimha
> >
>
Re: Flatten keyword in Pig
Posted by Arvind S <ar...@gmail.com>.
see if "BagToString" helps you
https://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/builtin/BagToString.java
*Cheers !!*
Arvind
On Fri, Aug 28, 2015 at 3:05 AM, Simha G <ha...@gmail.com> wrote:
> Hi Team,
>
> I have a source data like below,
>
>
> {(03-07-2011),(03-07-2011)},{(Jackson),(Jackson)},{(Mississippi),(Mississippi)},{(Team
> Sports),(Team Sports)},{(credit),(credit)}
> {(11-14-2011),(11-14-2011)},{(Des Moines),(Des
> Moines)},{(Iowa),(Iowa)},{(Outdoor Play Equipment),(Outdoor Play
> Equipment)},{(credit),(credit)}
>
> And i want the data in tupples, I tried like below
>
> mydata = load '/user/flatendata' USING PigStorage(',') as (id:chararray,
> name:chararray, lastname:chararray, hobby:chararray, city:chararray);
>
> flatted = foreach mydata genereate flatten(id), flatten(name),
> flatten(lastname), flatten(hobby);
>
> but i am getting below output ,
>
> ({(03-07-2011),(03-07-2011)},{(Jackson),(Jackson)})
> ({(11-14-2011),(11-14-2011)},{(Des Moines),(Des Moines)})
>
>
> Could you please suggest me how to get the below flatten data.
>
> (03-07-2011,03-07-2011,Jackson,Jackson,Mississippi,Mississippi,Team
> Sports,Team Sports,credit,credit)
> (11-14-2011,11-14-2011,Des Moines,Des Moines,Iowa,Iowa,Outdoor Play
> Equipment,Outdoor Play Equipment,credit,credit)
>
>
> Thanks,
> Narasimha
>