You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by Divya Gehlot <di...@gmail.com> on 2016/01/25 10:04:21 UTC
getting error in passing tuple to GroupBy dynamically
Hi,
I have two files
Group_condition.txt
Colun1|Y
Column2|N
Column3|Y
Load_cfl = LOAD '/user/hdfs/file.txt' USING PigStorage('|') as
(code:chararray,book_code:int,currency_code:chararray,start_date:datetime,end_date:datetime,type:chararray,amount:double
);
Load_GroupBy = LOAD '/user/hdfs/Groupby_Condition.txt' USING
PigStorage('|') as (groupBy:chararray , condition :chararray);
filter_col = FILTER Load_GroupBy BY (condition == 'Y');
flatn_col = FOREACH filter_col GENERATE FLATTEN(groupBy);
tuple_col = FOREACH flatn_col GENERATE TOTUPLE(groupBy);
cfl_grp = GROUP Load_cfl BY (flatn_col);
getting error :
> Invalid scalar projection: flatn_col : A column needs to be projected from
> a relation for it to be used as a scalar
What am I missing in above ?
Would really appreciate the help.
Thanks,
Divya
Re: getting error in passing tuple to GroupBy dynamically
Posted by Prashant Kommireddi <pr...@apache.org>.
Load_cfl = LOAD '/user/hdfs/file.txt' USING PigStorage('|') as
(code:chararray,book_code:int,currency_code:chararray,start_
date:datetime,end_date:datetime,type:chararray,amount:double
);
.......
cfl_grp = GROUP Load_cfl BY (*flatn_col*);
I don't see a column "*flatn_col" *in cfl_grp. Typo?
On Mon, Jan 25, 2016 at 6:12 PM, Divya Gehlot <di...@gmail.com>
wrote:
> Hi,
> The file Groupby_condition.txt looks like below where data is seperated by
> |
>
> Column1|Y
> Column2|N
> Column3|Y
>
> On 26 January 2016 at 07:17, Prashant Kommireddi <pr...@apache.org>
> wrote:
>
>> You are flattening a scalar column "groupBy". What does the file
>> "GroupBy_Condition" look like? I'm guessing you want to load that as a
>> bag/tuple before flattening, but you're reading as a chararray instead.
>>
>> On Monday, January 25, 2016, Divya Gehlot <di...@gmail.com>
>> wrote:
>>
>> > Hi,
>> > I have two files
>> > Group_condition.txt
>> > Colun1|Y
>> > Column2|N
>> > Column3|Y
>> >
>> > Load_cfl = LOAD '/user/hdfs/file.txt' USING PigStorage('|') as
>> >
>> >
>> (code:chararray,book_code:int,currency_code:chararray,start_date:datetime,end_date:datetime,type:chararray,amount:double
>> > );
>> > Load_GroupBy = LOAD '/user/hdfs/Groupby_Condition.txt' USING
>> > PigStorage('|') as (groupBy:chararray , condition :chararray);
>> > filter_col = FILTER Load_GroupBy BY (condition == 'Y');
>> > flatn_col = FOREACH filter_col GENERATE FLATTEN(groupBy);
>> > tuple_col = FOREACH flatn_col GENERATE TOTUPLE(groupBy);
>> > cfl_grp = GROUP Load_cfl BY (flatn_col);
>> >
>> > getting error :
>> >
>> > > Invalid scalar projection: flatn_col : A column needs to be projected
>> > from
>> > > a relation for it to be used as a scalar
>> >
>> >
>> >
>> > What am I missing in above ?
>> > Would really appreciate the help.
>> >
>> > Thanks,
>> > Divya
>> >
>>
>
>
Re: getting error in passing tuple to GroupBy dynamically
Posted by Divya Gehlot <di...@gmail.com>.
Hi,
The file Groupby_condition.txt looks like below where data is seperated by |
Column1|Y
Column2|N
Column3|Y
On 26 January 2016 at 07:17, Prashant Kommireddi <pr...@apache.org>
wrote:
> You are flattening a scalar column "groupBy". What does the file
> "GroupBy_Condition" look like? I'm guessing you want to load that as a
> bag/tuple before flattening, but you're reading as a chararray instead.
>
> On Monday, January 25, 2016, Divya Gehlot <di...@gmail.com> wrote:
>
> > Hi,
> > I have two files
> > Group_condition.txt
> > Colun1|Y
> > Column2|N
> > Column3|Y
> >
> > Load_cfl = LOAD '/user/hdfs/file.txt' USING PigStorage('|') as
> >
> >
> (code:chararray,book_code:int,currency_code:chararray,start_date:datetime,end_date:datetime,type:chararray,amount:double
> > );
> > Load_GroupBy = LOAD '/user/hdfs/Groupby_Condition.txt' USING
> > PigStorage('|') as (groupBy:chararray , condition :chararray);
> > filter_col = FILTER Load_GroupBy BY (condition == 'Y');
> > flatn_col = FOREACH filter_col GENERATE FLATTEN(groupBy);
> > tuple_col = FOREACH flatn_col GENERATE TOTUPLE(groupBy);
> > cfl_grp = GROUP Load_cfl BY (flatn_col);
> >
> > getting error :
> >
> > > Invalid scalar projection: flatn_col : A column needs to be projected
> > from
> > > a relation for it to be used as a scalar
> >
> >
> >
> > What am I missing in above ?
> > Would really appreciate the help.
> >
> > Thanks,
> > Divya
> >
>
Re: getting error in passing tuple to GroupBy dynamically
Posted by Prashant Kommireddi <pr...@apache.org>.
You are flattening a scalar column "groupBy". What does the file
"GroupBy_Condition" look like? I'm guessing you want to load that as a
bag/tuple before flattening, but you're reading as a chararray instead.
On Monday, January 25, 2016, Divya Gehlot <di...@gmail.com> wrote:
> Hi,
> I have two files
> Group_condition.txt
> Colun1|Y
> Column2|N
> Column3|Y
>
> Load_cfl = LOAD '/user/hdfs/file.txt' USING PigStorage('|') as
>
> (code:chararray,book_code:int,currency_code:chararray,start_date:datetime,end_date:datetime,type:chararray,amount:double
> );
> Load_GroupBy = LOAD '/user/hdfs/Groupby_Condition.txt' USING
> PigStorage('|') as (groupBy:chararray , condition :chararray);
> filter_col = FILTER Load_GroupBy BY (condition == 'Y');
> flatn_col = FOREACH filter_col GENERATE FLATTEN(groupBy);
> tuple_col = FOREACH flatn_col GENERATE TOTUPLE(groupBy);
> cfl_grp = GROUP Load_cfl BY (flatn_col);
>
> getting error :
>
> > Invalid scalar projection: flatn_col : A column needs to be projected
> from
> > a relation for it to be used as a scalar
>
>
>
> What am I missing in above ?
> Would really appreciate the help.
>
> Thanks,
> Divya
>